List of commitTask() Examples

Examples of commitTask()

org.apache.hadoop.mapreduce.OutputCommitter.commitTask()

To promote the task's temporary output to final output location. If {@link #needsTaskCommit(TaskAttemptContext)} returns true and thistask is the task that the AM determines finished first, this method is called to commit an individual task's output. This is to mark that tasks output as complete, as {@link #commitJob(JobContext)} will also be called later on if the entire job finished successfully. This is called from a task's process. This may be called multiple times for the same task, but different task attempts. It should be very rare for this to be called multiple times and requires odd networking failures to make this happen. In the future the Hadoop framework may eliminate this race. @param taskContext Context of the task whose output is being written. @throws IOException if commit is not successful.

org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask()

Move the files from the work directory to the job output directory @param context the task context

        }
        rw.close(tao);
        
        FileOutputCommitter cm = (FileOutputCommitter) outputFormat.getOutputCommitter(tao);
        if (cm.needsTaskCommit(tao)) {
          cm.commitTask(tao);
          cm.cleanupJob(jobContext);
        } else {
          cm.cleanupJob(jobContext);
          tryAgain = true;
        }

All source code are property of their respective owners. Java is a trademark of Sun Microsystems, Inc and owned by ORACLE Inc. Contact coftware#gmail.com.

Examples of commitTask()

Examples of org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.commitTask()