TaskInProgress (Hadoop 0.20.2-cdh3u3 API)
Date: 2012-03-20

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.hadoop.mapred
Class TaskInProgress

    java.lang.Object
      org.apache.hadoop.mapred.TaskInProgress

    public class TaskInProgress

extends Object

TaskInProgress maintains all the info needed for a Task in the lifetime of its owning Job. A given Task might be speculatively executed or reexecuted, so we need a level of indirection above the running-id itself.
A given TaskInProgress contains multiple taskids, 0 or more of which might be executing at any one time. (That's what allows speculative execution.) A taskid is now *never* recycled. A TIP allocates enough taskids to account for all the speculation and failures it will ever have to handle. Once those are up, the TIP is dead. **************************************************************

Field Summary
`static org.apache.commons.logging.Log` | `LOG`

Constructor Summary
`TaskInProgress(JobID jobid, String jobFile, int numMaps, int partition, JobTracker jobtracker, JobConf conf, JobInProgress job, int numSlotsRequired)` Constructor for ReduceTask
`TaskInProgress(JobID jobid, String jobFile, JobSplit.TaskSplitMetaInfo split, JobTracker jobtracker, JobConf conf, JobInProgress job, int partition, int numSlotsRequired)` Constructor for MapTask

Method Summary
`void` | `addDiagnosticInfo(TaskAttemptID taskId, String diagInfo)` Save diagnostic information for a given task.
`Task` | `addRunningTask(TaskAttemptID taskid, String taskTracker)`
`Task` | `addRunningTask(TaskAttemptID taskid, String taskTracker, boolean taskCleanup)` Adds a previously running task to this tip.
`void` | `completed(TaskAttemptID taskid)` Indicate that one of the taskids in this TaskInProgress has successfully completed!
`void` | `doCommit(TaskAttemptID taskid)` Commit this task attempt for the tip.
`TaskReport` | `generateSingleReport()` Creates a "status report" for this task.
`Counters` | `getCounters()` Get the task's counters
`List<String>` | `getDiagnosticInfo(TaskAttemptID taskId)` Get the diagnostic messages for a given task within this tip.
`long` | `getExecFinishTime()` Return the exec finish time
`long` | `getExecStartTime()` Return the exec start time
`int` | `getIdWithinJob()` Get the id of this map or reduce task.
`JobInProgress` | `getJob()` Return the parent job
`long` | `getMapInputSize()`
`int` | `getNumberOfFailedMachines()` Get the number of machines where this task has failed.
`double` | `getProgress()` Get the overall progress (from 0 to 1.0) for this TIP
`String[]` | `getSplitLocations()` Get the split locations
`String` | `getSplitNodes()` Gets the Node list of input split locations sorted in rack order.
`long` | `getStartTime()` Return the start time
`int` | `getSuccessEventNumber()` Get the event number that was raised for this tip
`TaskStatus` | `getTaskStatus(TaskAttemptID taskid)` Get the status of the specified task
`TaskStatus[]` | `getTaskStatuses()` Get the Status of the tasks managed by this TIP
`Task` | `getTaskToRun(String taskTracker)` Return a Task that can be sent to a TaskTracker for execution.
`TaskID` | `getTIPId()` Return an ID for this task, not its component taskid-threads
`boolean` | `hasFailedOnMachine(String trackerHost)` Has this task already failed on this machine?
`boolean` | `hasRunOnMachine(String trackerHost, String trackerName)` Was this task ever scheduled to run on this machine?
`int` | `idWithinJob()` Return the index of the tip within the job, so "task_200707121733_1313_0002_m_012345" would return 12345;
`void` | `incompleteSubTask(TaskAttemptID taskid, JobStatus jobStatus)` Indicate that one of the taskids in this TaskInProgress has failed.
`boolean` | `isCommitPending(TaskAttemptID taskId)`
`boolean` | `isComplete()` Is this tip complete?
`boolean` | `isComplete(TaskAttemptID taskid)` Is the given taskid the one that took this tip to completion?
`boolean` | `isFailed()` Is the tip a failure?
`boolean` | `isFirstAttempt(TaskAttemptID taskId)` Is the Task associated with taskid is the first attempt of the tip?
`boolean` | `isJobCleanupTask()`
`boolean` | `isJobSetupTask()`
`boolean` | `isMapTask()` Whether this is a map task
`boolean` | `isOnlyCommitPending()`
`boolean` | `isRunning()` Is this tip currently running any tasks?
`void` | `kill()` The TIP's been ordered kill()ed.
`int` | `numKilledTasks()` Number of times the TaskInProgress has been killed by the framework.
`int` | `numTaskFailures()` Number of times the TaskInProgress has failed.
`void` | `setExecFinishTime(long finishTime)` Set the exec finish time
`void` | `setExecStartTime(long startTime)` Set the exec start time
`void` | `setJobCleanupTask()`
`void` | `setJobSetupTask()`
`void` | `setSuccessEventNumber(int eventNumber)` Set the event number that was raised for this tip
`boolean` | `shouldClose(TaskAttemptID taskid)` Returns whether a component task-thread should be closed because the containing JobInProgress has completed or the task is killed by the user
`boolean` | `shouldCommit(TaskAttemptID taskid)` Returns whether the task attempt should be committed or not
`boolean` | `wasKilled()` Was the task killed?

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### LOG

    public static final org.apache.commons.logging.Log LOG

Constructor Detail

### TaskInProgress

    public TaskInProgress(JobID jobid,
                          String jobFile,
                          JobSplit.TaskSplitMetaInfo split,
                          JobTracker jobtracker,
                          JobConf conf,
                          JobInProgress job,
                          int partition,
                          int numSlotsRequired)

  Constructor for MapTask

### TaskInProgress

    public TaskInProgress(JobID jobid,
                          String jobFile,
                          int numMaps,
                          int partition,
                          JobTracker jobtracker,
                          JobConf conf,
                          JobInProgress job,
                          int numSlotsRequired)

  Constructor for ReduceTask

Method Detail

### idWithinJob

    public int idWithinJob()

  Return the index of the tip within the job, so "task_200707121733_1313_0002_m_012345" would return 12345;

  Returns:
    int the tip index

### isJobCleanupTask

    public boolean isJobCleanupTask()

### setJobCleanupTask

    public void setJobCleanupTask()

### isJobSetupTask

    public boolean isJobSetupTask()

### setJobSetupTask

    public void setJobSetupTask()

### isOnlyCommitPending

    public boolean isOnlyCommitPending()

### isCommitPending

    public boolean isCommitPending(TaskAttemptID taskId)

### getStartTime

    public long getStartTime()

  Return the start time

### getExecStartTime

    public long getExecStartTime()

  Return the exec start time

### setExecStartTime

    public void setExecStartTime(long startTime)

  Set the exec start time

### getExecFinishTime

    public long getExecFinishTime()

  Return the exec finish time

### setExecFinishTime

    public void setExecFinishTime(long finishTime)

  Set the exec finish time

### getJob

    public JobInProgress getJob()

  Return the parent job

### getTIPId

    public TaskID getTIPId()

  Return an ID for this task, not its component taskid-threads

### isMapTask

    public boolean isMapTask()

  Whether this is a map task

### isFirstAttempt

    public boolean isFirstAttempt(TaskAttemptID taskId)

  Is the Task associated with taskid is the first attempt of the tip?

  Parameters:
    `taskId` -
  Returns:
    Returns true if the Task is the first attempt of the tip

### isRunning

    public boolean isRunning()

  Is this tip currently running any tasks?

  Returns:
    true if any tasks are running

### isComplete

    public boolean isComplete()

  Is this tip complete?

  Returns:
    `true` if the tip is complete, else `false`

### isComplete

    public boolean isComplete(TaskAttemptID taskid)

  Is the given taskid the one that took this tip to completion?

  Parameters:
    `taskid` - taskid of attempt to check for completion
  Returns:
    `true` if taskid is complete, else `false`

### isFailed

    public boolean isFailed()

  Is the tip a failure?

  Returns:
    `true` if tip has failed, else `false`

### numTaskFailures

    public int numTaskFailures()

  Number of times the TaskInProgress has failed.

### numKilledTasks

    public int numKilledTasks()

  Number of times the TaskInProgress has been killed by the framework.

### getProgress

    public double getProgress()

  Get the overall progress (from 0 to 1.0) for this TIP

### getCounters

    public Counters getCounters()

  Get the task's counters

### shouldClose

    public boolean shouldClose(TaskAttemptID taskid)

  Returns whether a component task-thread should be closed because the containing JobInProgress has completed or the task is killed by the user

### doCommit

    public void doCommit(TaskAttemptID taskid)

  Commit this task attempt for the tip.

  Parameters:
    `taskid` -

### shouldCommit

    public boolean shouldCommit(TaskAttemptID taskid)

  Returns whether the task attempt should be committed or not

### generateSingleReport

    public TaskReport generateSingleReport()

  Creates a "status report" for this task. Includes the task ID and overall status, plus reports for all the component task-threads that have ever been started.

### getDiagnosticInfo

    public List<String> getDiagnosticInfo(TaskAttemptID taskId)

  Get the diagnostic messages for a given task within this tip.

  Parameters:
    `taskId` - the id of the required task
  Returns:
    the list of diagnostics for that task

### addDiagnosticInfo

    public void addDiagnosticInfo(TaskAttemptID taskId,
                                  String diagInfo)

  Save diagnostic information for a given task.

  Parameters:
    `taskId` - id of the task
    `diagInfo` - diagnostic information for the task

### incompleteSubTask

    public void incompleteSubTask(TaskAttemptID taskid,
                                  JobStatus jobStatus)

  Indicate that one of the taskids in this TaskInProgress has failed.

### completed

    public void completed(TaskAttemptID taskid)

  Indicate that one of the taskids in this TaskInProgress has successfully completed!

### getSplitLocations

    public String[] getSplitLocations()

  Get the split locations

### getTaskStatuses

    public TaskStatus[] getTaskStatuses()

  Get the Status of the tasks managed by this TIP

### getTaskStatus

    public TaskStatus getTaskStatus(TaskAttemptID taskid)

  Get the status of the specified task

  Parameters:
    `taskid` -

### kill

    public void kill()

  The TIP's been ordered kill()ed.

### wasKilled

    public boolean wasKilled()

  Was the task killed?

  Returns:
    true if the task killed

### getTaskToRun

    public Task getTaskToRun(String taskTracker)
                      throws IOException

  Return a Task that can be sent to a TaskTracker for execution.

  Throws:
    `IOException`

### addRunningTask

    public Task addRunningTask(TaskAttemptID taskid,
                               String taskTracker)

### addRunningTask

    public Task addRunningTask(TaskAttemptID taskid,
                               String taskTracker,
                               boolean taskCleanup)

  Adds a previously running task to this tip. This is used in case of jobtracker restarts.

### hasFailedOnMachine

    public boolean hasFailedOnMachine(String trackerHost)

  Has this task already failed on this machine?

  Parameters:
    `trackerHost` - The task tracker hostname
  Returns:
    Has it failed?

### hasRunOnMachine

    public boolean hasRunOnMachine(String trackerHost,
                                   String trackerName)

  Was this task ever scheduled to run on this machine?

  Parameters:
    `trackerHost` - The task tracker hostname
    `trackerName` - The tracker name
  Returns:
    Was task scheduled on the tracker?

### getNumberOfFailedMachines

    public int getNumberOfFailedMachines()

  Get the number of machines where this task has failed.

  Returns:
    the size of the failed machine set

### getIdWithinJob

    public int getIdWithinJob()

  Get the id of this map or reduce task.

  Returns:
    The index of this tip in the maps/reduces lists.

### setSuccessEventNumber

    public void setSuccessEventNumber(int eventNumber)

  Set the event number that was raised for this tip

### getSuccessEventNumber

    public int getSuccessEventNumber()

  Get the event number that was raised for this tip

### getSplitNodes

    public String getSplitNodes()

  Gets the Node list of input split locations sorted in rack order.

### getMapInputSize

    public long getMapInputSize()

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2009 The Apache Software Foundation