Package org.apache.nemo.runtime.master.scheduler

Interface Scheduler

All Known Implementing Classes:

BatchScheduler, SimulationScheduler, StreamingScheduler

public interface Scheduler

Only two threads call scheduling code: RuntimeMaster thread (RMT), and SchedulerThread(ST). RMT and ST meet only at two points: ExecutorRegistry, and PendingTaskCollectionPointer, which are synchronized(ThreadSafe). Other scheduler-related classes that are accessed by only one of the two threads are not synchronized(NotThreadSafe).

Method Summary

Modifier and Type	Method	Description
void	onExecutorAdded(ExecutorRepresenter executorRepresenter)	Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.
void	onExecutorRemoved(java.lang.String executorId)	Called when an executor is removed from Runtime, so that faults related to the removal can be handled.
void	onSpeculativeExecutionCheck()	Called to check for speculative execution.
void	onTaskStateReportFromExecutor(java.lang.String executorId, java.lang.String taskId, int attemptIdx, TaskState.State newState, java.lang.String taskPutOnHold, TaskState.RecoverableTaskFailureCause failureCause)	Called when a Task's execution state changes.
void	schedulePlan(PhysicalPlan physicalPlan, int maxScheduleAttempt)	Schedules the given plan.
void	terminate()	To be called when a job should be terminated.
void	updatePlan(PhysicalPlan newPhysicalPlan)	Receives and updates the scheduler with a new physical plan for a job.

Method Detail

schedulePlan

void schedulePlan(PhysicalPlan physicalPlan,
                  int maxScheduleAttempt)

Schedules the given plan.

Parameters:

physicalPlan - the plan of the job being submitted.

maxScheduleAttempt - the max number of times this plan/sub-part of the plan should be attempted.

updatePlan

void updatePlan(PhysicalPlan newPhysicalPlan)

Receives and updates the scheduler with a new physical plan for a job.

Parameters:

newPhysicalPlan - new physical plan for the job.

onExecutorAdded

void onExecutorAdded(ExecutorRepresenter executorRepresenter)

Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.

Parameters:

executorRepresenter - a representation of the added executor.

onExecutorRemoved

void onExecutorRemoved(java.lang.String executorId)

Called when an executor is removed from Runtime, so that faults related to the removal can be handled.

Parameters:

executorId - of the executor that has been removed.

onTaskStateReportFromExecutor

void onTaskStateReportFromExecutor(java.lang.String executorId,
                                   java.lang.String taskId,
                                   int attemptIdx,
                                   TaskState.State newState,
                                   @Nullable
                                   java.lang.String taskPutOnHold,
                                   TaskState.RecoverableTaskFailureCause failureCause)

Called when a Task's execution state changes.

Parameters:

executorId - of the executor in which the Task is executing.

taskId - of the Task whose state must be updated.

newState - for the Task.

attemptIdx - the number of times this Task has executed. ************** the below parameters are only valid for failures *****************

taskPutOnHold - the ID of task that are put on hold. It is null otherwise.

failureCause - for which the Task failed in the case of a recoverable failure.

onSpeculativeExecutionCheck

void onSpeculativeExecutionCheck()

Called to check for speculative execution.

terminate

void terminate()

To be called when a job should be terminated. Any clean up code should be implemented in this method.