Package org.apache.nemo.runtime.master.scheduler

Class StreamingScheduler

java.lang.Object

org.apache.nemo.runtime.master.scheduler.StreamingScheduler

All Implemented Interfaces:

Scheduler

@NotThreadSafe
public final class StreamingScheduler
extends java.lang.Object
implements Scheduler

A simple scheduler for streaming workloads. - Keeps track of new executors - Schedules all tasks in the plan at once. - Crashes the system upon any other events (should be fixed in the future) - Never stops running.

Method Summary

Modifier and Type	Method	Description
void	onExecutorAdded(ExecutorRepresenter executorRepresenter)	Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.
void	onExecutorRemoved(java.lang.String executorId)	Called when an executor is removed from Runtime, so that faults related to the removal can be handled.
void	onSpeculativeExecutionCheck()	Called to check for speculative execution.
void	onTaskStateReportFromExecutor(java.lang.String executorId, java.lang.String taskId, int taskAttemptIndex, TaskState.State newState, java.lang.String vertexPutOnHold, TaskState.RecoverableTaskFailureCause failureCause)	Called when a Task's execution state changes.
void	schedulePlan(PhysicalPlan submittedPhysicalPlan, int maxScheduleAttempt)	Schedules the given plan.
void	terminate()	To be called when a job should be terminated.
void	updatePlan(PhysicalPlan newPhysicalPlan)	Receives and updates the scheduler with a new physical plan for a job.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

schedulePlan

public void schedulePlan(PhysicalPlan submittedPhysicalPlan,
                         int maxScheduleAttempt)

Description copied from interface: Scheduler

Schedules the given plan.

Specified by:

schedulePlan in interface Scheduler

Parameters:

submittedPhysicalPlan - the plan of the job being submitted.

maxScheduleAttempt - the max number of times this plan/sub-part of the plan should be attempted.

updatePlan

public void updatePlan(PhysicalPlan newPhysicalPlan)

Description copied from interface: Scheduler

Receives and updates the scheduler with a new physical plan for a job.

Specified by:

updatePlan in interface Scheduler

Parameters:

newPhysicalPlan - new physical plan for the job.

onTaskStateReportFromExecutor

public void onTaskStateReportFromExecutor(java.lang.String executorId,
                                          java.lang.String taskId,
                                          int taskAttemptIndex,
                                          TaskState.State newState,
                                          @Nullable
                                          java.lang.String vertexPutOnHold,
                                          TaskState.RecoverableTaskFailureCause failureCause)

Description copied from interface: Scheduler

Called when a Task's execution state changes.

Specified by:

onTaskStateReportFromExecutor in interface Scheduler

Parameters:

executorId - of the executor in which the Task is executing.

taskId - of the Task whose state must be updated.

taskAttemptIndex - the number of times this Task has executed. ************** the below parameters are only valid for failures *****************

newState - for the Task.

vertexPutOnHold - the ID of task that are put on hold. It is null otherwise.

failureCause - for which the Task failed in the case of a recoverable failure.

onSpeculativeExecutionCheck

public void onSpeculativeExecutionCheck()

Description copied from interface: Scheduler

Called to check for speculative execution.

Specified by:

onSpeculativeExecutionCheck in interface Scheduler

onExecutorAdded

public void onExecutorAdded(ExecutorRepresenter executorRepresenter)

Description copied from interface: Scheduler

Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.

Specified by:

onExecutorAdded in interface Scheduler

Parameters:

executorRepresenter - a representation of the added executor.

onExecutorRemoved

public void onExecutorRemoved(java.lang.String executorId)

Description copied from interface: Scheduler

Called when an executor is removed from Runtime, so that faults related to the removal can be handled.

Specified by:

onExecutorRemoved in interface Scheduler

Parameters:

executorId - of the executor that has been removed.

terminate

public void terminate()

Description copied from interface: Scheduler

To be called when a job should be terminated. Any clean up code should be implemented in this method.

Specified by:

terminate in interface Scheduler