Package org.apache.nemo.runtime.master

Class PlanStateManager

  • @ThreadSafe
public final class PlanStateManager
extends java.lang.Object
    Maintains three levels of state machines (PlanState, StageState, and TaskState) of a physical plan. The main API this class provides is onTaskStateReportFromExecutor(), which directly changes a TaskState. PlanState and StageState are updated internally in the class, and can only be read from the outside.

    (CONCURRENCY) The public methods of this class are synchronized.

    • Method Detail

      • newInstance

        public static PlanStateManager newInstance​(java.lang.String dagDirectory)
        Static constructor for manual usage.
        Parameters:
        dagDirectory - the DAG directory to store the JSON to.
        Returns:
        a new PlanStateManager instance.

      • setMetricStore

        public void setMetricStore​(MetricStore metricStore)
        Parameters:
        metricStore - set the metric store of the paln state manager.

      • updatePlan

        public void updatePlan​(PhysicalPlan physicalPlanToUpdate,
                       int maxScheduleAttemptToSet)
        Update the physical plan and maximum attempt.
        Parameters:
        physicalPlanToUpdate - the physical plan to manage.
        maxScheduleAttemptToSet - the maximum number of times this plan/sub-part of the plan should be attempted.

      • getTaskAttemptsToSchedule

        public java.util.List<java.lang.String> getTaskAttemptsToSchedule​(java.lang.String stageId)
        Get task attempts that are "READY".
        Parameters:
        stageId - to run
        Returns:
        executable task attempts

      • getAllTaskAttemptsOfStage

        public java.util.Set<java.lang.String> getAllTaskAttemptsOfStage​(java.lang.String stageId)
        Parameters:
        stageId - to query.
        Returns:
        all task attempt ids of the stage.

      • getExecutingTaskToRunningTimeMs

        public java.util.Map<java.lang.String,​java.lang.Long> getExecutingTaskToRunningTimeMs​(java.lang.String stageId)
        Parameters:
        stageId - to query.
        Returns:
        a map from an EXECUTING task to its running time so far.

      • getCompletedTaskTimeListMs

        public java.util.List<java.lang.Long> getCompletedTaskTimeListMs​(java.lang.String stageId)
        List of task times so far for this stage.
        Parameters:
        stageId - of the stage.
        Returns:
        a copy of the list, empty if none completed.

      • setNumOfClones

        public boolean setNumOfClones​(java.lang.String stageId,
                              int taskIndex,
                              int numOfClones)
        Parameters:
        stageId - of the clone.
        taskIndex - of the clone.
        numOfClones - of the clone.
        Returns:
        true if the numOfClones has been modified, false otherwise

      • onTaskStateChanged

        public void onTaskStateChanged​(java.lang.String taskId,
                               TaskState.State newTaskState)
        Updates the state of a task. Task state changes can occur both in master and executor. State changes that occur in master are initiated in BatchScheduler. State changes that occur in executors are sent to master as a control message, and the call to this method is initiated in BatchScheduler when the message/event is received.
        Parameters:
        taskId - the ID of the task.
        newTaskState - the new state of the task.

      • waitUntilFinish

        public PlanState.State waitUntilFinish()
        Wait for this plan to be finished and return the final state.
        Returns:
        the final state of this plan.

      • waitUntilFinish

        public PlanState.State waitUntilFinish​(long timeout,
                                       java.util.concurrent.TimeUnit unit)
        Wait for this plan to be finished and return the final state. It wait for at most the given time.
        Parameters:
        timeout - of waiting.
        unit - of the timeout.
        Returns:
        the final state of this plan.

      • getAllTaskAttemptIdsToItsState

        public java.util.Map<java.lang.String,​TaskState.State> getAllTaskAttemptIdsToItsState()
        Returns:
        a map from task attempt id to its current state.

      • isPlanDone

        public boolean isPlanDone()
        Returns:
        whether the execution for the plan is done or not.

      • getPlanId

        public java.lang.String getPlanId()
        Returns:
        the ID of the plan.

      • getPlanState

        public PlanState.State getPlanState()
        Returns:
        the state of the plan.

      • getStageState

        public StageState.State getStageState​(java.lang.String stageId)
        Parameters:
        stageId - the stage ID to query.
        Returns:
        the state of the stage.

      • getTaskState

        public TaskState.State getTaskState​(java.lang.String taskId)
        Parameters:
        taskId - the ID of the task to query.
        Returns:
        the state of the task.

      • getPhysicalPlan

        public PhysicalPlan getPhysicalPlan()
        Returns:
        the physical plan.

      • getMaxScheduleAttempt

        public int getMaxScheduleAttempt()
        Returns:
        the maximum number of task scheduling.

      • isInitialized

        public boolean isInitialized()
        Returns:
        whether any plan has been submitted and initialized.

      • storeJSON

        public void storeJSON​(java.lang.String suffix)
        Stores JSON representation of plan state into a file.
        Parameters:
        suffix - suffix for file name

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object