Class BlockManagerMaster


  • @ThreadSafe
    public final class BlockManagerMaster
    extends java.lang.Object
    Master-side block manager. This implementation assumes that only a single user application can submit (maybe multiple) plans through Scheduler.
    • Method Detail

      • removeWorker

        public java.util.Set<java.lang.String> removeWorker​(java.lang.String executorId)
        Manages the block information when a executor is removed.
        Parameters:
        executorId - the id of removed executor.
        Returns:
        the set of tasks have to be recomputed.
      • getBlockHandlers

        public java.util.List<BlockManagerMaster.BlockRequestHandler> getBlockHandlers​(java.lang.String blockIdOrWildcard,
                                                                                       BlockState.State state)
        Get handlers of blocks that are in a particular state.
        Parameters:
        blockIdOrWildcard - to query
        state - of the block
        Returns:
        the handlers, empty if none matches.
      • getProducerTaskIds

        public java.util.Set<java.lang.String> getProducerTaskIds​(java.lang.String blockId)
        Gets the ids of the tasks which already produced or will produce data for a specific block.
        Parameters:
        blockId - the id of the block.
        Returns:
        the ids of the producer tasks.
      • onProducerTaskScheduled

        public void onProducerTaskScheduled​(java.lang.String taskId,
                                            java.util.Set<java.lang.String> blockIds)
        To be called when a potential producer task is scheduled.
        Parameters:
        taskId - the ID of the scheduled task.
        blockIds - this task will produce
      • onProducerTaskFailed

        public void onProducerTaskFailed​(java.lang.String failedTaskId)
        To be called when a potential producer task fails. Only the Tasks that have not yet completed (i.e. blocks not yet committed) will call this method.
        Parameters:
        failedTaskId - the ID of the task that failed.
      • onBlockStateChanged

        public void onBlockStateChanged​(java.lang.String blockId,
                                        BlockState.State newState,
                                        @Nullable
                                        java.lang.String location)
        Deals with state change of a block.
        Parameters:
        blockId - the id of the block.
        newState - the new state of the block.
        location - the location of the block (e.g., worker id, remote store). null if not committed or lost.