Package org.apache.nemo.runtime.executor.data

Class BlockManagerWorker

java.lang.Object

org.apache.nemo.runtime.executor.data.BlockManagerWorker

@ThreadSafe
public final class BlockManagerWorker
extends java.lang.Object

Executor-side block manager.

Method Summary

Modifier and Type	Method	Description
Block	createBlock(java.lang.String blockId, DataStoreProperty.Value blockStore)	Creates a new block.
void	onInputContext(ByteInputContext inputContext)	Respond to a block notification by another executor.
void	onOutputContext(ByteOutputContext outputContext)	Respond to a block request by another executor.
java.util.concurrent.CompletableFuture<DataUtil.IteratorWithNumBytes>	readBlock(java.lang.String blockIdWildcard, java.lang.String runtimeEdgeId, ExecutionPropertyMap<EdgeExecutionProperty> edgeProperties, KeyRange keyRange)	Inquiries the location of the specific block and routes the request to the local block manager worker or to the lower data plane.
void	removeBlock(java.lang.String blockId, DataStoreProperty.Value blockStore)	Remove the block from store.
void	writeBlock(Block block, DataStoreProperty.Value blockStore, int expectedReadTotal, DataPersistenceProperty.Value persistence)	Writes a block to a store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createBlock

public Block createBlock(java.lang.String blockId,
                         DataStoreProperty.Value blockStore)

Creates a new block.

Parameters:

blockId - the ID of the block to create.

blockStore - the store to place the block.

Returns:

the created block.

readBlock

public java.util.concurrent.CompletableFuture<DataUtil.IteratorWithNumBytes> readBlock(java.lang.String blockIdWildcard,
                                                                                       java.lang.String runtimeEdgeId,
                                                                                       ExecutionPropertyMap<EdgeExecutionProperty> edgeProperties,
                                                                                       KeyRange keyRange)

Inquiries the location of the specific block and routes the request to the local block manager worker or to the lower data plane. This can be invoked multiple times per blockId (maybe due to failures).

Parameters:

blockIdWildcard - of the block.

runtimeEdgeId - id of the runtime edge that corresponds to the block.

edgeProperties - for the edge.

keyRange - the key range descriptor

Returns:

the CompletableFuture of the block.

writeBlock

public void writeBlock(Block block,
                       DataStoreProperty.Value blockStore,
                       int expectedReadTotal,
                       DataPersistenceProperty.Value persistence)

Writes a block to a store.

Parameters:

block - the block to write.

blockStore - the store to save the block.

expectedReadTotal - the expected number of read for this block.

persistence - how to handle the used block.

removeBlock

public void removeBlock(java.lang.String blockId,
                        DataStoreProperty.Value blockStore)

Remove the block from store.

Parameters:

blockId - the ID of the block to remove.

blockStore - the store which contains the block.

onOutputContext

public void onOutputContext(ByteOutputContext outputContext)
                     throws com.google.protobuf.InvalidProtocolBufferException

Respond to a block request by another executor.

This method is executed by {org.apache.nemo.runtime.executor.data.blocktransfer.BlockTransport} thread. \ Never execute a blocking call in this method!

Parameters:

outputContext - ByteOutputContext

Throws:

com.google.protobuf.InvalidProtocolBufferException - from errors during parsing context descriptor

onInputContext

public void onInputContext(ByteInputContext inputContext)

Respond to a block notification by another executor.

This method is executed by {org.apache.nemo.runtime.executor.data.blocktransfer.BlockTransport} thread. Never execute a blocking call in this method!

Parameters:

inputContext - ByteInputContext