Package org.apache.nemo.runtime.executor.data.block

Class SerializedMemoryBlock<K extends java.io.Serializable>

java.lang.Object

org.apache.nemo.runtime.executor.data.block.SerializedMemoryBlock<K>

Type Parameters:

K - the key type of its partitions.

All Implemented Interfaces:

Block<K>

@NotThreadSafe
public final class SerializedMemoryBlock<K extends java.io.Serializable>
extends java.lang.Object
implements Block<K>

This class represents a block which is serialized and stored in local memory. Concurrent read is supported, but concurrent write is not supported.

Constructor Summary

Constructors

Constructor	Description
SerializedMemoryBlock(java.lang.String blockId, Serializer serializer, MemoryPoolAssigner memoryPoolAssigner)	Constructor.

Method Summary

Modifier and Type	Method	Description
java.util.Optional<java.util.Map<K,java.lang.Long>>	commit()	Commits this block to prevent further write.
void	commitPartitions()	Commits all un-committed partitions.
java.lang.String	getId()
boolean	isCommitted()
java.lang.Iterable<NonSerializedPartition<K>>	readPartitions(KeyRange keyRange)	Retrieves the NonSerializedPartitions in a specific hash range from this block.
java.lang.Iterable<SerializedPartition<K>>	readSerializedPartitions(KeyRange keyRange)	Retrieves the SerializedPartitions in a specific hash range.
void	release()	Releases the resource (i.e., off-heap memory) that the block holds.
void	write(K key, java.lang.Object element)	Writes an element to non-committed block.
void	writePartitions(java.lang.Iterable<NonSerializedPartition<K>> partitions)	Serialized and stores NonSerializedPartitions to this block.
void	writeSerializedPartitions(java.lang.Iterable<SerializedPartition<K>> partitions)	Stores SerializedPartitions to this block.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SerializedMemoryBlock

public SerializedMemoryBlock(java.lang.String blockId,
                             Serializer serializer,
                             MemoryPoolAssigner memoryPoolAssigner)

Constructor.

Parameters:

blockId - the ID of this block.

serializer - the Serializer.

memoryPoolAssigner - the MemoryPoolAssigner for memory allocation.

Method Detail

write

public void write(K key,
                  java.lang.Object element)

Writes an element to non-committed block. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

write in interface Block<K extends java.io.Serializable>

Parameters:

key - the key.

element - the element to write.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

writePartitions

public void writePartitions(java.lang.Iterable<NonSerializedPartition<K>> partitions)

Serialized and stores NonSerializedPartitions to this block. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

writePartitions in interface Block<K extends java.io.Serializable>

Parameters:

partitions - the NonSerializedPartitions to store.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

writeSerializedPartitions

public void writeSerializedPartitions(java.lang.Iterable<SerializedPartition<K>> partitions)

Stores SerializedPartitions to this block. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

writeSerializedPartitions in interface Block<K extends java.io.Serializable>

Parameters:

partitions - the SerializedPartitions to store.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

readPartitions

public java.lang.Iterable<NonSerializedPartition<K>> readPartitions(KeyRange keyRange)

Retrieves the NonSerializedPartitions in a specific hash range from this block. Because the data is stored in a serialized form, it have to be deserialized. Invariant: This should not be invoked before this block is committed.

Specified by:

readPartitions in interface Block<K extends java.io.Serializable>

Parameters:

keyRange - the key range to retrieve.

Returns:

an iterable of NonSerializedPartitions.

Throws:

BlockFetchException - for any error occurred while trying to fetch a block.

readSerializedPartitions

public java.lang.Iterable<SerializedPartition<K>> readSerializedPartitions(KeyRange keyRange)

Retrieves the SerializedPartitions in a specific hash range. Invariant: This should not be invoked before this block is committed.

Specified by:

readSerializedPartitions in interface Block<K extends java.io.Serializable>

Parameters:

keyRange - the key range to retrieve.

Returns:

an iterable of SerializedPartitions.

Throws:

BlockFetchException - for any error occurred while trying to fetch a block.

commit

public java.util.Optional<java.util.Map<K,java.lang.Long>> commit()

Commits this block to prevent further write.

Specified by:

commit in interface Block<K extends java.io.Serializable>

Returns:

the size of each partition.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

commitPartitions

public void commitPartitions()

Commits all un-committed partitions.

Specified by:

commitPartitions in interface Block<K extends java.io.Serializable>

getId

public java.lang.String getId()

Specified by:

getId in interface Block<K extends java.io.Serializable>

Returns:

the ID of this block.

isCommitted

public boolean isCommitted()

Specified by:

isCommitted in interface Block<K extends java.io.Serializable>

Returns:

whether this block is committed or not.

release

public void release()

Releases the resource (i.e., off-heap memory) that the block holds.