Package org.apache.nemo.runtime.executor.data.block

Class NonSerializedMemoryBlock<K extends java.io.Serializable>

java.lang.Object

org.apache.nemo.runtime.executor.data.block.NonSerializedMemoryBlock<K>

Type Parameters:

K - the key type of its partitions.

All Implemented Interfaces:

Block<K>

@NotThreadSafe
public final class NonSerializedMemoryBlock<K extends java.io.Serializable>
extends java.lang.Object
implements Block<K>

This class represents a block which is stored in local memory and not serialized. Concurrent read is supported, but concurrent write is not supported.

Constructor Summary

Constructors

Constructor	Description
NonSerializedMemoryBlock(java.lang.String blockId, Serializer serializer, MemoryPoolAssigner memoryPoolAssigner)	Constructor.

Method Summary

Modifier and Type	Method	Description
java.util.Optional<java.util.Map<K,java.lang.Long>>	commit()	Commits this block to prevent further write.
void	commitPartitions()	Commits all un-committed partitions.
java.lang.String	getId()
boolean	isCommitted()
java.lang.Iterable<NonSerializedPartition<K>>	readPartitions(KeyRange keyRange)	Retrieves the NonSerializedPartitions in a specific hash range from this block.
java.lang.Iterable<SerializedPartition<K>>	readSerializedPartitions(KeyRange keyRange)	Retrieves the SerializedPartitions in a specific hash range.
void	write(K key, java.lang.Object element)	Writes an element to non-committed block.
void	writePartitions(java.lang.Iterable<NonSerializedPartition<K>> partitions)	Stores NonSerializedPartitions to this block.
void	writeSerializedPartitions(java.lang.Iterable<SerializedPartition<K>> partitions)	Stores SerializedPartitions to this block.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NonSerializedMemoryBlock

public NonSerializedMemoryBlock(java.lang.String blockId,
                                Serializer serializer,
                                MemoryPoolAssigner memoryPoolAssigner)

Constructor.

Parameters:

blockId - the ID of this block.

serializer - the Serializer.

memoryPoolAssigner - the MemoryPoolAssigner for memory allocation.

Method Detail

write

public void write(K key,
                  java.lang.Object element)
           throws BlockWriteException

Writes an element to non-committed block. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

write in interface Block<K extends java.io.Serializable>

Parameters:

key - the key.

element - the element to write.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

writePartitions

public void writePartitions(java.lang.Iterable<NonSerializedPartition<K>> partitions)
                     throws BlockWriteException

Stores NonSerializedPartitions to this block. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

writePartitions in interface Block<K extends java.io.Serializable>

Parameters:

partitions - the NonSerializedPartitions to store.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

writeSerializedPartitions

public void writeSerializedPartitions(java.lang.Iterable<SerializedPartition<K>> partitions)
                               throws BlockWriteException

Stores SerializedPartitions to this block. Because all data in this block is stored in a non-serialized form, the data in these partitions have to be deserialized. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

writeSerializedPartitions in interface Block<K extends java.io.Serializable>

Parameters:

partitions - the SerializedPartitions to store.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

readPartitions

public java.lang.Iterable<NonSerializedPartition<K>> readPartitions(KeyRange keyRange)
                                                             throws BlockFetchException

Retrieves the NonSerializedPartitions in a specific hash range from this block. Invariant: This should not be invoked before this block is committed.

Specified by:

readPartitions in interface Block<K extends java.io.Serializable>

Parameters:

keyRange - the hash range to retrieve.

Returns:

an iterable of NonSerializedPartitions.

Throws:

BlockFetchException - for any error occurred while trying to fetch a block.

readSerializedPartitions

public java.lang.Iterable<SerializedPartition<K>> readSerializedPartitions(KeyRange keyRange)
                                                                    throws BlockFetchException

Retrieves the SerializedPartitions in a specific hash range. Because the data is stored in a non-serialized form, it have to be serialized. Invariant: This should not be invoked before this block is committed.

Specified by:

readSerializedPartitions in interface Block<K extends java.io.Serializable>

Parameters:

keyRange - the key range to retrieve.

Returns:

an iterable of SerializedPartitions.

Throws:

BlockFetchException - for any error occurred while trying to fetch a block.

commit

public java.util.Optional<java.util.Map<K,java.lang.Long>> commit()

Commits this block to prevent further write.

Specified by:

commit in interface Block<K extends java.io.Serializable>

Returns:

empty optional because the data is not serialized.

commitPartitions

public void commitPartitions()
                      throws BlockWriteException

Commits all un-committed partitions.

Specified by:

commitPartitions in interface Block<K extends java.io.Serializable>

Throws:

BlockWriteException

getId

public java.lang.String getId()

Specified by:

getId in interface Block<K extends java.io.Serializable>

Returns:

the ID of this block.

isCommitted

public boolean isCommitted()

Specified by:

isCommitted in interface Block<K extends java.io.Serializable>

Returns:

whether this block is committed or not.