Package org.apache.nemo.runtime.executor.data.partition

Class NonSerializedPartition<K>

java.lang.Object

org.apache.nemo.runtime.executor.data.partition.NonSerializedPartition<K>

Type Parameters:

K - the key type of its partitions.

All Implemented Interfaces:

Partition<java.lang.Iterable,K>

public final class NonSerializedPartition<K>
extends java.lang.Object
implements Partition<java.lang.Iterable,K>

A collection of data elements. The data is stored as an iterable of elements. This is a unit of read / write towards Blocks.

Constructor Summary

Constructors

Constructor	Description
NonSerializedPartition(K key)	Creates a non-serialized Partition without actual data.
NonSerializedPartition(K key, java.util.List data, long numSerializedBytes, long numEncodedBytes)	Creates a non-serialized Partition with actual data.

Method Summary

Modifier and Type	Method	Description
void	commit()	Commits a partition to prevent further data write.
java.lang.Iterable	getData()
K	getKey()
long	getNumEncodedBytes()
long	getNumSerializedBytes()
boolean	isSerialized()
void	write(java.lang.Object element)	Writes an element to non-committed partition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NonSerializedPartition

public NonSerializedPartition(K key)

Creates a non-serialized Partition without actual data. Data can be written to this partition until it is committed.

Parameters:

key - the key of this partition.

NonSerializedPartition

public NonSerializedPartition(K key,
                              java.util.List data,
                              long numSerializedBytes,
                              long numEncodedBytes)

Creates a non-serialized Partition with actual data. Data cannot be written to this partition after the construction.

Parameters:

key - the key.

data - the non-serialized data.

numSerializedBytes - the number of bytes in serialized form (which is, for example, encoded and compressed)

numEncodedBytes - the number of bytes in encoded form (which is ready to be decoded)

Method Detail

write

public void write(java.lang.Object element)
           throws java.io.IOException

Writes an element to non-committed partition.

Specified by:

write in interface Partition<java.lang.Iterable,K>

Parameters:

element - element to write.

Throws:

java.io.IOException - if the partition is already committed.

commit

public void commit()

Commits a partition to prevent further data write.

Specified by:

commit in interface Partition<java.lang.Iterable,K>

getNumSerializedBytes

public long getNumSerializedBytes()
                           throws DataUtil.IteratorWithNumBytes.NumBytesNotSupportedException

Returns:

the number of bytes in serialized form (which is, for example, encoded and compressed)

Throws:

DataUtil.IteratorWithNumBytes.NumBytesNotSupportedException - when then information is not available

getNumEncodedBytes

public long getNumEncodedBytes()
                        throws DataUtil.IteratorWithNumBytes.NumBytesNotSupportedException

Returns:

the number of bytes in encoded form (which is ready to be decoded)

Throws:

DataUtil.IteratorWithNumBytes.NumBytesNotSupportedException - when then information is not available

getKey

public K getKey()

Specified by:

getKey in interface Partition<java.lang.Iterable,K>

Returns:

the key value.

isSerialized

public boolean isSerialized()

Specified by:

isSerialized in interface Partition<java.lang.Iterable,K>

Returns:

whether the data in this Partition is serialized or not.

getData

public java.lang.Iterable getData()
                           throws java.io.IOException

Specified by:

getData in interface Partition<java.lang.Iterable,K>

Returns:

the non-serialized data.

Throws:

java.io.IOException - if the partition is not committed yet.