Package org.apache.nemo.common.ir.vertex

Class InMemorySourceVertex<T>

  • Type Parameters:
    T - type of data.
    All Implemented Interfaces:
    java.io.Serializable, Cloneable<IRVertex>
    public final class InMemorySourceVertex<T>
extends SourceVertex<T>
    Source vertex with the data in memory.
    See Also:
    Serialized Form

    • Constructor Detail

      • InMemorySourceVertex

        public InMemorySourceVertex​(java.lang.Iterable<T> initializedSourceData)
        Constructor for InMemorySourceVertex.
        Parameters:
        initializedSourceData - the initial data object.

    • Method Detail

      • getClone

        public InMemorySourceVertex<T> getClone()
        Description copied from interface: Cloneable
        Creates and returns a copy of this object.

        The precise meaning of "copy" may depend on the class of the object. The general intent is that, all fields of the object are copied.

        Returns:
        a clone of this object.

      • isBounded

        public boolean isBounded()
        Specified by:
        isBounded in class SourceVertex<T>
        Returns:
        true if it is bounded source

      • getReadables

        public java.util.List<Readable<T>> getReadables​(int desiredNumOfSplits)
                                         throws java.lang.Exception
        Description copied from class: SourceVertex
        Gets parallel readables.
        Specified by:
        getReadables in class SourceVertex<T>
        Parameters:
        desiredNumOfSplits - number of splits desired.
        Returns:
        the list of readables.
        Throws:
        java.lang.Exception - if fail to get.

      • getEstimatedSizeBytes

        public long getEstimatedSizeBytes()
        Description copied from class: SourceVertex
        Gets the estimated size of bytes. Returns 0L if not applicable.
        Specified by:
        getEstimatedSizeBytes in class SourceVertex<T>
        Returns:
        size of input bytes.

      • clearInternalStates

        public void clearInternalStates()
        Description copied from class: SourceVertex
        Clears internal states, must be called after getReadables(). Concretely, this clears the huge list of input splits held by objects like BeamBoundedSourceVertex before sending the vertex to remote executors. Between clearing states of an existing vertex, and creating a new vertex, we've chosen the former approach to ensure consistent use of the same IRVertex object across the compiler, the master, and the executors.
        Specified by:
        clearInternalStates in class SourceVertex<T>