Package org.apache.nemo.common.ir.vertex

Class CachedSourceVertex<T>

  • Type Parameters:
    T - the type of data to emit.
    All Implemented Interfaces:
    java.io.Serializable, Cloneable<IRVertex>
    public final class CachedSourceVertex<T>
extends SourceVertex<T>
    Bounded source vertex for cached data. It does not have actual data but just wraps the cached input data.
    See Also:
    Serialized Form

    • Constructor Detail

      • CachedSourceVertex

        public CachedSourceVertex​(int numPartitions)
        Constructor.
        Parameters:
        numPartitions - the number of partitions.

    • Method Detail

      • getClone

        public CachedSourceVertex getClone()
        Description copied from interface: Cloneable
        Creates and returns a copy of this object.

        The precise meaning of "copy" may depend on the class of the object. The general intent is that, all fields of the object are copied.

        Returns:
        a clone of this object.

      • isBounded

        public boolean isBounded()
        Specified by:
        isBounded in class SourceVertex<T>
        Returns:
        true if it is bounded source

      • getReadables

        public java.util.List<Readable<T>> getReadables​(int desiredNumOfSplits)
        Description copied from class: SourceVertex
        Gets parallel readables.
        Specified by:
        getReadables in class SourceVertex<T>
        Parameters:
        desiredNumOfSplits - number of splits desired.
        Returns:
        the list of readables.

      • getEstimatedSizeBytes

        public long getEstimatedSizeBytes()
        Description copied from class: SourceVertex
        Gets the estimated size of bytes. Returns 0L if not applicable.
        Specified by:
        getEstimatedSizeBytes in class SourceVertex<T>
        Returns:
        size of input bytes.

      • clearInternalStates

        public void clearInternalStates()
        Description copied from class: SourceVertex
        Clears internal states, must be called after getReadables(). Concretely, this clears the huge list of input splits held by objects like BeamBoundedSourceVertex before sending the vertex to remote executors. Between clearing states of an existing vertex, and creating a new vertex, we've chosen the former approach to ensure consistent use of the same IRVertex object across the compiler, the master, and the executors.
        Specified by:
        clearInternalStates in class SourceVertex<T>