Package org.apache.nemo.common.ir.vertex.utility

Class TaskSizeSplitterVertex

  • All Implemented Interfaces:
    java.io.Serializable, Cloneable<IRVertex>
    public final class TaskSizeSplitterVertex
extends LoopVertex
    This vertex works as a partition-based sampling vertex of dynamic task sizing pass. It covers both sampling vertices and optimized vertices known from sampling by iterating same vertices, giving different properties in each iteration.
    See Also:
    Serialized Form

    • Constructor Detail

      • TaskSizeSplitterVertex

        public TaskSizeSplitterVertex​(java.lang.String splitterVertexName,
                              java.util.Set<IRVertex> originalVertices,
                              java.util.Set<IRVertex> groupStartingVertices,
                              java.util.Set<IRVertex> verticesWithGroupOutgoingEdges,
                              java.util.Set<IRVertex> groupEndingVertices,
                              java.util.Set<IREdge> edgesBetweenOriginalVertices,
                              int partitionerProperty)
        Default constructor of TaskSizeSplitterVertex.
        Parameters:
        splitterVertexName - for now, this doesn't do anything. This is inserted to enable extension from LoopVertex.
        originalVertices - Set of vertices which form one stage and which splitter will wrap up.
        groupStartingVertices - The first vertex in stage. Although it is given as a Set, we assert that this set has only one element (guaranteed by stage partitioner logic)
        verticesWithGroupOutgoingEdges - Vertices which has outgoing edges to other stage.
        groupEndingVertices - Vertices which has only outgoing edges to other stage.
        edgesBetweenOriginalVertices - Edges which connects original vertices.
        partitionerProperty - PartitionerProperty of incoming stage edge regarding to job data size. For more information, check

    • Method Detail

      • getOriginalVertices

        public java.util.Set<IRVertex> getOriginalVertices()

      • getGroupStartingVertices

        public java.util.Set<IRVertex> getGroupStartingVertices()

      • getVerticesWithGroupOutgoingEdges

        public java.util.Set<IRVertex> getVerticesWithGroupOutgoingEdges()

      • getGroupEndingVertices

        public java.util.Set<IRVertex> getGroupEndingVertices()

      • increaseTestingTrial

        public void increaseTestingTrial()

      • getEdgesFromOutsideToOriginal

        public java.util.Set<IREdge> getEdgesFromOutsideToOriginal​(DAG<IRVertex,​IREdge> dag)
        Get edges which come to original vertices from outer sources by observing the dag. This will be the 'dagIncomingEdges' in Splitter vertex. Edge case: Happens when previous vertex(i.e. outer source) is also a splitter vertex. In this case, we need to get original edges which is invisible from the dag by hacking into previous splitter vertex.
        Parameters:
        dag - dag to insert Splitter Vertex.
        Returns:
        a set of edges from outside to original vertices.

      • getEdgesFromOriginalToOutside

        public java.util.Set<IREdge> getEdgesFromOriginalToOutside​(DAG<IRVertex,​IREdge> dag)
        Get edges which come from original vertices to outer destinations by observing the dag. This will be the 'dagOutgoingEdges' in Splitter vertex. Edge case: Happens when the vertex to be executed after the splitter vertex (i.e. outer destination) is also a splitter vertex. In this case, we need to get original edges which is invisible from the dag by hacking into next splitter vertex.
        Parameters:
        dag - dag to insert Splitter Vertex.
        Returns:
        a set of edges from original vertices to outside.

      • getEdgesFromOutsideToSplitter

        public java.util.Set<IREdge> getEdgesFromOutsideToSplitter​(DAG<IRVertex,​IREdge> dag)
        Get edges which come to splitter from outside sources. These edges have a one-to-one relationship with edgesFromOutsideToOriginal. Edge case: Happens when previous vertex(i.e. outer source) is also a splitter vertex. In this case, we need to modify the prevSplitter's LoopEdge - InternalEdge mapping relationship, since inserting this Splitter Vertex changes the destination of prevSplitter's LoopEdge from the original vertex to this Splitter Vertex
        Parameters:
        dag - dag to insert Splitter Vertex
        Returns:
        a set of edges pointing at Splitter Vertex

      • getEdgesFromSplitterToOutside

        public java.util.Set<IREdge> getEdgesFromSplitterToOutside​(DAG<IRVertex,​IREdge> dag)
        Get edges which come out from splitter to outside destinations. These edges have a one-to-one relationship with edgesFromOriginalToOutside. Edge case: Happens when vertex to be executed after this Splitter Vertex(i.e. outer destination) is also a Splitter Vertex. In this case, we need to modify the nextSplitter's LoopEdge - InternalEdge mapping relationship, since inserting this Splitter Vertex changes the source of prevSplitter's LoopEdge from the original vertex to this Splitter Vertex.
        Parameters:
        dag - dag to insert Splitter Vertex.
        Returns:
        a set of edges coming out from Splitter Vertex.

      • printLogs

        public void printLogs()