Class ParallelismProphet
- java.lang.Object
-
- org.apache.nemo.compiler.backend.nemo.prophet.ParallelismProphet
-
-
Constructor Summary
Constructors Constructor Description ParallelismProphet(IRDAG irdag, PhysicalPlan physicalPlan, SimulationScheduler simulationScheduler, PhysicalPlanGenerator physicalPlanGenerator, java.util.Set<StageEdge> edgesToOptimize)
Default constructor for ParallelismProphet.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.Map<java.lang.String,java.lang.Long>
calculate()
Launch SimulationScheduler and find out the optimal parallelism.
-
-
-
Constructor Detail
-
ParallelismProphet
public ParallelismProphet(IRDAG irdag, PhysicalPlan physicalPlan, SimulationScheduler simulationScheduler, PhysicalPlanGenerator physicalPlanGenerator, java.util.Set<StageEdge> edgesToOptimize)
Default constructor for ParallelismProphet.- Parameters:
irdag
- current IRDAGphysicalPlan
- current PhysicalPlansimulationScheduler
- SimulationScheduler to launchphysicalPlanGenerator
- PhysicalPlanGenerator to make physical plan which will be launched by simulation scheduleredgesToOptimize
- edges to optimize at runtime pass
-
-
Method Detail
-
calculate
public java.util.Map<java.lang.String,java.lang.Long> calculate()
Launch SimulationScheduler and find out the optimal parallelism. For now, the number of candidate parallelisms is seven, so we iterate seven times. In each iteration index i, the candidate parallelism is calculated by dividing the i-th power of two from partitonerProperty (which is guaranteed to be one of 1024, 2048, 4096. For more information, please refer to SamplingTaskSizingPass). This approach is taken to guarantee the equal length of each partition, which will be updated in DynamicTaskSizingRuntimePass.
-
-