Package org.apache.nemo.compiler.optimizer.pass.runtime

Class SkewRunTimePass

java.lang.Object

org.apache.nemo.common.pass.Pass

org.apache.nemo.compiler.optimizer.pass.runtime.RunTimePass<java.util.Map<java.lang.Object,java.lang.Long>>

org.apache.nemo.compiler.optimizer.pass.runtime.SkewRunTimePass

All Implemented Interfaces:

java.util.function.BiFunction<IRDAG,Message<java.util.Map<java.lang.Object,java.lang.Long>>,IRDAG>

public final class SkewRunTimePass
extends RunTimePass<java.util.Map<java.lang.Object,java.lang.Long>>

Dynamic optimization pass for handling data skew. Using a map of key to partition size as a metric used for dynamic optimization, this RunTimePass identifies a number of keys with big partition sizes(skewed key) and evenly redistributes data via overwriting incoming edges of destination tasks.

Constructor Summary

Constructors

Constructor	Description
SkewRunTimePass()
SkewRunTimePass(int numOfSkewedKeys)

Method Summary

Modifier and Type	Method	Description
IRDAG	apply(IRDAG irdag, Message<java.util.Map<java.lang.Object,java.lang.Long>> message)

Methods inherited from class org.apache.nemo.common.pass.Pass

addCondition, getCondition

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.function.BiFunction

andThen

Constructor Detail

SkewRunTimePass

public SkewRunTimePass()

SkewRunTimePass

public SkewRunTimePass(int numOfSkewedKeys)

Method Detail

apply

public IRDAG apply(IRDAG irdag,
                   Message<java.util.Map<java.lang.Object,java.lang.Long>> message)