Class GBKTransform<K,​InputT,​OutputT>

  • Type Parameters:
    K - key type
    InputT - input type
    OutputT - output type
    All Implemented Interfaces:
    java.io.Serializable, Transform<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,​InputT>>,​org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,​OutputT>>>

    public final class GBKTransform<K,​InputT,​OutputT>
    extends AbstractDoFnTransform<org.apache.beam.sdk.values.KV<K,​InputT>,​org.apache.beam.runners.core.KeyedWorkItem<K,​InputT>,​org.apache.beam.sdk.values.KV<K,​OutputT>>
    This transform executes GroupByKey transformation and CombinePerKey transformation when input data is unbounded or is not in a global window.
    See Also:
    Serialized Form
    • Constructor Summary

      Constructors 
      Constructor Description
      GBKTransform​(org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.values.KV<K,​InputT>> inputCoder, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,​org.apache.beam.sdk.coders.Coder<?>> outputCoders, org.apache.beam.sdk.values.TupleTag<org.apache.beam.sdk.values.KV<K,​OutputT>> mainOutputTag, org.apache.beam.sdk.values.WindowingStrategy<?,​?> windowingStrategy, org.apache.beam.sdk.options.PipelineOptions options, org.apache.beam.runners.core.SystemReduceFn reduceFn, org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation, org.apache.beam.sdk.transforms.display.DisplayData displayData, boolean isPartialCombining)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected void beforeClose()
      This advances the input watermark and processing time to the timestamp max value in order to emit all data.
      boolean getIsPartialCombining()
      Accessor for isPartialCombining.
      void onData​(org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,​InputT>> element)
      Every time a single element arrives, this method invokes runner to process a single element.
      void onWatermark​(Watermark watermark)
      Trigger timers that need to be fired at watermark and emit output watermark.
      protected org.apache.beam.sdk.transforms.DoFn wrapDoFn​(org.apache.beam.sdk.transforms.DoFn doFn)
      This creates a new DoFn that groups elements by key and window.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Constructor Detail

      • GBKTransform

        public GBKTransform​(org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.values.KV<K,​InputT>> inputCoder,
                            java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,​org.apache.beam.sdk.coders.Coder<?>> outputCoders,
                            org.apache.beam.sdk.values.TupleTag<org.apache.beam.sdk.values.KV<K,​OutputT>> mainOutputTag,
                            org.apache.beam.sdk.values.WindowingStrategy<?,​?> windowingStrategy,
                            org.apache.beam.sdk.options.PipelineOptions options,
                            org.apache.beam.runners.core.SystemReduceFn reduceFn,
                            org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation,
                            org.apache.beam.sdk.transforms.display.DisplayData displayData,
                            boolean isPartialCombining)
    • Method Detail

      • wrapDoFn

        protected org.apache.beam.sdk.transforms.DoFn wrapDoFn​(org.apache.beam.sdk.transforms.DoFn doFn)
        This creates a new DoFn that groups elements by key and window.
        Parameters:
        doFn - original doFn.
        Returns:
        GroupAlsoByWindowViaWindowSetNewDoFn
      • onData

        public void onData​(org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,​InputT>> element)
        Every time a single element arrives, this method invokes runner to process a single element.
        Parameters:
        element - input data element.
      • onWatermark

        public void onWatermark​(Watermark watermark)
                         throws java.lang.RuntimeException
        Trigger timers that need to be fired at watermark and emit output watermark.
        Parameters:
        watermark - watermark
        Throws:
        java.lang.RuntimeException
      • beforeClose

        protected void beforeClose()
        This advances the input watermark and processing time to the timestamp max value in order to emit all data.
      • getIsPartialCombining

        public boolean getIsPartialCombining()
        Accessor for isPartialCombining.
        Returns:
        whether it is partial combining.