Nemo Release 0.4

Sep 10, 2022 • Won Wook SONG

Release Notes - Apache Nemo - Version 0.4

Here’s a brief summary about the release:

  • This release fixes a few high-priority bugs in 0.3 and has a variety of smaller fixes.
  • The release also includes several major functionalities regarding stream processing.
  • Installation script for nemo prerequisites for easier usage by new users
  • Cleanup of unused stream metrics
  • Addition of critical stream metrics (e.g. latency, throughput)
  • Addition of examples of stream applications
  • Scripts for network profiling within the Nemo cluster
  • Bump library versions to react to security risks and keep libraries up-to-date with new features
  • Fix the heisenbug regarding communication through stream pipe channels
  • Upgrading the CI with new platforms

Sub-task

  • [NEMO-398] - ExecutorRepresenter interface and LambdaExecutorRepresenter

Bug

  • [NEMO-299] - WindowedWordCountITCase Hangs (Heisenbug)
  • [NEMO-460] - Setting coders in CombinePerKey transformation

New Feature

Improvement

  • [NEMO-394] - Exchange data via shared memory when two tasks are in the same Executor in streaming
  • [NEMO-429] - Dec 2nd, 2019 Code Session
  • [NEMO-434] - Logical DAG modification for Dynamic sampling of task metrics during the execution of a stage
  • [NEMO-436] - Dynamic re-configuration based on the Sampled Metric Data

Task

  • [NEMO-422] - SonarCloud issues
  • [NEMO-440] - Migrate to Java11 and use Java 11 Features
  • [NEMO-482] - Keep library versions updated

Umbrella

  • [NEMO-483] - Record Metrics associated with stream processing