Nemo Release 0.2
Mar 9, 2020 • Wooyeon Lee
Release Notes - Apache Nemo - Version 0.2
Sub-task
- [NEMO-350] - Implement Off-heap SerializedMemoryStore
- [NEMO-385] - Support Lambda Pass with lambda policy and lambda resource property
- [NEMO-387] - Support Lambda scheduler
Bug
- [NEMO-80] - SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”
- [NEMO-304] - Fail-fast for mis-configuration in user application
- [NEMO-321] - Fix the data skew pass metric mismatch
- [NEMO-327] - Fix skew handling for multi shuffle edge receiver
- [NEMO-357] - Fix broken link on README
- [NEMO-368] - NEMO-353 breaks the application from running in YARN environments
- [NEMO-402] - Broken guava version conflicts cause ERROR: Trying to remove a RunningJob that is unknown
- [NEMO-413] - Fix index checking for byte access of MemoryChunk using UNSAFE
- [NEMO-414] - Command-line specified runtime data plane configurations not applied
- [NEMO-416] - Guava vendor version conflict when deserializing Task object
- [NEMO-420] - OffHeapMemory configuration only supports a single type of executor
New Feature
- [NEMO-335] - Using a database for recording metric data
- [NEMO-336] - Cost prediction using the metric data
- [NEMO-359] - implementation of getEstimatedSizeBytes in SourceVertex
- [NEMO-360] - Implementing an 'XGBoostPolicy'
- [NEMO-388] - Off-heap memory management (reuse ByteBuffer)
Improvement
- [NEMO-11] - Generalize Equality of Int Predicates for Loops
- [NEMO-162] - Add insertVertex() API in optimization pass
- [NEMO-182] - Consider reshaping in run-time optimization
- [NEMO-253] - Refactor getInternal(Main/Additional)OutputMap in TaskExecutor
- [NEMO-275] - Eager Garbage Collection for GroupByKey
- [NEMO-319] - Fix path to beam resources in examples in README
- [NEMO-320] - Make WebUI scale to big workloads
- [NEMO-323] - Upgrade current working version to 0.2-SNAPSHOT
- [NEMO-328] - Refactor IRDAG
- [NEMO-332] - Refactor RunTimePass
- [NEMO-337] - IRDAG Unit Tests
- [NEMO-338] - SkewSamplingPass
- [NEMO-340] - SonarCloud for PRs
- [NEMO-353] - Launch NEXMark applications
- [NEMO-356] - Visualize the name of beam transform in DAG
- [NEMO-358] - Recycling vertex ids while cloning a vertex
- [NEMO-369] - DirectByteArrayOutputStream usage refactoring
- [NEMO-390] - Address SonarCloud issues for the IR package
- [NEMO-391] - Set GrpcMessageEnvironment as a default implementation
- [NEMO-395] - Address SonarCloud issues for the scheduler package
- [NEMO-404] - Provide user argument to use lambda executor representer
- [NEMO-412] - Address Sonar Cloud issue for MemoryChunk
- [NEMO-418] - BlockFetchFailureProperty
- [NEMO-424] - Fix Sonarcloud bugs regarding Optional
- [NEMO-428] - Ignore .factorypath for rat check and version control
Task
- [NEMO-25] - Improve WebUI to use RESTful APIs by Nemo
- [NEMO-153] - IR-based dynamic optimization for WordCount application
- [NEMO-154] - Handle skewness information in SchedulingConstraint
- [NEMO-193] - Revised version of IR-based dynamic optimization
- [NEMO-322] - Committer's guide
- [NEMO-361] - Consistency on indentations
- [NEMO-362] - Upgrade of checkstyle version
- [NEMO-364] - Upgrade Beam
- [NEMO-375] - Add option to turn off metric collection to DB
- [NEMO-379] - Change javadoc goal to a proper one
- [NEMO-383] - Implement DirectByteBufferOutputStream for Off-heap SerializedMemoryStore
- [NEMO-384] - Implement DirectByteBufferInputStream for Off-heap SerializedMemoryStore
- [NEMO-397] - Separation of JVM heap region and off-heap memory region
- [NEMO-399] - Include the official WordCount example on the Beam website
- [NEMO-400] - Javadoc compile error
- [NEMO-421] - Release v0.2
- [NEMO-437] - Support Java version 11
Umbrella
- [NEMO-60] - IR-based dynamic optimization
- [NEMO-156] - Support Beam Nemo Runner
- [NEMO-157] - Support Nemo Streaming
- [NEMO-158] - Support Spark SQL Example
- [NEMO-159] - Nemo Web UI
List of Contributors
Nemo 0.2 was the work of many contributors:
- Arun Lakshman R
- Byung-Gon Chun
- Davor Bonaci
- Eunji Jeong
- Geon Woo Kim
- Gyewon Lee
- Haeyoon Cho
- Jae Hyeon Park
- JangHo Seo
- Jangho Seo
- Jean-Baptiste Onofré
- Jeongyoon Eo
- John Yang
- Joo Yeon Kim
- Kenn Knowles
- Markus Weimer
- Minhyeok Kweun
- Sanha Lee
- Seonghyun Park
- Soojeong Kim
- Taegeon Um
- Won Wook SONG
- Wooyeon Lee
- Yunseong Lee
- Older
- Newer