Skip to content

v0.7.0

Pre-release
Pre-release

Choose a tag to compare

View all tags
@zaynt4606 zaynt4606 released this 02 Aug 08:11
v0.7.0-rc0
d6005db

Apache Celeborn community is glad to announce the
new release candidate of Apache Celeborn 0.7.0-rc0

Highlight

  • Engine Support: Spark 4.1/4.2 and Flink 2.1/2.2/2.3 support; Spark 4.0 and Flink 2.0 client maturity. Removed Spark 2.4, Flink 1.16/1.17, and Hadoop 2 client support.
  • C++ Native Client (CIP-14): Full push/merge data path, revive, LZ4/ZSTD compression, retry logic, heartbeat decoding, native transport, and C++-write/Java-read hybrid integration tests — a major push toward feature parity with the Java client.
  • Encryption at Rest (CIP-22): Spark-side implementation landed.
  • Shuffle Integrity: End-to-end integrity check for Flink, iterator fully-consumed validation, and extended E2E checked zone.
  • Remote Storage (S3/OSS/HDFS): S3 client caching, credentials provider support, DFS replication factor configuration, MapPartitionData on DFS, and multiple robustness fixes.
  • Observability: Flush latency per storage tier, eviction counts, read buffer availability, metadata operation metrics, and more — all reflected in the Grafana dashboards.
  • Performance: Zero-copy sendfile in Netty native transports, parallelized open stream and commit paths, and reduced lock contention on hot paths.
  • Durability & Reliability: fsync on commit, automatic RocksDB restoration, disk-full rejection, Ratis stepdown and outbox retry, and numerous race-condition and resource-leak fixes.
  • Security: RPC authorization, Java deserialization filter, and configurable HTTP auth bypass paths.
  • Standalone LifecycleManager and Rust SDK, plus Helm chart improvements.

Download

https://dist.apache.org/repos/dist/dev/celeborn/v0.7.0-rc0/

Maven Staging Repository

https://repository.apache.org/content/repositories/orgapacheceleborn-1135

Assets 2
Loading