Skip to content

v1.16.0 - 5M-on-one-node bucket backend

Choose a tag to compare

View all tags
@benzsevern benzsevern released this 18 May 23:51
· 610 commits to main since this release
v1.16.0
a931ca6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

5M dedupe in 9.94 min wall, 6.4 GB peak RSS on a 16c/64GB Linux runner. New backend=bucket replaces the chunked path that was hanging at 63 GB plateau on the same fixture. Distributed Plan v1 ray auto-pick soft-reverted (pass GOLDENMATCH_ENABLE_DISTRIBUTED_RAY=1 to opt back in). See CHANGELOG.md and examples/at_scale_bucket_backend.py for details. PRs #310-#326.

Assets 2
Loading