Skip to content

v0.12.0

Choose a tag to compare

@github-actions github-actions released this 12 Oct 19:58
· 145 commits to main since this release
v0.12.0
4be1631

Release 0.12.0: scripts/bump_version.py 0.12.0 moves 26 manifests in lockstep, regenerates Cargo.lock and sets the CITATION.cff date; the file set equals the 0.11.2 bump (#1430). This body is the release-notes source for scripts/tag_release.py 0.12.0 --notes-from-pr <N>. The range is v0.11.2..main at a81046f25 (31 commits).

Fixed

Validated ordered reads on the 64-bit map could return a key that was never present, or skip one (#1466). MapReader next_* / prev_*, range_cursor and the batch cursor sampled a linear branch's version after loading its header. An optimistic insert of a new digit shifts a linear branch's edges and rewrites its header in place under that branch's version alone, so a write completing between the header load and the version sample left the walk with the old digits, the new even version and the shifted edges. A digit then paired with its neighbour's child edge, and the walk's validation passed. The set form shares the walk (sync_nav::next_in / prev_in) and is fixed with it. A deterministic test reproduces it on the unfixed code (MapReader::next_at_or_after(0x2100) returned 0x3040, the model answer being 0x3001; prev_at_or_before(0x2FFF) returned 0x1040, the model answer being 0x1021).

  • Affected: every release that contains the walk. next_in / prev_in loaded the header first in v0.7.0 (sync_nav.rs), which is the first tag containing #940; git tag --contains b868fb2eb lists v0.7.0 through v0.11.2. The batch cursor site (sync_cursor.rs) is present from v0.11.0.
  • Not affected, per the PR body: the plain point read, the writers (they sample first), and the 32-bit reader (one tree-level version is sampled before its walk).
  • Cost of the corrected order, from #1466's instruction-counts run (Callgrind, 251 arms, 0 regressed by the gate, 243 No change; valgrind 3.27.1 on an x86_64 Linux host; base 72bf75a7a, head 7c3af928a). The eight arms that moved:
arm main PR change
sync_map_next_after_scan sequential 35,051,479 35,301,486 +0.713%
sync_map_next_after_scan clustered 26,221,848 26,299,706 +0.297%
sync_map_scan random 10,247,031 10,225,628 -0.209%
sync_strmap_scan_pointops paths 186,583,378 186,879,806 +0.159%
sync_strmap_scan_pointops paths_dense 175,902,481 176,149,241 +0.140%
sync_map_prev random 34,114,722 34,088,020 -0.078%
sync_map_scan sequential 4,540,683 4,541,471 +0.017%
sync_map_scan clustered 4,544,134 4,544,417 +0.006%

Every plain-tree arm and sync_map_get read No change. No wall-clock measurement was taken in #1466.

ExpanseStrMap ordered reads panic on a node left linked and empty (#1449). first, last, next_at_or_after, next_after, prev_at_or_before and prev_before ran map.first().expect(...) on a StrNode that an optimistic remove had emptied and whose prune had not yet gone through. A locked reader (with_locked, or the locked fallback of an optimistic reader) could enter that window and panic. These reads now pass over such a node. Callgrind in #1449: every arm No change except the two halves of sync_strmap_next_after_scan_locked, +0.04189% (paths) and +0.04905% (paths_dense).

Breaking changes

Verified in the .github/public-api/expanse-trie.txt diff against v0.11.2; no removal appears in it other than the replaced signatures below.

  1. Blob compare-exchange error type (#1461). ExpanseBlobMap::compare_exchange, ExpanseBlobMap::compare_exchange_meta, SyncExpanseBlobMap::compare_exchange and SyncExpanseBlobMap::compare_exchange_meta no longer return Err(current) for both a mismatch and an unstorable new. The Err type is now blobmap::CompareExchangeError<Option<..>>: Mismatch(current) when the current value is not expected, Unstorable(ArenaError) when it is and new was refused. Replacement: match both variants; a retry loop that feeds a Mismatch value back as expected and stops on Unstorable terminates. The enum implements Display and Error.
  2. ArenaError::NoMetaField is a new variant, returned by compare_exchange_meta for non-zero metadata offered to an absent key or an inline payload. ArenaError is #[non_exhaustive], so a match with a wildcard arm still compiles.
  3. occ_stats array widths. occ_stats::NAMES changes from [&str; 49] to [&str; 52] and occ_stats::snapshot() from [u64; 49] to [u64; 52]; Stat gains StrOrderedAnswers = 49, StrOrderedCrossLevel = 50 and StrOrderedOverflows = 51 (#1448). Code that names the array length must change it.

C ABI: include/Judy.h is unchanged (no diff in v0.11.2..main). include/expanse.h and crates/expanse-capi are also unchanged, so no expanse_* symbol, signature or status code moved. docs/COMPAT.md changes are documentation only (below).

Changed

  • A reader used with a map from another collector now panics (#1443; extended to the set and string readers by #1465 and #1467). DetachedMapReader (get, first, last, next_*, prev_*, cursor, range_cursor) and SyncMapCursor::new / range / range_bounds compare the reader's collector with the map's with Arc::ptr_eq and panic with "reader was registered on a different collector" on a mismatch. Before this, the mismatched pair returned normally. A caller that built a reader on one collector and used it with another map will now panic; the public API snapshot did not change for #1443. The Python binding's reader pairing passes the check (#1443 body). Per #1443, the C ABI exposes only readers bound to their own map.
  • Release workflows (#1432, #1433, #1434, #1462). central-publishing-maven-plugin in bindings/java/pom.xml goes from 0.4.0 to 0.9.0; Java release workflows pin Apache Maven 3.9.16 (checksum-verified) because runner images from 20261004.327.1 pre-install Maven 3.10.0; the Java recovery workflow (java.yml) pins its runner and actions, cleans the Central staging directory before mvn deploy, and skips a version Central already serves. In release.yml, the Maven deploy step is skipped when repo1.maven.org already serves the version (a re-run no longer re-deploys); an unset MAVEN_CENTRAL_TOKEN or HOMEBREW_TAP_DEPLOY_KEY now fails the step instead of warning; verify-registries also runs after a failed registry job, on release tags only. scripts/check_release_workflow.py pins each of these.

Added

  • Ordered bytes map bindings (#1397): expanse_ordered_bytesmap_* is bound in Java (ExpanseOrderedBytesMap, #1441), .NET (ExpanseOrderedBytesMap, #1445), Go (OrderedBytesMap, #1444), Python (ExpanseOrderedBytesMap, #1446) and Node (ExpanseOrderedBytesMap, #1447). The C functions are not new in this range.
  • Validated ordered reads on StrReader (#1448): first, last, next_at_or_after, next_after, prev_at_or_before, prev_before, each returning Option<(Vec<u8>, u64)>, with a locked fallback and three occ_stats counters. This is the point-read half of #1143; the batch cursor is not in this release.
  • Owned and detached readers with the identity check (#1194): SyncExpanseSet::owned_reader / detached_reader (OwnedSetReader, DetachedSetReader, contains; #1465) and SyncExpanseStrMap::owned_reader / detached_reader (OwnedStrReader, DetachedStrReader: get, contains and the six ordered reads; #1467) and SyncExpanseBytesMap<S>::owned_reader / detached_reader (OwnedBytesReader<S>, DetachedBytesReader: get and contains; #1470). All four types are Send and not Sync.
  • Membership: SyncExpanseMap::contains_key and MapReader::contains (#1442).
  • get_or_insert, get_or_insert_with and update over compare_exchange (#1194): SyncExpanseMap (#1451), SyncExpanseStrMap (#1452), SyncExpanseBytesMap (#1453; get_or_insert, get_or_insert_with), SyncExpanseBlobMap (#1464; get_or_insert, get_or_insert_with, update, each returning Result<_, ArenaError>).
  • Ordered bytes map read-modify-write: SyncExpanseOrderedBytesMap::compare_exchange and update, and OrderedBytesExclusive::compare_exchange (#1459).
  • 32-bit: Writer32<ExpanseMap32>::compare_exchange, update and get_or_insert (#1463).
  • Benchmark arms and methodology: Callgrind arms sync_strmap_scan_pointops/{paths,paths_dense} (#1448) and the string next_after per-element arm under with_locked (#1439); the set's ordered-read arms (set_nav, set_prev, sync_set_prev_locked, sync_set_next_after_scan_locked; #1468); insert-if-absent compare_exchange arms for the map, string map and bytes map (#1450).

Documentation

  • Arena-cap compaction stall documented in docs/design/large-values.md §6.3.1 and docs/COMPAT.md (#1460).
  • docs/COMPAT.md: the RMW row records that a future C blob form needs an unstorable status distinct from its mismatch status (#1461); the Send assertion list names the new reader types (#1465, #1467).
  • String read-set bound derived by enumeration, with an appended amendment to the §33 pre-registration (#1436).
  • docs/ARCHITECTURE.md states the ordered-read sampling order (#1466), the reader-to-map rule (#1443) and the string ordered reads (#1448).

Internal

  • Dead-code allows dropped and bytesmap helpers gated (#1437); unused expanse-wasm dependencies and retired script flags removed (#1438); chart-theme copy count corrected (#1435); SVG chart helpers shared (#1440).

Known limits

  • Insert refused at the blob arena's capacity cap can run a whole-arena compaction and stall; see docs/design/large-values.md §6.3.1 for the cost and how to avoid it.
  • StrCursor under with_locked still takes map.first()? in Walk::finish_descent (crates/expanse/src/strmap.rs). #1449's body records that a scan through a linked empty node can end early there; #1449 changed only the single-threaded ordered reads, and no commit in this range changes the cursor. Tracked in #1454.
  • The C ABI has no blob compare-exchange (docs/COMPAT.md, RMW row: follow-up).

Not in this release

  • The string batch cursor on StrReader (#1143 remains open).
  • The draft benchmark PR #1377 (string batch cursor scan scaling).
  • #1194 (Sync* operation gaps) remains open; docs/COMPAT.md lists the C ABI and the non-Rust bindings of the RMW entries as follow-up.

Verified by running

  • python3 scripts/bump_version.py --check 0.12.0: every manifest in lockstep at 0.12.0.
  • python3 scripts/check_release_workflow.py (17 call sites as written), python3 scripts/check_public_api.py (matches its snapshot, no diff), python3 scripts/check_abi_parity.py (149 C ABI symbols covered), cargo metadata --no-deps --locked (lockfile in step).
  • Mutation proof: with crates/expanse/Cargo.toml set to 0.11.9 in a temp copy, bump_version.py --check 0.12.0 exits 1 with crates/expanse/Cargo.toml (package.version): found '0.11.9', expected '0.12.0'.
  • Release canary: release.yml dispatched with dry_run=true on chore/release-0.12.0 at 074495e70: https://github.com/orieg/expanse/actions/runs/38273786145, conclusion success. The gate asserted the CI run it dispatched (https://github.com/orieg/expanse/actions/runs/38273830113, success); build, smoke and promote jobs succeeded; crates.io, Homebrew, Maven Central and verify-registries were skipped, as for a dry run.

Not verified

  • The real-tag paths cannot be exercised before a real tag: the registry publishes (including Maven Central and the Homebrew tap), and the Maven skip and unset-secret behaviour added by #1462, which sit behind github.event_name == 'push' and a non-pre-release tag. The dry_run canary does not reach them.
  • No release-candidate tag has been created.
  • The Callgrind figures above are copied from #1466's body and were not re-measured here.

claims: 36 verified, 5 derived, 3 labelled

What's Changed

Other Changes

  • ci(release): upgrade central plugin and clean staging before deploy (Refs #1431, #1425) by @orieg in #1432
  • ci(release): harden java recovery workflow and ensure central plugin 0.9.0 (Refs #1431) by @orieg in #1433
  • ci(release): pin Apache Maven 3.9.16 for Java release workflows (Refs #1431) by @orieg in #1434
  • docs(bounds): derive the string read-set bound by enumeration and amend the section 33 pre-registration (Refs #1143) by @orieg in #1436
  • chore(scripts): correct chart-theme copy count (Refs #1371) by @orieg in #1435
  • bench(callgrind): per-element next_after scan arm for the string map under with_locked (Refs #1143) by @orieg in #1439
  • chore(deps): remove unused expanse-wasm deps and retired script flags (Refs #1367) by @orieg in #1438
  • chore(trie): drop no-op dead_code allows, gate bytesmap helpers, fix doc placement (Refs #1367) by @orieg in #1437
  • refactor(scripts): share esc and save_svg across the SVG chart generators by @orieg in #1440
  • feat(java): bind expanse_ordered_bytesmap_* as ExpanseOrderedBytesMap (Refs #1397) by @orieg in #1441
  • feat(dotnet): bind expanse_ordered_bytesmap_* as ExpanseOrderedBytesMap (Refs #1397) by @orieg in #1445
  • feat(sync): SyncExpanseMap::contains_key and MapReader::contains (Refs #1194) by @orieg in #1442
  • fix(sync): reject a reader registered on another map's collector (Refs #1194) by @orieg in #1443
  • fix(strmap): ordered reads pass over a linked empty child node (Refs #1143) by @orieg in #1449
  • feat(go): bind the expanse_ordered_bytesmap_* surface as OrderedBytesMap (Refs #1397) by @orieg in #1444
  • feat(python): bind expanse_ordered_bytesmap_* as ExpanseOrderedBytesMap (Refs #1397) by @orieg in #1446
  • feat(node): bind expanse_ordered_bytesmap_* as ExpanseOrderedBytesMap (Refs #1397) by @orieg in #1447
  • bench(rmw): insert-if-absent compare_exchange arms for map, string map and bytes map (Refs #1194) by @orieg in #1450
  • feat(sync): validated ordered point reads on StrReader (Refs #1143) by @orieg in #1448
  • feat(sync): add SyncExpanseMap::get_or_insert and get_or_insert_with (Refs #1194) by @orieg in #1451
  • feat(sync): add SyncExpanseStrMap::get_or_insert and get_or_insert_with over compare_exchange (Refs #1194) by @orieg in #1452
  • feat(sync): add SyncExpanseBytesMap::get_or_insert and get_or_insert_with over compare_exchange (Refs #1194) by @orieg in #1453
  • docs(blob): document the arena-cap compaction stall and how to avoid it (Refs #1320) by @orieg in #1460
  • feat(sync): add SyncExpanseOrderedBytesMap::compare_exchange and update (Refs #1194) by @orieg in #1459
  • feat(blobmap)!: blob compare-exchange reports an unstorable new value apart from a mismatch (Refs #1398) by @orieg in #1461
  • ci(release): skip a Maven version already on Central, fail on unset channel secrets, and verify every registry (Refs #1425) by @orieg in #1462
  • feat(sync): add SyncExpanseBlobMap::get_or_insert, get_or_insert_with and update over compare_exchange (Refs #1194) by @orieg in #1464
  • feat(sync32): add Writer32::compare_exchange, update and get_or_insert (Refs #1194) by @orieg in #1463
  • feat(sync): add OwnedSetReader and DetachedSetReader with the reader identity check (Refs #1194) by @orieg in #1465
  • fix(sync): sample a linear branch's version before loading its header in the ordered walks by @orieg in #1466
  • feat(sync): add OwnedStrReader and DetachedStrReader with the reader identity check (Refs #1194) by @orieg in #1467
  • bench(sync): add set ordered-read Callgrind arms (Refs #900) by @orieg in #1468
  • chore(release): 0.12.0 by @orieg in #1469
  • feat(sync): add OwnedBytesReader and DetachedBytesReader with the reader identity check (Refs #1194) by @orieg in #1470

Full Changelog: v0.11.2...v0.12.0