Skip to content

fix: bound hierarchical scan scratch ranges - #16

Merged
SamJSui merged 2 commits into
mainfrom
fix/scan-scratch-binding-ranges
Aug 8, 2026
Merged

fix: bound hierarchical scan scratch ranges#16
SamJSui merged 2 commits into
mainfrom
fix/scan-scratch-binding-ranges

Conversation

@SamJSui

@SamJSui SamJSui commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • bind hierarchical scan data and auxiliary storage views to exact logical byte ranges
  • add a 4,194,305-item regression that forces a third high-end hierarchy level
  • publish the previously withheld 100M scan and compaction comparisons on RTX and both Jetsons
  • update the roadmap now that the correctness blocker is closed

Root cause

Deeper hierarchy levels place data and auxiliary at different offsets in one scratch buffer. Both bindings previously used size: None, so the data view extended through the next auxiliary level. Rounded-up WGSL threads used that larger arrayLength(&data) and could overwrite the next level, producing a scheduling-sensitive writable-storage race.

The fix bounds data to num_items * 4 bytes and auxiliary to ceil(num_items / items_per_block) * 4 bytes. It changes views only; it does not add allocation or copies.

Validation

  • all three captured 100M scan and compaction processes pass on RTX, 8-TPC dopey, and 4-TPC grumpy
  • 10M controls changed by -0.25% to -1.39%; no measured regression
  • cargo fmt --all --check
  • cargo test --lib --tests
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo doc --no-deps
  • cargo package --allow-dirty
  • JSON parse and git diff --check

Performance finding

Correctness is closed, but valid 100M data confirms the next optimization target. Massively is 1.95x faster for scan and 1.49x for compaction on RTX, and 2.03x/1.71x faster on the 4-TPC Jetson.

@SamJSui
SamJSui marked this pull request as ready for review August 8, 2026 22:41
@SamJSui
SamJSui merged commit b489192 into main Aug 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant