Grow into memory instead of reserving it, and raise the ceiling - #4
Merged
Conversation
The banks move off .bss to computed offsets, and the module grows linear memory to reach them. An instance starts at 1.25 MiB (u32) or 1.75 MiB (u64) instead of 21 or 29. The hash widens to 64 bits so h1 can address the whole slot space. The ceiling is 117,440,512 entries for u32 and 58,720,256 for u64. The bindings rebuild their cached views after any call that can rehash, since a grow detaches them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
The banks move off
.bssto computed offsets above the static data, and the module grows linear memory to reach them. An instance starts at 1.25 MiB (u32) or 1.75 MiB (u64) instead of 21 or 29, and grows with the table.The hash widens to 64 bits so
h1can address the whole slot space, which is what capped the table at2^20slots. The ceiling is now 117,440,512 entries for u32 and 58,720,256 for u64, up from 917,504. They differ because three u64 banks run out of wasm32 address space an exponent earlier.Placement
Bank 0 sits at the base of the heap, bank 1 two of its own lengths above it. Consecutive banks are always disjoint, which is all a rehash needs, and same-parity banks reuse committed pages. A
reserve()crossing several doublings at once produces a bank too large for the anchor to clear, so that one goes directly above the live bank.Address extent is 3x a bank on the odd index and 1.5x on the even one. The 3x case sets each module's ceiling, since three banks have to address inside 4 GiB. Resident memory is lower because same-parity banks share pages, measured at 1.8x to 1.9x a bank across a fill.
Bindings
A grow replaces the backing
ArrayBufferand detaches every view over it, so both bindings rebuild their cached views after any call that can rehash — including one that then fails, which is where the first draft was wrong. An open walk needs no such check: a grow only happens inside a rehash, and the walk already tests the generation counter before each window.Breaking-ish
Each module now declares a
--max-memoryof 3.4 GiB (u32) or 2.4 GiB (u64). That is address space, not a reservation, and nothing is committed until a table reaches it — but a 32-bit or memory-constrained host may decline it at instantiation where the previous 21 MiB succeeded. Noted inSECURITY.mdand the changelog. Build with a lowerSWISS_MAX_CAPACITY_LOG2for those hosts.Memory is never returned to the host.
clear()andshrinkToFit()ready a table for reuse, but an instance holds the high-water mark of every bank it used untildispose().Tests
Two test-only modules cover the refusal paths, since the shipped ceiling is no longer reachable in bounded time:
swiss_u32_capped(2^16 slots) for the slot ceiling, andswiss_u32_starved(2^20 slots against 8 MiB) for a host declining a grow. Neither is embedded or published.Verification
bun run build— all four modules linkbun run typecheck— cleanbun test— 252 pass, 0 failbun run check:ubsan— 905,504 operations, no trapBenchmarks reran (5 runtimes x 3 passes) and both charts were regenerated. Lookup is flat within noise, confirming the 64-bit hash costs nothing.