TsavoriteLog: remove the 47-bit address space limit - #2014
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Removes TsavoriteLog’s dependency on the 47-bit LogAddress masking by pushing logical-address → page mapping into the allocator wrapper (IAllocator.GetPageOfAddress), and refactors scan iterators to inline allocator-specific address interpretation (main store masks read-cache bit; TsavoriteLog does not).
Changes:
- Refactors
ScanIteratorBaseintoScanIteratorBase<TAllocatorWrapper>and updates all scan iterators to pass the allocator wrapper for inlined address mapping. - Extends
IAllocatorwithGetPageOfAddressand removesIAllocatorCallbacks, shifting page-mapping and page-management calls onto the wrapper interface. - Adds a TsavoriteLog-focused test and a small TsavoriteLog helper to validate full logical-address-range behavior.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| website/docs/dev/tsavorite/storefunctions.md | Updates developer documentation to reflect the allocator interface cleanup and allocator-specific address interpretation. |
| libs/storage/Tsavorite/cs/test/test.hlog/TsavoriteLogAddressRangeTests.cs | Adds a regression test ensuring TsavoriteLog page mapping does not mask off bit 47. |
| libs/storage/Tsavorite/cs/src/core/TsavoriteLog/TsavoriteLogScanIterator.cs | Updates iterator inheritance and constructor to use the generic scan base with allocator wrapper injection. |
| libs/storage/Tsavorite/cs/src/core/TsavoriteLog/TsavoriteLog.cs | Adds an internal helper (AllocatorGetPage) to expose allocator page mapping for tests. |
| libs/storage/Tsavorite/cs/src/core/Allocator/TsavoriteLogAllocator.cs | Implements IAllocator.GetPageOfAddress without read-cache-bit masking for TsavoriteLog. |
| libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteScanIterator.cs | Switches to ScanIteratorBase<TAllocator> and passes allocator wrapper into the base constructor. |
| libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteAllocator.cs | Implements GetPageOfAddress using LogAddress.GetPageOfAddress (masked behavior). |
| libs/storage/Tsavorite/cs/src/core/Allocator/ScanIteratorBase.cs | Makes the scan base generic over allocator wrapper and uses wrapper-provided GetPageOfAddress in disk buffering paths. |
| libs/storage/Tsavorite/cs/src/core/Allocator/ObjectScanIterator.cs | Switches to ScanIteratorBase<TAllocator> and passes allocator wrapper into the base constructor. |
| libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocator.cs | Implements GetPageOfAddress using LogAddress.GetPageOfAddress (masked behavior). |
| libs/storage/Tsavorite/cs/src/core/Allocator/IAllocatorCallbacks.cs | Removes the now-redundant callbacks interface. |
| libs/storage/Tsavorite/cs/src/core/Allocator/IAllocator.cs | Adds AllocatePage/FreePage/OverflowPageCount/GetPageOfAddress to IAllocator and folds callback responsibilities into the main interface. |
| libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorBase.cs | Routes page computations through _wrapper.GetPageOfAddress to respect allocator-specific address interpretation. |
badrishc
approved these changes
Aug 3, 2026
- Fix 'Genric' -> 'Generic' typo in IAllocator XML doc comment. - Restructure TsavoriteLogAddressRangeTests to use [SetUp]/[TearDown] so cleanup runs even when an assertion fails, and TestUtils.OnTearDown runs the standard LightEpoch leak checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 721ce207-7e07-4a5d-9694-339ef88678c8
tiagonapoli
approved these changes
Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TsavoriteLog does not use RecordInfo so its addresses should not be bound by the 47-bit limit. This PR moves some address operations (logical -> physical, GetPageOfAddress, etc.) into the IAllocator rather than being on AllocatorBase.
Key changes include:
Scan Iterator Refactor
ScanIteratorBasegeneric over the allocator wrapper type (ScanIteratorBase<TAllocatorWrapper>) and updated all scan iterator implementations (ObjectScanIterator,SpanByteScanIterator,TsavoriteLogScanIterator) to inherit from the generic base class. This allows inlining of allocator-specific logic and eliminates virtual/interface dispatch in performance-critical paths. [1] [2] [3] [4] [5]Address Mapping Optimization
GetPageOfAddressto theIAllocatorinterface, implemented it in all allocator wrappers, and updated all usages in scan iterators and allocator base to use the inlined version from the allocator wrapper, further removing interface dispatch and improving performance. [1] [2] [3] [4] [5] [6] [7]Interface Cleanup
IAllocatorCallbacks<TStoreFunctions>interface and merged its relevant methods into the mainIAllocatorinterface, simplifying the type hierarchy and reducing indirection. [1] [2]Testing and Utility
AllocatorGetPagetoTsavoriteLogfor testing address mapping behavior, ensuring that TsavoriteLog uses the full logical address range as intended.These changes collectively improve the performance and maintainability of the scan path in Tsavorite by making address mapping logic more efficient and type-safe.