Skip to content

TsavoriteLog: remove the 47-bit address space limit - #2014

Merged
TedHartMS merged 8 commits into
mainfrom
tedhar/tlog-address
Aug 4, 2026
Merged

TsavoriteLog: remove the 47-bit address space limit#2014
TedHartMS merged 8 commits into
mainfrom
tedhar/tlog-address

Conversation

@TedHartMS

Copy link
Copy Markdown
Contributor

TsavoriteLog does not use RecordInfo so its addresses should not be bound by the 47-bit limit. This PR moves some address operations (logical -> physical, GetPageOfAddress, etc.) into the IAllocator rather than being on AllocatorBase.

Key changes include:

Scan Iterator Refactor

  • Made ScanIteratorBase generic over the allocator wrapper type (ScanIteratorBase<TAllocatorWrapper>) and updated all scan iterator implementations (ObjectScanIterator, SpanByteScanIterator, TsavoriteLogScanIterator) to inherit from the generic base class. This allows inlining of allocator-specific logic and eliminates virtual/interface dispatch in performance-critical paths. [1] [2] [3] [4] [5]
  • Updated constructors of all scan iterator classes to pass the allocator wrapper to the base class, ensuring that address interpretation is allocator-specific and inlinable. [1] [2] [3] [4] [5]

Address Mapping Optimization

  • Added a new method GetPageOfAddress to the IAllocator interface, implemented it in all allocator wrappers, and updated all usages in scan iterators and allocator base to use the inlined version from the allocator wrapper, further removing interface dispatch and improving performance. [1] [2] [3] [4] [5] [6] [7]

Interface Cleanup

  • Removed the IAllocatorCallbacks<TStoreFunctions> interface and merged its relevant methods into the main IAllocator interface, simplifying the type hierarchy and reducing indirection. [1] [2]

Testing and Utility

  • Added a helper method AllocatorGetPage to TsavoriteLog for testing address mapping behavior, ensuring that TsavoriteLog uses the full logical address range as intended.

These changes collectively improve the performance and maintainability of the scan path in Tsavorite by making address mapping logic more efficient and type-safe.

Copilot AI review requested due to automatic review settings August 3, 2026 18:12
@TedHartMS
TedHartMS requested a review from badrishc August 3, 2026 18:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes TsavoriteLog’s dependency on the 47-bit LogAddress masking by pushing logical-address → page mapping into the allocator wrapper (IAllocator.GetPageOfAddress), and refactors scan iterators to inline allocator-specific address interpretation (main store masks read-cache bit; TsavoriteLog does not).

Changes:

  • Refactors ScanIteratorBase into ScanIteratorBase<TAllocatorWrapper> and updates all scan iterators to pass the allocator wrapper for inlined address mapping.
  • Extends IAllocator with GetPageOfAddress and removes IAllocatorCallbacks, shifting page-mapping and page-management calls onto the wrapper interface.
  • Adds a TsavoriteLog-focused test and a small TsavoriteLog helper to validate full logical-address-range behavior.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
website/docs/dev/tsavorite/storefunctions.md Updates developer documentation to reflect the allocator interface cleanup and allocator-specific address interpretation.
libs/storage/Tsavorite/cs/test/test.hlog/TsavoriteLogAddressRangeTests.cs Adds a regression test ensuring TsavoriteLog page mapping does not mask off bit 47.
libs/storage/Tsavorite/cs/src/core/TsavoriteLog/TsavoriteLogScanIterator.cs Updates iterator inheritance and constructor to use the generic scan base with allocator wrapper injection.
libs/storage/Tsavorite/cs/src/core/TsavoriteLog/TsavoriteLog.cs Adds an internal helper (AllocatorGetPage) to expose allocator page mapping for tests.
libs/storage/Tsavorite/cs/src/core/Allocator/TsavoriteLogAllocator.cs Implements IAllocator.GetPageOfAddress without read-cache-bit masking for TsavoriteLog.
libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteScanIterator.cs Switches to ScanIteratorBase<TAllocator> and passes allocator wrapper into the base constructor.
libs/storage/Tsavorite/cs/src/core/Allocator/SpanByteAllocator.cs Implements GetPageOfAddress using LogAddress.GetPageOfAddress (masked behavior).
libs/storage/Tsavorite/cs/src/core/Allocator/ScanIteratorBase.cs Makes the scan base generic over allocator wrapper and uses wrapper-provided GetPageOfAddress in disk buffering paths.
libs/storage/Tsavorite/cs/src/core/Allocator/ObjectScanIterator.cs Switches to ScanIteratorBase<TAllocator> and passes allocator wrapper into the base constructor.
libs/storage/Tsavorite/cs/src/core/Allocator/ObjectAllocator.cs Implements GetPageOfAddress using LogAddress.GetPageOfAddress (masked behavior).
libs/storage/Tsavorite/cs/src/core/Allocator/IAllocatorCallbacks.cs Removes the now-redundant callbacks interface.
libs/storage/Tsavorite/cs/src/core/Allocator/IAllocator.cs Adds AllocatePage/FreePage/OverflowPageCount/GetPageOfAddress to IAllocator and folds callback responsibilities into the main interface.
libs/storage/Tsavorite/cs/src/core/Allocator/AllocatorBase.cs Routes page computations through _wrapper.GetPageOfAddress to respect allocator-specific address interpretation.

Comment thread libs/storage/Tsavorite/cs/src/core/Allocator/IAllocator.cs
Comment thread libs/storage/Tsavorite/cs/test/test.hlog/TsavoriteLogAddressRangeTests.cs Outdated
TedHartMS and others added 4 commits August 3, 2026 21:04
- Fix 'Genric' -> 'Generic' typo in IAllocator XML doc comment.
- Restructure TsavoriteLogAddressRangeTests to use [SetUp]/[TearDown]
  so cleanup runs even when an assertion fails, and TestUtils.OnTearDown
  runs the standard LightEpoch leak checks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 721ce207-7e07-4a5d-9694-339ef88678c8
@TedHartMS
TedHartMS merged commit 34d14a8 into main Aug 4, 2026
416 of 417 checks passed
@TedHartMS
TedHartMS deleted the tedhar/tlog-address branch August 4, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants