Skip to content

Add WASM support foundation: IPageStorage/IWriteAheadLog abstractions + in-memory backends#62

Merged
mrdevrobot merged 3 commits intomainfrom
copilot/add-wasm-support
Apr 16, 2026
Merged

Add WASM support foundation: IPageStorage/IWriteAheadLog abstractions + in-memory backends#62
mrdevrobot merged 3 commits intomainfrom
copilot/add-wasm-support

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 14, 2026

BLite's storage layer was tightly coupled to PageFile (memory-mapped files) and WriteAheadLog (FileStream), both unavailable in browser WASM environments. This PR decouples the engine from its concrete I/O backends by introducing pluggable interfaces, adds in-memory implementations usable in WASM today, and documents the remaining roadmap as discrete sub-issues.

New interfaces

  • IPageStorage — extracted from PageFile's public surface; all storage backends implement this
  • IWriteAheadLog — extracted from WriteAheadLog's public surface; all WAL backends implement this

In-memory backends (zero file-system dependencies)

  • MemoryPageStorageConcurrentDictionary<uint, byte[]>-backed page store; suitable for WASM, unit tests, and ephemeral caches
  • MemoryWriteAheadLogList<WalRecord>-backed WAL; supports full ReadAll() / TruncateAsync() semantics

Engine changes

  • PageFile and WriteAheadLog now implement their respective interfaces — no behavioral changes
  • StorageEngine internal fields changed to IPageStorage / IWriteAheadLog; new pluggable constructor added:
    new StorageEngine(IPageStorage pageStorage, IWriteAheadLog wal)
  • BLiteEngine.CreateInMemory() factory — no files created, data lives in process memory:
    using var engine = BLiteEngine.CreateInMemory();          // default 16 KB pages
    using var engine = BLiteEngine.CreateInMemory(pageSize: 8192);
  • DocumentDbContext(StorageEngine, BLiteKvOptions?) protected constructor — enables typed context subclasses to use any backend

Roadmap

WASM_SUPPORT.md breaks the remaining work into 5 sub-issues:

  1. OPFS page storage backend (highest browser throughput)
  2. IndexedDB page storage backend (universal compatibility fallback)
  3. Persistent browser WAL (OpfsWriteAheadLog / IndexedDbWriteAheadLog)
  4. BLite.Wasm NuGet package + auto-detection factory
  5. Blazor WASM sample + docs

Copilot AI linked an issue Apr 14, 2026 that may be closed by this pull request
Copilot AI and others added 2 commits April 14, 2026 22:59
…ateInMemory factory, and WASM roadmap

Agent-Logs-Url: https://github.com/EntglDb/BLite/sessions/e4b3c63e-3286-4d25-9134-1f34c9e7100d

Co-authored-by: mrdevrobot <12503462+mrdevrobot@users.noreply.github.com>
…eoutMs in Dispose, improve test variable name

Agent-Logs-Url: https://github.com/EntglDb/BLite/sessions/e4b3c63e-3286-4d25-9134-1f34c9e7100d

Co-authored-by: mrdevrobot <12503462+mrdevrobot@users.noreply.github.com>
Copilot AI changed the title [WIP] Add WASM support for Blitedb in the browser Add WASM support foundation: IPageStorage/IWriteAheadLog abstractions + in-memory backends Apr 14, 2026
Copilot AI requested a review from mrdevrobot April 14, 2026 23:04
@mrdevrobot mrdevrobot marked this pull request as ready for review April 16, 2026 22:58
Copilot AI review requested due to automatic review settings April 16, 2026 22:58
@mrdevrobot mrdevrobot merged commit 5625152 into main Apr 16, 2026
3 checks passed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR lays the groundwork for WASM support by decoupling BLite’s storage engine from file-system–dependent I/O (PageFile, WriteAheadLog) via pluggable abstractions, and adding in-memory backends plus a BLiteEngine.CreateInMemory() factory.

Changes:

  • Introduces IPageStorage / IWriteAheadLog interfaces and updates PageFile / WriteAheadLog to implement them.
  • Adds MemoryPageStorage and MemoryWriteAheadLog in-memory implementations and a BLiteEngine.CreateInMemory() factory path.
  • Adds integration/unit tests for the in-memory storage stack and a WASM roadmap document.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/BLite.Tests/InMemoryStorageTests.cs New unit/integration tests covering in-memory page storage, WAL, and CreateInMemory().
src/BLite.Core/Transactions/WriteAheadLog.cs Implements IWriteAheadLog (no functional change intended).
src/BLite.Core/Transactions/MemoryWriteAheadLog.cs Adds in-memory WAL implementation for non-filesystem environments.
src/BLite.Core/Transactions/IWriteAheadLog.cs New WAL abstraction interface.
src/BLite.Core/Storage/StorageEngine.cs Switches internal fields to abstractions and adds constructor for injected backends.
src/BLite.Core/Storage/StorageEngine.Memory.cs Updates multi-file routing helpers to return IPageStorage.
src/BLite.Core/Storage/PageFile.cs Implements IPageStorage (no functional change intended).
src/BLite.Core/Storage/MemoryPageStorage.cs Adds in-memory page storage backend.
src/BLite.Core/Storage/IPageStorage.cs New page storage abstraction interface.
src/BLite.Core/DocumentDbContext.cs Adds protected constructor taking a pre-built StorageEngine.
src/BLite.Core/BLiteEngine.cs Adds internal constructor + CreateInMemory() factory.
WASM_SUPPORT.md Adds a WASM support roadmap and design notes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +119 to +120
{
cancellationToken.ThrowIfCancellationRequested();
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReadPageAsync slices destination.Span to _pageSize without first validating that destination.Length >= _pageSize. If a caller passes a too-small buffer, this throws an ArgumentOutOfRangeException instead of the consistent ArgumentException used by ReadPage, and the error message won’t match the contract in IPageStorage. Add the same length check as ReadPage (and ideally also call ThrowIfDisposed() here for consistency).

Suggested change
{
cancellationToken.ThrowIfCancellationRequested();
{
ThrowIfDisposed();
cancellationToken.ThrowIfCancellationRequested();
if (destination.Length < _pageSize)
throw new ArgumentException($"Destination must be at least {_pageSize} bytes.");

Copilot uses AI. Check for mistakes.
Comment on lines +161 to +179
if (_lock.Wait(_writeTimeoutMs))
{
try
{
_records.Clear();
_disposed = true;
}
finally
{
_lock.Release();
_lock.Dispose();
}
}
else
{
_records.Clear();
_disposed = true;
_lock.Dispose();
}
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dispose() falls back to _records.Clear() when it can’t acquire _lock within the timeout. Clearing the underlying List<WalRecord> without holding the semaphore can race with concurrent writers that do hold the lock, potentially corrupting the list or throwing. Prefer an unconditional lock acquisition during dispose (or skip clearing if the lock can’t be obtained) so _records is never mutated concurrently.

Suggested change
if (_lock.Wait(_writeTimeoutMs))
{
try
{
_records.Clear();
_disposed = true;
}
finally
{
_lock.Release();
_lock.Dispose();
}
}
else
{
_records.Clear();
_disposed = true;
_lock.Dispose();
}
_lock.Wait();
try
{
_records.Clear();
_sizeBytes = 0;
_disposed = true;
}
finally
{
_lock.Release();
_lock.Dispose();
}

Copilot uses AI. Check for mistakes.
Comment on lines +306 to +313
public void CreateInMemory_NoFileCreated()
{
// Verify that using an in-memory engine doesn't create any .db files.
var tempDir = Path.GetTempPath();
var before = Directory.GetFiles(tempDir, "*.db").Length;
using (var engine = BLiteEngine.CreateInMemory()) { }
var after = Directory.GetFiles(tempDir, "*.db").Length;
Assert.Equal(before, after);
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CreateInMemory_NoFileCreated is likely to be flaky because it counts *.db files in the global temp directory; unrelated processes/tests can create or delete .db files between the before and after snapshots, causing spurious failures. Consider removing this assertion or rewriting it to use an isolated temporary directory / file-system abstraction so the test only observes files created by this test.

Suggested change
public void CreateInMemory_NoFileCreated()
{
// Verify that using an in-memory engine doesn't create any .db files.
var tempDir = Path.GetTempPath();
var before = Directory.GetFiles(tempDir, "*.db").Length;
using (var engine = BLiteEngine.CreateInMemory()) { }
var after = Directory.GetFiles(tempDir, "*.db").Length;
Assert.Equal(before, after);
public void CreateInMemory_CanBeCreatedAndDisposed()
{
// Avoid asserting over the global temp directory because unrelated
// processes or parallel tests can create/remove .db files there and
// make the test flaky. This still verifies that the in-memory engine
// can be constructed and disposed successfully.
using var engine = BLiteEngine.CreateInMemory();
Assert.NotNull(engine);

Copilot uses AI. Check for mistakes.
public StorageEngine(IPageStorage pageStorage, IWriteAheadLog wal)
{
_config = PageFileConfig.Default;
_pageFile = pageStorage ?? throw new ArgumentNullException(nameof(pageStorage));
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new constructor doesn’t assign several readonly fields declared on StorageEngine (e.g. _indexFile, _collectionFiles, _collectionNameToSlot, _collectionSlotToName, _slotsFilePath). In C#, unassigned readonly fields cause a compile error (CS0171). Explicitly initialize these to null (and any other readonly fields not applicable in single-backend mode) within this constructor.

Suggested change
_pageFile = pageStorage ?? throw new ArgumentNullException(nameof(pageStorage));
_pageFile = pageStorage ?? throw new ArgumentNullException(nameof(pageStorage));
_indexFile = null;
_collectionFiles = null;
_collectionNameToSlot = null;
_collectionSlotToName = null;
_slotsFilePath = null;

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add WASM Support

3 participants