Skip to content

perf: offload async API methods to worker threads via spawn_blocking#51

Merged
Nebu1eto merged 4 commits into
mainfrom
perf/phase5-async-model
Feb 11, 2026
Merged

perf: offload async API methods to worker threads via spawn_blocking#51
Nebu1eto merged 4 commits into
mainfrom
perf/phase5-async-model

Conversation

@Nebu1eto

Copy link
Copy Markdown
Owner

Summary

  • Convert async API methods to use tokio::task::spawn_blocking() for CPU-heavy work
  • Frees the tokio async runtime threads for other concurrent operations
  • open(), openBuffer(), openWithPassword() fully offloaded to blocking thread pool
  • save(), saveWithPassword() partially offloaded (disk write + encryption on blocking pool)
  • writeBuffer() unchanged (requires &self, documenting the limitation)

Changes

  • packages/sheetkit/Cargo.toml: Add tokio = { version = "1", features = ["rt"] } direct dependency
  • packages/sheetkit/src/lib.rs: Convert 5 async methods to use spawn_blocking
  • packages/sheetkit/binding.d.ts: Regenerated with updated doc comments
  • packages/sheetkit/__test__/index.spec.ts: 8 new concurrency and correctness tests

Design Decisions

  • spawn_blocking over AsyncTask: Simpler, same result (offload to dedicated thread pool)
  • No TypeScript changes: napi interface unchanged, wrapper works as-is
  • writeBuffer not offloaded: Requires &self reference, cloning Workbook too expensive
  • save partial offload: Serialization needs &self, but disk I/O + encryption offloaded

Test plan

  • 8 new tests (async open/save/buffer parity, concurrent ops, error propagation)
  • 1,956 Rust tests pass
  • 494 Node.js tests pass (486 existing + 8 new)
  • clippy clean, fmt clean
  • Stacked on Phase 3 (Node decode optimization)

🤖 Generated with Claude Code

@claude

claude Bot commented Feb 11, 2026

Copy link
Copy Markdown

Code Review - CLAUDE.md Compliance Issues

Found 2 issues that should be addressed:

Issue 1: Misleading documentation for writeBuffer()

Location: packages/sheetkit/src/lib.rs:156-162

The doc comment claims serialization "runs on the tokio runtime thread" but the implementation just calls write_buffer_sync() without spawn_blocking().

CLAUDE.md requires: "Doc comments: Describe inputs, behavior, and outputs concisely."

See:

sheetkit/CLAUDE.md

Lines 56 to 57 in c956513

- Doc comments (`///` for Rust, `/** */` for TypeScript): Describe inputs, behavior, and outputs concisely.
- Inline comments: Only for complex logic that is not self-evident from the code.

Issue 2: Verbose doc comments with implementation details

Locations: packages/sheetkit/src/lib.rs (multiple methods)

Doc comments include threading implementation details instead of concisely describing behavior. Async methods should follow same concise style as sync equivalents.

Example - compare:

  • openSync: "Open an existing .xlsx file from disk."
  • open: Has 3 lines explaining worker threads and event loop

CLAUDE.md requires: "Doc comments: Describe inputs, behavior, and outputs concisely."

See:

sheetkit/CLAUDE.md

Lines 56 to 57 in c956513

- Doc comments (`///` for Rust, `/** */` for TypeScript): Describe inputs, behavior, and outputs concisely.
- Inline comments: Only for complex logic that is not self-evident from the code.

@Nebu1eto

Copy link
Copy Markdown
Owner Author

P2: Async save now serializes the entire workbook into memory via save_to_buffer() before writing to disk, while saveSync writes directly through the core save path. This changes memory behavior and can significantly increase peak memory / OOM risk for large workbooks.\n\nCould you either document this as an explicit limitation for async save, or redesign the async path to avoid full in-memory buffering?

@Nebu1eto
Nebu1eto force-pushed the perf/phase5-async-model branch from 8fd5dfc to 48e6bc0 Compare February 11, 2026 15:32
@Nebu1eto

Copy link
Copy Markdown
Owner Author

I think the new async encrypted-save memory note is still a bit misleading.

saveWithPassword docs currently imply higher peak memory than saveWithPasswordSync, but the sync encrypted path also buffers ZIP data in memory before encrypt/write (see core implementation around save_with_password).

Could we reword this to avoid implying that the sync encrypted path is lower-memory than async for encryption-specific saves?

@Nebu1eto

Copy link
Copy Markdown
Owner Author

The async save memory caveat was added in native binding docs, but the main public wrapper typings are generated from index.ts (types points to index.d.ts).

Could we mirror the same caveat in the index.ts docs for save / saveWithPassword so users see it in the primary API surface as well?

Base automatically changed from perf/phase3-node-decode to main February 11, 2026 15:57
Nebu1eto and others added 4 commits February 12, 2026 01:01
Previously, async methods (open, openBuffer, save, etc.) called their
sync counterparts directly, blocking the tokio runtime thread. Now:

- open(), openBuffer(), openWithPassword(): File read, parsing, and
  decryption run on tokio's blocking thread pool via spawn_blocking,
  freeing the Node.js event loop during the entire operation.

- save(), saveWithPassword(): Serialization still requires &self on the
  current thread, but the file write (and encryption for password-
  protected saves) is offloaded to a blocking worker thread.

- writeBuffer(): Kept as-is since the entire operation needs &self.
  Doc comment clarifies the behavior.

Adds tokio as a direct dependency (already a transitive dep via napi)
and 8 new tests for async correctness and concurrency safety.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove verbose threading implementation details from open/openBuffer/
  openWithPassword doc comments to match CLAUDE.md conciseness rule
- Fix misleading writeBuffer doc comment (was referencing "tokio runtime
  thread"); now accurately states it delegates to writeBufferSync
- Document async save/saveWithPassword memory limitation: workbook is
  serialized to an in-memory buffer before disk write, increasing peak
  memory vs the sync variants that stream directly to disk

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ypeScript API

The saveWithPassword doc incorrectly implied higher peak memory than
saveWithPasswordSync, but both paths buffer ZIP data in memory before
encrypting. Updated the Rust doc to accurately describe this. Also
added memory behavior notes to save() and saveWithPassword() in
index.ts so they appear in the generated .d.ts for end users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Nebu1eto
Nebu1eto force-pushed the perf/phase5-async-model branch from 903743f to 264a4a1 Compare February 11, 2026 16:05
@Nebu1eto
Nebu1eto merged commit fc13ff8 into main Feb 11, 2026
6 of 7 checks passed
@Nebu1eto
Nebu1eto deleted the perf/phase5-async-model branch February 11, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant