Refactor StreamWriter to use temp files for constant memory usage

[Nebu1eto]

Summary

Refactor StreamWriter to write row data directly to temporary files on disk instead of accumulating rows in memory. This enables writing sheets with millions of rows while maintaining constant memory usage, regardless of the number of rows written.

Key Changes

• Streaming to disk: Replace in-memory Vec<Row> with a BufWriter writing to a tempfile::NamedTempFile. Each row is serialized to XML and written immediately.

• Inline strings instead of SST: Change from shared string table references to inline strings (<is><t>...</t></is>). This eliminates the need for SST index remapping when applying the stream writer to a workbook and allows each row to be serialized independently.

• New StreamedSheetData struct: Introduced to hold the temporary file and pre-built XML fragments (sheetViews, cols, mergeCells) for later composition during save.

• API changes:

  • finish() → into_streamed_data() returns (sheet_name, StreamedSheetData) instead of XML bytes
  • Removed into_parts() and into_worksheet_parts() methods
  • Removed sst() getter since streamed sheets no longer use the shared string table

• New helper functions:

  • build_worksheet_header() and build_worksheet_footer() to compose the full worksheet XML
  • write_streamed_sheet() to write streamed data directly to the ZIP archive during save
  • build_row_xml() to serialize individual rows with inline strings
  • xml_escape_into() for efficient XML escaping

• Workbook integration:

  • Added streamed_sheets: HashMap<usize, StreamedSheetData> to store streamed data
  • Updated apply_stream_writer() to store streamed data instead of merging SST entries
  • Updated sheet deletion to properly clean up streamed data
  • Modified save logic to stream rows from temp files directly to ZIP

• Documentation updates: Updated API docs and architecture docs to reflect that streamed sheets use inline strings and that cell values cannot be read before save/reopen.

• Test updates: Updated tests to verify inline string usage and added tests for save/reopen roundtrips.

Implementation Details

• StreamWriter now implements Send via unsafe impl since the temp file is exclusively owned
• Row XML is built incrementally with string concatenation for efficiency
• Temp files are automatically cleaned up when StreamedSheetData is dropped
• The worksheet header/footer are built once during into_streamed_data() and reused during save
• Freeze panes, column widths, and merge cells are pre-serialized to XML fragments before consuming the writer

https://claude.ai/code/session_012QobdGuj2a6P915GgXArXN

[claude]

Code review

Found 1 issue that should be addressed:

CLAUDE.md compliance issue in crates/sheetkit-core/Cargo.toml

Issue: The CLAUDE.md Tech Stack section lists tempfile as a testing utility:

**tempfile**, **pretty_assertions**: Testing utilities

However, this PR moves tempfile from dev-dependencies to regular dependencies because StreamWriter now uses temp files at runtime (see StreamedSheetData).

Recommendation: Update the CLAUDE.md Tech Stack section to reflect that tempfile is now a runtime dependency for the streaming writer functionality. For example:

- **tempfile**: Temporary file handling for StreamWriter  
- **pretty_assertions**: Testing utilities

Reference: CLAUDE.md Tech Stack section

---

Otherwise, the PR looks good. No bugs or other compliance issues found.

[Nebu1eto]

Thanks for the substantial StreamWriter rewrite and test expansion. I rebuilt and reran both Rust and Node test suites before reviewing (cargo build --workspace, cargo test --workspace, pnpm build, pnpm test) and found three blocking behavioral issues.

1. [P1] Formula cached result types are not serialized correctly

   • File: crates/sheetkit-core/src/stream.rs
   • In CellValue::Formula { result: ... }, cached String / Bool / Error results are written as <v>...</v> without setting the corresponding cell type (t="str", t="b", t="e").
   • On reopen, xml_cell_to_value uses t to decode formula results. This causes:
     • bool result -> decoded as Number(1.0)
     • string/error result -> dropped (result: None)

2. [P1] Edits after apply_stream_writer are silently ignored on save

   • File: crates/sheetkit-core/src/workbook/io.rs
   • Save path always prioritizes streamed_sheets for that index and writes temp-file payload directly.
   • Any later mutation on the same sheet (e.g. set_cell_value) updates the placeholder WorksheetXml, but that data is never serialized because streamed data wins.
   • Repro: write old via stream, call set_cell_value("S", "A1", "new"), save+reopen => value is still old.

3. [P1] copy_sheet does not copy streamed payload

   • File: crates/sheetkit-core/src/workbook/sheet_ops.rs
   • copy_sheet clones worksheet metadata/parallel vectors, but does not clone source entry from streamed_sheets.
   • For streamed source sheets, copied target sheet saves as empty (placeholder only).
   • Repro: stream-write Src!A1 = "x", copy_sheet("Src", "Dst"), save+reopen => Src!A1 = "x", Dst!A1 = Empty.

Given these data integrity issues, this should be fixed before merge.