Sub-task of #479. Write side, cgns backend — hardest write task.
Goal
Make the cgns save_to_disk write path incremental/remote-aware so peak local disk drops to ~1 sample instead of the whole dataset.
Current state (verified on main)
cgns writes file-per-sample via Sample.save_to_dir → pyCGNS CGM.save (native C file handle, local file required).- No fsspec store abstraction on the write side.
Constraint
Like the read side (#488), pyCGNS needs a real local file. Remote write therefore means: write each sample to a small per-sample local temp buffer, upload it via fsspec to the target, then discard — still bounded peak disk (~1 sample), consistent with the #479 open questions.
Scope
- Per-sample temp write → fsspec upload → discard loop.
- fsspec handle management under
num_proc > 1 (process-safe per worker).
- Optional protocol deps behind a
[remote] extra with a clear error when missing.
Tests
memory:// target: write a small dataset, read it back and compare.
Priority
★★★ (5/5) — most invasive write task; do after the zarr write path (#485) is validated as the pattern.
Sub-task of #479. Write side,
cgnsbackend — hardest write task.Goal
Make the
cgnssave_to_diskwrite path incremental/remote-aware so peak local disk drops to ~1 sample instead of the whole dataset.Current state (verified on
main)cgnswrites file-per-sample viaSample.save_to_dir→ pyCGNSCGM.save(native C file handle, local file required).Constraint
Like the read side (#488), pyCGNS needs a real local file. Remote write therefore means: write each sample to a small per-sample local temp buffer, upload it via fsspec to the target, then discard — still bounded peak disk (~1 sample), consistent with the #479 open questions.
Scope
num_proc > 1(process-safe per worker).[remote]extra with a clear error when missing.Tests
memory://target: write a small dataset, read it back and compare.Priority
★★★ (5/5) — most invasive write task; do after the zarr write path (#485) is validated as the pattern.