Skip to content

feat: protocol file sync — changes-since, chunked writes, package hash (M2b)#70

Merged
Yona-Appletree merged 1 commit into
mainfrom
feature/protocol-file-sync
Jul 14, 2026
Merged

feat: protocol file sync — changes-since, chunked writes, package hash (M2b)#70
Yona-Appletree merged 1 commit into
mainfrom
feature/protocol-file-sync

Conversation

@Yona-Appletree

Copy link
Copy Markdown
Member

Summary

Roadmap M2b: files on the wire, revision-gated and pull-model honest — the surface that load-as-push, save-as-pull, and connect-as-pull (M3) plus device push (M5) all share.

  • ChangesSince { prefix, since, cursor } — stateless cursor pagination rather than multi-frame streaming: each page is an ordinary single-frame response packed under the frame budget, and the client requests the next page only after fully receiving the previous one — which is the pull model's auto-throttle rule by construction, with zero server-side client state and zero transport special-casing. since = 0 → full enumeration (the change log needn't cover pre-tracking files); deletes are tombstones; every page carries the fs revision and clients adopt the first page's version as the next since (mid-pagination changes re-pull convergently, never lost).
  • WriteChunk — stateless chunked upload: offset == 0 truncates; offset > 0 validates against file_size and appends. Project-deploy writes are now chunk-aware, fixing the pre-existing oversized-single-Write hazard.
  • HashPackage — canonical lph1 package hash for end-to-end push/pull verification (lpa-server gains lpc-history; the hash travels as a hex string so lpc-wire stays independent).
  • LpFs gains append_file + file_size as default trait methods (native LpFsMemory overrides, LpFsView/LpFsOpfs delegation) — chunked writes are O(n) instead of O(n²) read-modify-write; LittleFS-native overrides are noted as M5 follow-ups.
  • Client: ChangesCollector fold + pure request builders (lpa-client::file_sync_ops), LpClient::{pull_changed_files, replace_project_files, hash_package}, and thin StudioServerClient wrappers ready for M3.
  • fw-browser executor (sans-IO ADR follow-up): the null-waker spin block_on is now a poll-once that panics loudly on Pending — dispatch futures are immediately-ready by construction, and the e2e suite runs on it.
  • Budget: FILE_SYNC_* constants derived beside the existing frame budget, with a compile-time fits-in-frame assertion and worst-case measurement tests against the firmware serializer (ser-write-json).

Committed JSON samples pin the new wire forms. No wire-compat shims (heavy-dev policy).

Plan: Planning/lp2025/2026-07-06-protocol-file-sync (M2b of the project-management roadmap; born from the D19/D20 "sim is an ephemeral place" reframe).

Test plan

  • lpc-wire 85 tests (serde samples incl. binary-goes-base64, budget measurements, compile-time chunk assert)
  • lpa-server contract suite (8 tests): since=current transfers nothing, full/incremental/empty pull cycle, tombstones, prefix isolation, multi-page large-file reassembly, entry caps, chunked-write offset validation, hash-vs-direct-computation
  • lpa-client 52 tests (collector reassembly/ordering, chunking boundaries) + lpa-studio-core 270 tests
  • fw-browser wasm e2e (headless Chrome): push project over protocol frames (chunked + mis-offset rejection) → hash-verify → LoadProject → paginated full pull byte-identical → exact delta after edit → empty after adopting the version
  • cargo check -p fw-esp32 (esp32c6,server) — handlers compile into firmware
  • lpa-fs-opfs wasm suites still green after the LpFs trait additions
  • just build-ci / just test green; clippy clean on all touched crates. Note: just check currently fails on main itself — pre-existing approx_constant errors in lps-builtins q32 test tables (task chip filed), unrelated to this change.

🤖 Generated with Claude Code

…h (M2b)

Files on the wire, revision-gated and pull-model honest, for sim and
device alike (load-as-push / save-as-pull / connect-as-pull foundations):

- FsRequest::ChangesSince { prefix, since, cursor }: stateless cursor
  pagination — each page an ordinary single-frame response packed under
  the frame budget; the client requests the next page after full receipt
  (the auto-throttle rule by construction, no server-side client state).
  since=0 is a full enumeration; deletes arrive as tombstones; every
  page carries the fs revision and clients adopt the FIRST page's as the
  next since (mid-pagination changes re-pull convergently).
- FsRequest::WriteChunk: stateless chunked upload; offset==0 truncates,
  offset>0 validates against file_size and appends. project deploy
  writes are now chunk-aware (fixes the oversized-single-Write hazard).
- FsRequest::HashPackage: canonical lph1 package hash for end-to-end
  verification (lpa-server gains lpc-history; hash travels as hex).
- LpFs gains append_file + file_size default methods (native LpFsMemory
  overrides; LpFsView/LpFsOpfs delegate) — chunked writes are O(n), not
  O(n²) read-modify-write; LittleFS-native overrides noted for M5.
- Client: ChangesCollector fold + request builders (lpa-client
  file_sync_ops), LpClient pull_changed_files / replace_project_files /
  hash_package, thin StudioServerClient wrappers for M3.
- fw-browser executor: the null-waker spin block_on is now a poll-once
  that panics loudly on Pending (sans-IO ADR follow-up discharged).
- fw-browser wasm e2e: push a project over protocol frames (chunked +
  mis-offset rejection) → hash-verify → load → paginated full pull →
  exact delta after an edit → empty after adopting the version.

Budget: FILE_SYNC_* constants derived beside the existing frame budget
with a compile-time fits-in-frame assertion and worst-case measurement
tests against the firmware serializer.

Plan: Planning/lp2025/2026-07-06-protocol-file-sync (roadmap M2b)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Yona-Appletree
Yona-Appletree merged commit 868eeed into main Jul 14, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant