feat(canvas): canvases are plain .excalidraw files in the vault - #946
Merged
Conversation
A canvas was the only user content with no file on disk: the scene lived
in canvases.snapshot_ciphertext, encrypted with the vault key. That tied
the ink to one machine's master key, so it died in two ordinary cases:
- a local-only user turns on sync — first-device setup mints a NEW master
key from a new recovery phrase, rebinds the vault, and every canvas
drawn before the upgrade becomes undecryptable (and unpushable, so no
server copy exists either);
- the vault folder is copied to another machine (USB, git, Dropbox) —
the key lives in the OS keychain, not in the folder, so the canvas
cannot be opened there at all.
Encrypting it bought nothing: notes, journals and attachments in the same
folder are already plaintext, so the threat model was unchanged while the
fragility was entirely real.
Now the file is the source of truth and the table is an index:
- `<vault>/canvases/<Title>.excalidraw`, valid Excalidraw JSON with a
`memry` sidecar (id + timestamps) so a single copied file is
self-describing; canonical key order so two devices emit identical
bytes for identical ink (the conflict-copy check compares text).
- `<vault>/canvases/library.excalidrawlib` replaces the encrypted
canvas_library_items rows (not a sync type — the file is the store).
- Vault open migrates legacy snapshots once and adopts files that
arrived with the folder. A snapshot we cannot decrypt KEEPS its
ciphertext and surfaces as `unreadable` instead of mounting an editor
that would autosave over recoverable ink.
- Sync is unchanged on the wire: push reads the file, apply writes it,
transport encryption in sync/encrypt.ts still wraps a per-item key
under the vault key, so the server never sees plaintext.
Additive migration (0045): `file_path` added, `snapshot_ciphertext` kept
NOT NULL and blanked ('') as rows migrate. No DB reset, no data dropped.
github-actions
Bot
added
documentation
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
The three sync-path emitters shipped `{ id, source: 'sync' }` with no
`changes`, which `NoteUpdatedEvent` requires. Renderer subscribers read
`changes.content` unguarded, so every note applied by a pull threw inside
the preload listener loop and that subscriber silently dropped the event —
link caches never invalidated after a pull.
Both emit paths are `(channel: string, data: unknown)`, so typecheck could
not see the violation. Route the three call sites through a typed
`emitNoteUpdated` helper instead, and normalize a missing `changes` once in
`onNoteUpdated` so an older main process stays tolerated.
The write-back emit carries `content` now, but `scheduleWriteback` also
fires for local typing on a 500ms debounce that beats the 1000ms save — so
note.tsx skips `source: 'sync'` rather than remounting the editor
mid-keystroke over bytes the IPC CRDT provider already applied.
…ux and Windows
Windows was the real bug, not a hypothetical: vault-relative paths were built
with path.join, so a canvas created on Windows stored `canvases\Plan.excalidraw`
in file_path. Copy that vault to a Mac — the whole point of this PR — and the
backslash is one bogus filename, not a directory. Paths are now always stored
forward-slashed (the same `normalizeRelativePath` convention notes use) and
re-joined natively at read time.
The rest of the platform gap, all of it on the write path:
- Transient EBUSY/EPERM/EACCES retry (50/150/450ms) around write, rename and
delete. On Windows a cloud-sync client or antivirus scanner holds vault files
for a moment; the async `withTransientFsRetry` in vault/file-ops.ts already
does this, but the canvas path is synchronous (the sync apply writes inside a
better-sqlite3 transaction), so this is its sync twin.
- Windows reserved device names (CON, NUL, COM1, LPT9…) are rejected by Win32
whatever the extension, and they are ordinary canvas titles. Suffixed, not
replaced, so the user still recognizes the file.
- Trailing dots and spaces are silently trimmed by Win32, so `Plan.` and `Plan `
both resolved to `Plan` and quietly collided. Stripped up front.
- Filename collisions compare case-insensitively: macOS and Windows default to
case-insensitive filesystems, so two canvases that coexist on Linux must not
merge into one when the folder is copied to a Mac. A canvas's own file is
excluded, or a case-only title edit ("Plan" → "plan") lands on "plan 2".
- The `.excalidraw` extension is matched case-insensitively when listing and
when deriving a title from a filename.
- A directory sitting where a document should be (ENOTDIR/EISDIR after a bad
copy) reads as "no document" instead of taking the canvas surface down.
Also closes the CodeQL "insecure temporary file" alert on the atomic write: the
temp file now uses a random name opened `wx` with owner-only permissions and is
cleaned up on failure, matching vault/file-ops.atomicWrite. A predictable temp
name in a user-writable directory is a symlink-swap target.
Test expectations that hard-coded path.join are now platform-independent, so the
suite means the same thing on the Windows runner.
… them macOS stores filenames decomposed: a canvas titled "Yağmur" is written NFC and comes back from readdir as NFD — different bytes for the same name. Reconcile compared raw strings, so every vault open would see a "new" document and rewrite the row's path. Comparisons now go through `canvasPathKey` (NFC + lowercase); the STORED path still keeps the bytes as they exist on disk, because Linux filesystems are normalization-sensitive and an NFC-normalized path would fail to open a file that arrived NFD from a Mac.
| const tmp = path.join(path.dirname(absolutePath), `.${randomBytes(6).toString('hex')}.tmp`) | ||
| try { | ||
| writeFileSync(tmp, content, { encoding: 'utf-8', mode: 0o600, flag: 'wx' }) | ||
| const fd = openSync(tmp, 'r+') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A canvas was the only user content with no file on disk. The scene lived in
canvases.snapshot_ciphertext, encrypted with the vault key — which tied the ink to one machine's master key. Two ordinary user journeys destroyed it:performFirstDeviceSetupmints a brand-new master key from a new recovery phrase and rebinds the vault;bindLocalVaultToMasterKeypurges sync state and agent chat but never touchedcanvases. Every pre-upgrade canvas became undecryptable — and unpushable, so no server copy existed to recover from. Silent, permanent.The encryption bought nothing: everything else in the same folder is already plaintext, so the local threat model was unchanged while the fragility was entirely real.
What changed
The file is the source of truth; the table is an index.
memrykey ({ id, createdAt, updatedAt }) rides inside the document. Excalidraw ignores unknown top-level keys, so the file still opens in excalidraw.com and a single copied file keeps its identity.memryAssetsimage sidecar) preserved and sorted. Two devices emit identical bytes for identical ink — which is what the sync conflict-copy comparison relies on.canvas/reconcile.ts): migrates legacy encrypted snapshots once, and adopts documents that arrived with the folder. A file renamed outside the app re-points its row instead of duplicating. Rows whose file is missing are reported, never tombstoned — a half-copied vault must not delete canvases.unreadable; the editor refuses to mount rather than autosave an empty scene over recoverable ink.getCanvasContext()no longer resolves a vault key; the keychain is consulted only by the one-way migration.Backward compatibility
0045is additive:file_pathadded;snapshot_ciphertextstaysNOT NULLand is blanked ('') only as a row successfully migrates. No DB reset, nothing dropped.canvas_library_itemsrows are migrated into the library file and tombstoned; the ciphertext column is left intact.Verification
pnpm --filter @memry/desktop test:main— 4796 passed, 1 skipped, 0 failedpnpm typecheck— 16/16 tasks green ·pnpm lint— 0 errorspnpm check:architecture,pnpm check:contracts,pnpm ipc:check,i18n:check— passpnpm docs:impact --base origin/main --strict+pnpm docs:build— passscene-file.test.ts(26),reconcile.test.ts(12 — including the free→paid key change and the copied-vault adoption), rewrittenstore.test.ts(15, one test opens a canvas from a copied vault folder),canvas-handler.test.ts(24),canvas-handlers.test.ts(23, asserts the keychain is never touched).Not covered by automated tests: a manual walkthrough in the running app (open an existing vault with legacy canvases, verify migration + rendering).
Also on this branch: the
notes:updatedsync payload fixCarried here rather than into its own PR. The three sync-path emitters shipped
{ id, source: 'sync' }with nochanges, whichNoteUpdatedEventrequires. Renderer subscribers readchanges.contentunguarded, so every note applied by a pull threw inside the preload listener loop and that subscriber silently dropped the event — link caches never invalidated after a pull.Both emit paths are
(channel: string, data: unknown), so typecheck could not see the violation. The three call sites now go through a typedemitNoteUpdatedhelper, andonNoteUpdatednormalizes a missingchangesonce so an older main process stays tolerated.sourcegains'sync', which was always emitted at runtime but never admitted by the union.The write-back emit carries
contentnow, butscheduleWritebackalso fires for local typing on a 500ms debounce that beats the 1000ms save — sonote.tsxskipssource: 'sync'rather than remounting the editor mid-keystroke over bytes the IPC CRDT provider already applied.🤖 Generated with Claude Code