feat(cryptify): persist upload-session state to SQLite (write-through)

[rubenhensen]

Closes #302. First half of #300 (epic #247, workstream E); restore-on-boot is #303 and is deliberately not here.

What this changes

Upload sessions lived only in Store's in-memory map, so any restart lost every in-flight upload. This makes the state durable without changing a single response: every transition is written through to SQLite, and nothing reads the rows back yet.

usage_db now names cryptify's whole state database rather than just the quota table: one file, one rusqlite::Connection behind one Mutex (StateDb), two tables. upload_sessions is created with CREATE TABLE IF NOT EXISTS at startup, so a database already holding usage rows gains it in place, and usage_db unset still means in-memory only. The key name was left alone on purpose — renaming it would be an ops-visible change for zero behavioural gain.

Store::persist_session writes one upsert and runs before the handler responds, at each of the three transitions:

transition	call site
init	Store::create (before the entry is visible in memory)
chunk accepted	upload_chunk, once the rolling token has advanced
finalize	upload_finalize, right after the unsealed sender is known, before the email

The reverse edges matter as much, so Store::remove and the purge task both delete the row — an unresumable session must not leave one behind, or the table would grow without bound and a future restore would resurrect sessions the purge already killed. Database errors are logged rather than propagated: persistence must not add a way for a healthy upload to fail.

Three schema decisions worth calling out

• The recovery token is stored as a hex SHA-256, never as the plaintext bearer credential for GET /fileupload/{uuid}/status. A restore path compares sha256(presented) against the column.
• created_at is absent from the upsert's DO UPDATE SET list. That omission is what keeps it recording when the session began; last_active_at moves on every transition.
• There is no expected-size column. Clients send Content-Range: bytes <s>-<e>/* on chunk PUTs, so the total is genuinely unknown to the server until finalize declares it (and finalize rejects a declaration that disagrees with uploaded).

Coverage

12 new tests, all reading rows back over a second connection to the same file so they assert what a restarting process would actually find on disk rather than what the writing connection holds:

• store.rs: the full row after init; progress plus a preserved created_at after a chunk (and that it upserts rather than appends); the sender after finalize; deletion on remove and on eviction; the recovery token present only as a hash; the no-database no-op path; schema creation on first boot and on reopen; a usage-only database gaining the table without disturbing its usage rows.
• main.rs mod integration, through the build_rocket seam with usage_db pointed inside the per-test data_dir: real HTTP init → chunk → finalize, with the columns read back using plain SQL. session_rows_are_visible_in_sqlite_after_init_and_chunk is the ticket's done bar, and it asserts the persisted token is the one the client was handed. upload_happy_path_is_unchanged_with_persistence_enabled pins the no-visible-change claim.
• email.rs: language_code_matches_serde_representation pins the new Language::code() (what the mail_lang column stores) to the serde/wire form.

cargo test -p cryptify --all-targets → 170 passed, 0 failed. cargo clippy -p cryptify --all-targets -- -D warnings and cargo fmt --all -- --check clean. No dependency changes — this uses the rusqlite already in the lockfile (the sqlx/rusqlite links = "sqlite3" coupling in root CLAUDE.md is untouched).

cryptify/CLAUDE.md is updated in the same commit, including the fact that prod's conf/config.toml does not set usage_db at all — so session persistence is off in any deployment that never set it, which #303 will need to know.

[dobby-coder]

Thanks for the PR! I'm on it — taking a look at the upload session persistence changes (SQLite write-through for the three transitions, the usage_db/StateDb rework, and the removal/purge cleanup paths) and I'll follow up with a review shortly.

[dobby-coder]

Review arrived after the merge (opened 07:41Z, merged 07:45Z, no reviews on it), so this is post-hoc — everything below is on main now. Posting as a comment because there is nothing left to block; the two bugs are worth follow-up tickets rather than a revert.

The design decisions in the description hold up. Write-through before the response, created_at omitted from the DO UPDATE SET list, the recovery token stored only as a hex SHA-256, and no expected-size column are all the right calls, and the tests reading rows back over a second connection is the detail that makes them mean anything. The CREATE TABLE IF NOT EXISTS in-place upgrade and leaving the usage_db key name alone are both correctly reasoned as ops-visible-change avoidance.

Two things I would not have merged without a second look, both verified against this branch:

1. The purge/persist race resurrects rows that nothing will ever delete again (store.rs). purge_expired now removes from state.files/expirations under the state lock, drops the lock, and only then calls delete_session. A chunk PUT that took its Arc from store.get before eviction is still awaiting the body read, and calls persist_session after — which goes straight to the INSERT ... ON CONFLICT without ever consulting state.files. The row comes back, but the session is gone from both state.files and expirations, so nothing deletes it a second time. That is exactly the unbounded-growth and resurrect-a-killed-session pair the description says the delete exists to prevent. Store::remove has the same shape and a wider window — it deletes the row before taking the state lock.

2. usage_db inside data_dir makes every session row publicly downloadable (main.rs). GET /filedownload/<filename> serves any name without a slash straight out of data_dir with no credential; is_safe_download_segment only blocks traversal and ../.. Not a live exposure — prod's conf/config.toml sets data_dir but never sets usage_db, so persistence is off there — but this PR is what turns that file from (email, timestamp, bytes) quota rows into recipients, mail_content, sender, api_key_tenant and recovery_token_hash, and the new test helper models the DB-inside-data_dir layout while cryptify/CLAUDE.md never says it must live outside.

Three smaller notes inline: an fsync per chunk PUT under the tokio guard, touch not persisting last_active_at, and secure_delete for a database that now holds message bodies.

None of this is a revert — the feature works and the response surface is genuinely unchanged. But #1 should land before #303 reads these rows back, because restore is the point at which a resurrected row stops being a leak and starts being a wrong answer.

[dobby-coder]

An evicted session's row is resurrected and then never deleted again. purge_expired removes the entry from state.files/expirations under the state lock, drops the lock, then deletes the row here. But a chunk PUT that took its Arc from store.get before eviction is still awaiting the body read and the file write, and calls persist_session afterwards — and upsert_session is an INSERT ... ON CONFLICT, so it re-inserts the row. The session is now gone from state.files and from expirations, so nothing will ever delete that row again: the table grows without bound and #303 would restore a session the purge already killed — precisely the two failure modes the PR description says this delete exists to prevent. Store::remove (store.rs:646) has the same race, and deletes before removing from the map.

Confirmed on this branch with a scratch test: Store::with_idle_ttl(20ms, .., Some(db)), create, hold the handle from store.get, poll until the row disappears, then persist_session — the row is back while store.get returns None. In prod the window is the whole chunk body-read, so a slow large chunk arriving near the 1h idle deadline hits it.

A fix needs the write and the liveness check to agree — e.g. have persist_session take the state lock and skip the upsert when state.files no longer contains the id, or keep a short-lived tombstone set that upsert_session consults.

[dobby-coder]

Putting the state DB inside data_dir makes every session row publicly downloadable, and this helper models exactly that layout. GET /filedownload/<filename> (main.rs:1395) serves any name without a slash straight out of data_dir, with no credential — is_safe_download_segment only blocks traversal. Verified against this branch: GET /filedownload/state.db → 200 (4096 bytes), and GET /filedownload/state.db-wal → 200 with 37 KB containing the session UUID (WAL mode means the live rows are in the -wal file). That is recipients, mail_content, sender, api_key_tenant and recovery_token_hash for every in-flight upload, readable by anyone who guesses the filename.

Not a live prod exposure — prod's conf/config.toml doesn't set usage_db, and config.dev.toml points it at /app/data/usage.db while data_dir is /tmp/data. But this PR is what turns that file from (email, timestamp, bytes) quota rows into full session state including a bearer-credential hash and the message body, and the new cryptify/CLAUDE.md text ('one file to mount and back up') never says it must live outside data_dir — while this test helper writes it inside. Worth doing here: point the test's DB somewhere outside data_dir so it isn't a template, and refuse (or loudly warn) at startup when usage_db resolves inside data_dir. Restricting is_safe_download_segment to a UUID shape would close it for good — data_dir files are always Uuid::new_v4().hyphenated() — but that's a bigger call than this PR.

[dobby-coder]

This is a synchronous rusqlite commit — a global std::sync::Mutex plus, under WAL with SQLite's default synchronous=FULL, an fsync — running on a Rocket worker thread with the FileState tokio guard still held (drop(state) is the next line). record_usage has the same shape but runs once per upload at finalize; this one runs on every chunk PUT, so with chunk_size = 5000000 that's an fsync per 5 MB per upload, and all concurrent uploads serialise on the one connection mutex while blocking the async executor.

Nothing here is wrong today at current traffic, but two cheap mitigations are worth considering: PRAGMA synchronous = NORMAL (safe under WAL — it risks only the last commit on power loss, and the row is a recovery aid, not the source of truth), and/or doing the write outside the state guard so a slow disk can't hold the per-session lock.

[dobby-coder]

last_active_at is refreshed only where persist_session is called, but the other thing that counts as activity isn't a persisted transition: GET /fileupload/{uuid}/status calls store.touch (main.rs:1109), which extends the in-memory expirations deadline and writes nothing. So a session kept alive across a page refresh by status polls has a durable last_active_at arbitrarily far behind its real idle deadline. Harmless now since nothing reads the column, but if #303 reconstructs the idle deadline from it, restore will evict exactly the cross-refresh-resume sessions the status endpoint exists to keep alive. Either persist on touch too, or say in this field doc (and cryptify/CLAUDE.md) that last_active_at tracks persisted transitions only and is a lower bound on activity.

[dobby-coder]

StateDb::open sets journal_mode=WAL but not secure_delete, so when the purge task or remove deletes a session row its plaintext — recipient addresses, mail_content, sender — stays in the database's free pages, and in the WAL until checkpoint, indefinitely. Before this PR the file only ever held (email, timestamp, bytes). For the database that now holds message bodies and recipient lists, PRAGMA secure_delete = ON is cheap at this write volume. Related and worth a line in cryptify/CLAUDE.md: a successfully finalized session's row is only removed by the idle purge, so recipient addresses and the mail body outlive the completed transfer by up to idle_ttl (1h).

[dobby-coder]

Heads up — this landed before the review could run (opened 07:41Z, merged 07:45Z, no reviews), so I couldn't block anything. I reviewed it anyway and left a consolidated comment with 5 inline notes on #319.

Overall the write-through design holds up well: write-before-respond, created_at excluded from DO UPDATE SET, recovery token stored only as a SHA-256 hex, and the tests reading back over a second connection. Two things worth turning into follow-ups before #303 (restore-on-boot):

• Purge/persist race can resurrect "deleted" sessions — purge_expired drops the lock before deleting the row, and persist_session/Store::remove don't consult state.files, so a chunk PUT still in flight can re-insert a session that's already been purged. Fine today since nothing reads these rows back, but it'll bite as soon as store: restore sessions on boot; chunk replay and resume survive restart #303 does.
• usage_db living inside data_dir would make session rows (recipients, mail_content, sender, recovery_token_hash) downloadable via GET /filedownload — not live in prod today since data_dir is set but usage_db isn't, but worth documenting the constraint in cryptify/CLAUDE.md now that this DB holds real content.

Plus three smaller notes inline (fsync-per-chunk while holding the tokio guard, touch not persisting last_active_at, and PRAGMA secure_delete for a DB with message bodies). No revert needed — nothing about the response behavior changed.