test(platform): make a truncated itest run fail loudly - #3198
Merged
Conversation
This was referenced Sep 4, 2026
Squash of the wt-3 batch for rebase onto main; the PR body carries the record.
larryro
force-pushed
the
fix/itest-harness-integrity
branch
from
September 4, 2026 05:38
2f4ce92 to
2a67766
Compare
larryro
marked this pull request as ready for review
September 4, 2026 05:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The truncation defect
backend:integrationcould run fewer checks than it contains without saying so. The two-factor lifecycle probe (#3187) drove/two-factor/enable→/disableon the suite's shared user. Better Auth 1.6.23 rotates the calling session ondisable(and onenableunderskipVerificationOnEnable): it issues a new token and deletes the old one, so the harness's fixed cookie died at that step and every later lane 401'd.Before (origin/main
1ea4cc9e9, fresh tale-db + MinIO): 178 checks recorded (171 PASS / 7 FAIL — six of them dead-session garbage right after the 2FA lane), thenNo tally line. The 76 lanes from
checkAutomationsonward —checkConnectorOauth,checkConversations,checkOutboundSendLane, governance, REST, SSO/SCIM, the agent lanes, the login-throttle/audit chain, the dev seeder — never ran (~207 checks), and nothing in the output said so.After (this branch, fresh containers): the run completes end to end —
385/385 checks passed across 119/119 lanes, exit 0.checkConnectorOauthran (5 PASS lines that were hidden before).Two changes, so it cannot happen silently again:
signUpOrgMember(hoisted fromcheckBlobRefAuthority's local copy) creates the enrollee; the lifecycle calls follow the session rotation like a browser (mergeCookieHeader). The shared session is never touched.runLanes— the truncation guard. Every session-bearing lane runs through it (the lane table is the old sequence 1:1, verified by diffing the extracted names). A lane that throws is recorded as a FAIL naming the lane and how many never ran; after every lane the shared session is asserted alive via/api/auth/get-session. The tally printsRUN TRUNCATED at lane N of M — the tally below covers only those, and the exit code is red. A throw before the lanes is recorded the same way, so the tally always prints.Probe vs gate: the gate is right, the probes were driving the wrong actor
#3187 put
viewerCanWrite(editor-or-above; the 0.4conversations/conversationMessagesRLS write rule) on the first line of every conversation write door, ahead of ownership and visibility. That order is intended: a read-onlymembermay not compose, reply, undo, retry or discard mail under the org's name at all, and #3187's unit tests pin it — member → 403FORBIDDENbefore anything loads; the opaque 404 for a hidden conversation is asserted for a role that passes the gate. Option (b) does not apply: the gate never refuses an authorized composer attaching their own upload — an editor reachesattachment_not_owneduntouched.What was wrong is that two integration probes drove the deeper check with a
member, so the gate shadowed what they meant to prove. Silencing them would have deleted the ownership/visibility coverage, so both now assert both layers:checkBlobRefAuthoritystep 10 (fix(platform): blob refs grant nothing; upload and import lanes gated #3160): member →403 FORBIDDEN(the gate); editor with the owner's blob →403 attachment_not_owned(ownership, reached through the gate).checkOutboundSendLane(hidden behind the truncation until now): member undo/retry/discard →403(the gate); an editor who cannot open the unassigned conversation → opaque404(visibility); rows untouched either way.Doc drift
environment-reference.md(en/de/fr) still calledINSTANCE_SECRETthe seed of the "deploy admin key" — the Convex-era role that left with the 0.5 backend — and said deploy fails without it, althoughtale init/tale deploygenerate a missing one. It now describes the 0.5 role: the instance's root secret from which the WebDAV app-password HMAC key is derived at boot unless set explicitly (docker-entrypoint.sh,lib/webdav/hmac-key.ts, #3132), with the sandbox blob-staging tokens signed by a subkey of that derivation; rotating it invalidates every app-password. The wider table re-alignment in those files is the edit hook's oxfmt output.Gates observed
bunx tsc --noEmit(platform): clean.bunx oxlint --type-aware backend/integration-check.ts: clean.backend:integration, fresh tale-db + MinIO,ITEST_S3_ENDPOINT+SANDBOX_LLM_GATEWAY_ADMIN_PASSWORDset: base 178 recorded then fatal; branch 385/385 across 119/119 lanes, exit 0.bun run --filter @tale/docs test: 194/194.Cross-class discoveries (not touched here)
services/platform/env.shstill definesensure_hex_instance_secretwith a Convex-era comment ("generate-admin-key"); nothing calls it, so the 64-hex shape ofINSTANCE_SECRETis only whattale initgenerates, not something boot enforces.format:checkdoes not coverdocs/**/*.md, while the edit hook formats them — hence tables on main that the hook re-aligns on first touch.signUpOrgMember.