fix(multiscan): record run warnings on bulk-scan receipts - #255
Open
rohanpoudel2 wants to merge 1 commit into
Open
fix(multiscan): record run warnings on bulk-scan receipts#255rohanpoudel2 wants to merge 1 commit into
rohanpoudel2 wants to merge 1 commit into
Conversation
runMultiscan called security.run() without an onWarning observer, and the per-attempt JSONL receipt recorded status, attempt, outputDir, cost and error but nothing about warnings. A repository whose target drifted mid-run kept coverage.completeness "complete", so the attempt genuinely succeeded and was written as status "completed" with the warning discarded: no failure, no non-zero exit, and no field a campaign consumer could read. Each attempt now installs an onWarning observer that collects the run's warnings, redacts them the way error is redacted because they are about to be persisted, and writes them to the receipt as an optional warnings array. The observer is used rather than the returned ScanResult because ScanResult does not carry warnings, and because it is the only channel that also sees the warnings run() emits from its finally block, which a failed attempt produces too. The key is omitted when an attempt warned about nothing, matching error and cost, so receipts for quiet attempts stay byte-identical to what earlier releases wrote and the resume path never requires the field. MultiscanResult gains a warned count beside failed: repositories that reported at least one warning during this run, plus those resumed from a receipt that carries warnings, the same way completed counts skipped repositories. Warnings are not failures, so the bulk-scan exit code is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #248
Problem
runMultiscannever asks for a repository's warnings, and the per-attempt ledger receipt has nowhere to put them:No
onWarning, so every warning the run reports is dropped on the floor. A repository whose target drifted mid-run keepscoverage.completeness === "complete", so the attempt genuinely succeeds: it is written asstatus: "completed",bulk-scanexits 0, and the campaign summary counts it amongcompleted. There is no failure, no non-zero status, and no field a consumer could read to notice the results describe a stale tree. The same holds for the other two warning producers — a cost limit that could not be verified, and the cleanup failuresrun()reports from itsfinallyblock.Note for reviewers:
ScanResultdoes not carry awarningsfield onmain. The only channel for these warnings is theonWarningobserver.Change
onWarningobserver that collects that attempt's warnings into a fresh array. The array is per attempt, so a retry does not inherit the previous attempt's warnings.redactedErrorMessage, the same wayerrordoes. The CLI's own observer only sanitizes for stderr; these strings are about to be written to disk and read back on resume, and the comment onwarnCleanupFailedis explicit that the warning text has so far never been persisted, so it has never been redacted upstream.warnings?: string[], written only when the attempt produced warnings.MultiscanResultgainswarned: numberbesidefailed. The exit code is untouched: a warning is not a failure.Warnings from a failed attempt are captured too, and deliberately so.
run()'s cleanup warnings are emitted from itsfinallyblock, so they happen whether the attempt returned a result or threw, and the completion and cost-limit warnings are emitted before a later step can throw. Readingresult.warnings— even if that field existed — would therefore lose exactly the warnings that accompany a failure. The observer is also immune to the two multiscan-side throws that happen afterrun()returns (the scope-escape check and thecoverage.completenesscheck), whose receipts now carry botherrorandwarnings.On ordering:
run()dispatches observers on a microtask. The receipt is serialized after this loop'sfinally { await rm(checkout, ...) }, which runs oncerun()has settled, so every queued warning has landed before the receipt is written.Why an omitted-when-empty
warnings?: string[]and not an unconditional arrayThe alternative is to always emit the key,
warnings: []included, matchingscan --json, which gets its unconditionalwarningsstraight fromworkbench_db.py. Rejected, for two reasons:costanderror— andscopeon the task half — are all omitted when absent.warnings: []on every one of thousands of receipts would be the only field in the record that insists on being present in order to say nothing.scans show/scans listalready omit the key whencompletion_warnings_jsonis"[]", so this side of the split is not novel either."warnings" in receiptbecomes a meaningful "this attempt warned" test rather than a tautology.Ledger compatibility
readReceiptsdoes an uncheckedJSON.parse(line) as MultiscanReceipt— there is no schema and no validator — and the resume path reads onlyid,status,outputDirandattempt.warningsis optional in exactly the wayerroris, so a receipt written before this change type-checks and resumes unchanged. The newwarnedcounter reads it throughArray.isArray(receipt.warnings) && receipt.warnings.length > 0, which isfalsefor a missing key and also for a hand-edited ledger holding a non-array there. Covered by a test that strips the key from a ledger and resumes: the repository stays skipped, is not rescanned, and the campaign reportswarned: 0.warningsis an extra key on a line that olderreadReceiptsparses withJSON.parseand casts; unknown keys are carried along and ignored. No existing field changed name, type, or emission condition.Impact, stated plainly
A drifted repository is still recorded as
completed, because it did complete — but its receipt now names the drift, and the campaign summary reportswarned: 1. A scripted campaign can act onwarnedwithout parsing the ledger; a human can read the offending message off the receipt.bulk-scan's exit code, thecompleted/failed/skippedcounts, the manifest, and the receipt fields that already existed are all unchanged.onProgressis deliberately left alone — this change adds a durable record, not a new stream of stderr chatter.Verification
From
sdk/typescript:bun test --timeout 30000 ./tests-ts— 776 pass, 5 skip, 0 fail (781 across 34 files).multiscan.test.tsalone — 17 pass, 0 fail.src/multiscan.tsreverted to its state onmainand the new tests left in place — 14 pass, 3 fail: each of the three new tests fails without the fix, and every pre-existing multiscan test still passes.tsc --noEmitclean;generate:models:checkclean;prettier --checkreports "All matched files use Prettier code style!".Three tests added to
sdk/typescript/tests-ts/multiscan.test.ts, in its existing mock-outcome style:records a completed attempt's warnings on its receipt and in the summary— two repositories, one warns. Asserts the warned receipt carrieswarnings, that the quiet receipt has nowarningsproperty at all, that the summary reportswarned: 1withfailed: 0, and that a syntheticsk-proj-credential inside the warning does not reach the ledger.records a failed attempt's warnings and counts its repository once— attempt 1 warns then throws, attempt 2 warns then completes. Asserts both receipts carry their own warning, that the failed one carrieserroralongside, and thatwarnedcounts the repository once rather than per attempt.resumes receipts written before the ledger carried warnings— resumes a ledger with the field, then rewrites it without the key and resumes again.