Skip to content

chore(audit): weekly cloud audit — 4 SQL read-only bypasses + 4 scanner false-clean masquerades#123

Draft
tzone85 wants to merge 5 commits into
mainfrom
fix/audit-cloud-2026-07-20
Draft

chore(audit): weekly cloud audit — 4 SQL read-only bypasses + 4 scanner false-clean masquerades#123
tzone85 wants to merge 5 commits into
mainfrom
fix/audit-cloud-2026-07-20

Conversation

@tzone85

@tzone85 tzone85 commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Weekly cloud audit — 2026-07-20

Fan-out audit across concurrency, silent failures, event-sourcing wiring, and security, then five loop-until-dry rounds with every candidate adversarially verified before fixing (rounds 4 and 5 both returned no findings → two consecutive dry passes → done; round 4 additionally verified the SQL findings against a live PostgreSQL 16 instance). Concurrency and event-sourcing wiring came back clean. Two real defect classes surfaced, each with several siblings — 8 verified fixes total, all in two files.

Class A — internal/sqlsafety/sql_safety.go: read-only SQL gate bypasses

vxd db sql (and the sql_query_returns QA criterion) run operator SQL under BEGIN READ ONLY, which does not block side-effecting function calls (pg_terminate_backend, pg_read_file, lo_import, …) — so those are rejected statically by ContainsDeniedFunction, which strips comments/strings then regex-matches <fn>(. The classifier's string/identifier lexer had gaps that let a denied call reach the DB without --write. Each was confirmed empirically (ValidateSQL(query, false, nil) returned nil).

  • stripSQLCommentsAndStrings — double-quoted identifiers. The stripper left "..." verbatim. Postgres folds an unquoted identifier to lowercase, so "pg_terminate_backend" names the same built-in — but the quote between the name and ( defeated the regex. SELECT "pg_terminate_backend"(pid) / SELECT "pg_read_file"('/etc/passwd') classified read-only. Fix: unwrap double-quoted identifiers to bare inner text.
  • stripSQLCommentsAndStringsE'…' escape strings. Single-quoted handling assumed only '' escaping; Postgres escape strings let \' escape a quote, so the stripper mis-tracked the boundary and swallowed a call after the literal. SELECT E'\'' || pg_read_file('/etc/passwd') (a single valid statement) bypassed. Fix: detect E'/e' at a token boundary and consume backslash escapes.
  • stripSQLCommentsAndStrings — dollar-quoted strings. No $tag$…$tag$ handling; a dollar-quoted literal with an odd number of apostrophes desynced the single-quote tracker and hid the trailing call. SELECT $$'$$, pg_read_file('/etc/passwd') bypassed. Fix: consume dollar-quoted strings before the single-quote branch (tag follows identifier rules, so $1 params are not delimiters).
  • ValidateSQL / hasUnicodeEscapeIdentifierU&"…" Unicode-escape identifiers. A U&"\XXXX…" identifier can name a function, but its escapes decode only server-side, so the regex (seeing raw \XXXX) never matches the decoded name. SELECT U&"\0070\0067_\0072\0065\0061\0064_\0066\0069\006C\0065"('/etc/passwd') (decodes to pg_read_file) bypassed. Decoding arbitrary escapes is error-prone, so the gate now rejects U&"…" identifiers in read-only mode (--write opts out). Only the identifier form is flagged — U&'…' is a string literal (its content never executes) and is already stripped correctly.

The classifier now understands the full Postgres string/identifier lexical set: '…' (incl. E'…'), "…" (incl. U&"…"), $tag$…$tag$, and U&'…'. stripSQLCommentsAndStrings was refactored into lexSQL. Documented assumption (unchanged, in-code): standard_conforming_strings=on, the Postgres default and the postgres:16 devdb default; with it off the mismatch direction is over-rejection, never concealment.

Class B — internal/security/scanners.go: failed scans masquerading as clean

RunScanners sorts each tool into ran (clean), skipped (tool absent), or failed (coverage lost). A parser returning (nil, nil) for a run that couldn't actually scan lands in ran, so the security gate — and security.require_scanners strict mode — believe the tool provided coverage it never did. The JSON scanners whose failure output is non-conforming already fail closed via json.Unmarshal; four parsers had a fail-open path:

  • parseNpmAudit — npm's valid-JSON error envelope ({"error":{"code":"ENOLOCK",…}}, e.g. no lockfile) decoded to a nil vuln map → clean. Fix: surface the error field as a failure.
  • parseSemgrep — decoded only results, ignoring the top-level errors array; a scan-level failure (e.g. --config auto can't load rules offline) emits results:[] + an error-level entry → clean. Fix: fail when results is empty and a level:"error" entry is present (a scan that produced findings is never flipped).
  • parseGovulncheck — text-mode parser with no failure detection; an unreachable vuln DB (offline host) produced zero Vulnerability # lines → clean. Fix: detect the govulncheck: fatal line and fail when no findings were parsed.
  • parseGosec — decoded only Issues, ignoring the "Golang errors" map; a compile/package-load failure emits valid JSON with empty Issues → clean. Fix: fail when Issues is empty and Golang errors are present.

All five parsers (parseGosec, parseGitleaks, parseNpmAudit, parseSemgrep, parseGovulncheck) now surface a failed run as failed, upholding the module's stated invariant that "a failed scan must never masquerade as a clean one."

Every fix restores already-documented behavior (denylist doc comment; RunScanners "must never masquerade" comment), so no CLI/config/doc-coverage changes are required. Regression tests were added for all eight.

Verification

Run with the project's required toolchain (GOTOOLCHAIN=go1.26.5); golangci-lint and govulncheck were re-installed against go1.26.5 (the pre-installed builds were compiled with go1.25 and refuse a go1.26 target).

  • go build ./...OK
  • go vet ./...OK
  • golangci-lint run ./... (config default: standard) — 0 issues
  • go test ./internal/sqlsafety/... ./internal/security/...PASS (the two changed packages, incl. all new regression tests)
  • Doc-coverage (TestDocCoverage_*) — PASS

Pre-existing, environment-only failures (present on unmodified main, unrelated to this diff): the cloud runner executes the suite as root (uid 0) and without the gh CLI, breaking tests that assert on filesystem-permission or gh-required failure paths — internal/cli TestApproveStory_*/TestApproveAll_*/TestRunApprove_* (no gh → "gh CLI is required" short-circuit), internal/engine TestWriteLockFile_InvalidPath + TestWriteMetadata_InvalidDir (root MkdirAll succeeds under /), internal/figma TestResolveToken_UnreadableFileIsDistinguished (root reads through chmod 0o000), internal/repolearn TestCountFilesInDir_NonexistentDir (polluted by /nonexistent/path the root engine test creates). Verified by stashing this diff and re-running on clean main — an identical 10-test set fails there. Not touched.

govulncheck / make vuln: could not run — the environment's egress policy denies vuln.go.dev (403 on CONNECT), so the vulnerability DB is unreachable. Not retried, per the proxy's report-don't-route-around rule. go build/vet/test served as the fallback gate. (This same unreachable-DB condition is exactly the failure the parseGovulncheck fix above surfaces instead of hiding.)

🤖 Generated with Claude Code

https://claude.ai/code/session_01U2huFn4mXqnCNn1jZCZKWi

claude added 5 commits July 20, 2026 04:22
…and npm-audit false-clean

Two verified defects from the weekly cloud audit:

sqlsafety: the read-only SQL gate's side-effecting-function denylist
(ContainsDeniedFunction) matched `<fn>[\s]*\(`, but stripSQLCommentsAndStrings
never unwrapped double-quoted identifiers. Postgres folds an unquoted
identifier to lowercase, so `"pg_terminate_backend"` names the same built-in
as `pg_terminate_backend`, yet the interposed quote defeated the regex.
`vxd db sql <db> 'SELECT "pg_terminate_backend"(pid) FROM pg_stat_activity'`
(and `"pg_read_file"('/etc/passwd')`) classified as read-only and ran without
--write — exactly the cross-user disruption the denylist exists to block, and
READ ONLY transactions do not stop these calls. The classifier now unwraps
double-quoted identifiers to bare inner text so the true call shape is seen;
this also hardens ClassifyQuery/IsMultiStatement which share the strip pass.

security: parseNpmAudit unmarshalled only {vulnerabilities}, so npm's
valid-JSON error envelope ({"error":{"code":"ENOLOCK",...}}, emitted when the
audit cannot run — e.g. a package.json with no lockfile) parsed as a clean,
empty scan. RunScanners then counted it in `ran` (coverage OK) instead of
`failed` (coverage lost), so the dependency-CVE scan silently no-ops while the
security gate — and security.require_scanners strict mode — believe the tool
ran. parseNpmAudit now surfaces the error envelope as a failure.

Both restore already-documented behavior (no doc/config change). Regression
tests added: quoted-identifier denied/allowed cases + strip-pass unwrap;
npm-audit ENOLOCK/code-only error envelopes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2huFn4mXqnCNn1jZCZKWi
…false-clean (round 2 siblings)

Two verified siblings of the round-1 fixes, found in the same two files:

sqlsafety: stripSQLCommentsAndStrings modelled single-quoted strings with
only ''-doubling escape semantics. Postgres *escape strings* (E'...'/e'...')
let a backslash escape a quote, so `\'` does NOT close the string. The stripper
mis-tracked the boundary and over-consumed, swallowing a denied call that
followed a literal:
`vxd db sql <db> "SELECT E'\'' || pg_read_file('/etc/passwd')"` — a single
valid statement — classified read-only and ran without --write (confirmed:
ValidateSQL returned nil). The stripper now detects escape-string mode (E'/e'
at a token boundary) and consumes backslash escapes so the boundary is correct;
the trailing pg_read_file(/pg_terminate_backend( call is then caught.

security: parseSemgrep decoded only "results" and ignored the top-level
"errors" array. On a scan-level failure (e.g. `--config auto` cannot load rules
on an offline host) semgrep emits valid JSON with results:[] + an error-level
entry and exits non-zero, so parseSemgrep returned ([], nil) — counted in
RunScanners' `ran` (clean) list, not `failed`, hiding total coverage loss from
the gate and require_scanners strict mode (same masquerade class as the
round-1 npm-audit fix). It now returns an error when results are empty AND a
fatal error-level entry is present; a scan that produced findings is never
flipped to failed by a coexisting non-fatal warning.

Both restore already-documented behavior (no doc/config change). Regression
tests added: E-string denied/allowed + strip-pass cases; semgrep scan-error
envelope + results-plus-warning stays-clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2huFn4mXqnCNn1jZCZKWi
…k false-clean (round 3 siblings)

Third pair of verified siblings in the same two files:

sqlsafety: stripSQLCommentsAndStrings had no dollar-quote ($tag$...$tag$)
handling. A dollar-quoted literal containing an ODD number of apostrophes
desynced the single-quote tracker, swallowing a denied call that followed the
literal: `vxd db sql <db> "SELECT $$'$$, pg_read_file('/etc/passwd')"` — a
single valid statement whose pg_read_file call is real executable code outside
any string — classified read-only and ran without --write (confirmed:
ValidateSQL returned nil). The stripper now consumes dollar-quoted strings
before the single-quote branch (tag follows identifier rules, so positional
params like $1 are not treated as delimiters), fixing the desync.

security: parseGovulncheck runs in text mode, so — unlike the JSON scanners
whose failure output fails json.Unmarshal and routes to `failed` — a run that
could not execute (e.g. the vulnerability DB unreachable on an offline host)
produced zero findings and returned (nil, nil), counted in RunScanners' `ran`
(clean) list. Same masquerade class as the npm-audit/semgrep fixes. It now
detects govulncheck's "govulncheck: <msg>" fatal line and returns an error
when no findings were parsed; a clean no-vuln run is never flipped to failed.

The complete Postgres string/identifier lexical set is now handled by the
classifier: '...' (incl. E'...' escape), "..." identifiers, and $tag$...$tag$.
Both restore already-documented behavior (no doc/config change). Regression
tests added for both.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2huFn4mXqnCNn1jZCZKWi
…y mode

Fourth SQL-lexer sibling. A Postgres Unicode-escape identifier U&"\XXXX…" can
name a function to call, but its \XXXX escapes decode only server-side — the
denylist regex sees the raw escapes, so it never recognises the decoded name.
`vxd db sql <db> "SELECT U&\"\0070\0067_\0072\0065\0061\0064_\0066\0069\006C\0065\"('/etc/passwd')"`
(the escapes decode to pg_read_file) classified read-only and ran without
--write (confirmed: ValidateSQL returned nil), and READ ONLY does not block
pg_read_file.

Rather than attempt to decode arbitrary \XXXX / custom-UESCAPE sequences, the
read-only gate now rejects U&"…" identifiers outright (re-run with --write if
intentional). Only the identifier form is flagged: U&'…' is a string literal
whose content never executes and is already stripped correctly. The stripper
is refactored into lexSQL, which returns the stripped text plus a flag for a
U&"…" identifier seen outside comments/strings; stripSQLCommentsAndStrings and
the new hasUnicodeEscapeIdentifier are thin wrappers.

This completes the Postgres string/identifier lexical set the classifier
understands: '…' (incl. E'…'), "…" (incl. U&"…"), $tag$…$tag$, U&'…'.
Regression tests added (rejected identifier forms, allowed U&'…' string /
plain quoted identifier / u-prefixed name, --write opt-out).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2huFn4mXqnCNn1jZCZKWi
…h no issues

Completes the fail-closed invariant across all five scanner parsers. gosec
reports compile/package-load failures in its "Golang errors" map while still
emitting valid JSON with an empty Issues list, so parseGosec (which decoded
only Issues) returned a clean empty scan for a run that could not analyse the
code — the same "failed scan masquerading as clean" the npm-audit/semgrep/
govulncheck fixes close. parseGosec now returns an error when Issues is empty
AND Golang errors are present; a scan that produced issues is never flipped to
failed by a coexisting per-file parse error.

With this, parseGosec, parseGitleaks, parseNpmAudit, parseSemgrep and
parseGovulncheck all surface a failed run as `failed` (coverage lost) rather
than counting it in RunScanners' clean `ran` list. Regression test added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2huFn4mXqnCNn1jZCZKWi
@tzone85 tzone85 changed the title chore(audit): weekly cloud audit — SQL denylist quoted-identifier bypass + npm-audit false-clean chore(audit): weekly cloud audit — 4 SQL read-only bypasses + 4 scanner false-clean masquerades Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants