fix(security): exclude benchmark fixture corpus from CodeQL analysis - #281
fix(security): exclude benchmark fixture corpus from CodeQL analysis#281parthrohit22 wants to merge 2 commits into
Conversation
With the default branch now scanned on push (Second-Origin#280), the Security tab shows the real remaining warning: 'Could not process some files due to syntax errors', once for python and once for javascript-typescript. Both point at the Repository Intelligence golden benchmark corpus: adversarial/adv-py-malformed/src/broken.py def broken(: adversarial/adv-ts-malformed/src/broken.ts export function broken( { These are adversarial inputs to the RI engine, not code PARTHA executes. Each manifest asserts the RI-SRC-MALFORMED diagnostic per RFC-0001 4.3, 6.2 and 8, so the malformed syntax is the fixture contract. Correcting the syntax would silently delete py.syntax_error and ts.syntax_error coverage from the benchmark. Add a CodeQL config excluding the fixture corpus and wire it into the init step. Application code is unaffected: 282 source files remain in scope and 23 fixture sources leave it.
The bare directory path did not take effect: the analysis still scanned 248 of 248 Python files, the same count as before the exclusion, and re-reported the parse errors in broken.py and broken.ts. Add an explicit trailing glob alongside the bare path.
|
Converting to draft — the fix in this PR does not work. Reporting it rather than leaving it looking merge-ready. What I claimed vs what happened The PR body says excluding the fixture corpus removes the warning. CI on this branch disproves it. Two attempts, identical result:
If the exclusion worked the count would be 235 (248 minus 13 fixture What is verified The config is loaded — the runner echoes it back verbatim: So this is not a typo, a wrong path, or a file GitHub failed to find. What I have not established Why. Candidates I did not confirm: an interaction with Not doing next, without a decision from @parthrohit22 The tempting move is to "fix" the fixtures' syntax. That would be wrong: their manifests assert Options as I see them:
My recommendation is (1) now and (3) documented, because the warning is cosmetic: it reports that CodeQL could not parse two files that are supposed to be unparseable. It is not hiding a vulnerability. The #280 half of this work is unaffected and already merged — the Security tab is current ( |
|
Closing. This change does not work and should not be carried as pending config debt. Decisive evidence — scanned-file counts before and after the exclusion are byte-identical:
A working exclusion would show 235 Python files (248 minus 13 fixture Why — the warning is emitted by the extraction-phase diagnostic queries ( Roadmap decision (§23 / §28 merge gate) — this advances no §23 workstream toward a §28 exit criterion. It is cosmetic: open code scanning alerts are 0, and the warning truthfully reports that CodeQL could not parse two files that are required to be unparseable (manifests assert The useful half of this work, #280, is merged and verified: a push-event CodeQL run now fires on Correct resolution is to dismiss the two alerts as used in tests in the Security tab — accurate, no code change, no fixture damage. Documented in #282. |
Summary
With the default branch now scanned on push (#280), the Security tab is current —
last scan nowat319870a7instead of 5 days stale — and that surfaces the real remaining warning:One per language (
language:pythonandlanguage:javascript-typescript), which accounts for the "1 warning" plus "CodeQL also found 1 other warning like this" on each configuration page.Both point at the Repository Intelligence golden benchmark corpus:
These are supposed to be unparseable. They are adversarial inputs to the RI engine, not code PARTHA executes. Each is pinned by a manifest asserting the malformed-source diagnostic:
So the malformed syntax is the fixture contract. GitHub's own suggestion — "check the syntax of the file and correct any invalid syntax" — is the wrong move here: it would silently delete
py.syntax_errorandts.syntax_errorcoverage from the benchmark while making the warning disappear. The other half of GitHub's advice, "or exclude the file from analysis", is the correct one.Linked issue
None — found while verifying #280 against the Security tab.
What changed
Infra
.github/codeql/codeql-config.yml—paths-ignoreforapps/backend/tests/benchmark/fixtures, with the rationale recorded inline so the next reader doesn't "fix" the fixtures..github/workflows/codeql.yml— wires the config in viaconfig-fileon theinitstep.No application code, contracts, schemas, or fixtures changed. The malformed files are deliberately left exactly as they are.
Acceptance criteria completed
pushtrigger from fix(security): scan the default branch on push to keep CodeQL current #280 verified intact.Testing performed
Config and wiring asserted, including a blast-radius check that the exclusion cannot leak into application code:
RI golden benchmark, confirming the fixtures still behave as their manifests require:
Honest limit: whether CodeQL stops emitting the warning can only be proven by GitHub running the analysis with this config. The definitive evidence is the two configuration pages dropping from "1 warning" to no warnings after this merges. I cannot produce that locally and am not claiming it.
Screenshots
Not applicable — no UI change. The relevant surface is the Security tab, which updates after merge.
Security and data considerations
Narrowly scoped and verified: the exclusion covers only
apps/backend/tests/benchmark/fixtures, a directory containing exclusively benchmark fixture data. All 282 application source files underapps/backend/appandapps/frontend/srcremain in scope, asserted in the test above.This is a genuine improvement to signal quality rather than suppression. A permanent unfixable warning on the Security tab trains reviewers to ignore that surface, which is how a real finding gets missed. No
permissions, secrets, egress, auth, or owner-scoping changes.Dependencies and blocked work
None. Builds on #280 (merged).
Scope changes or remaining work
None.
Contributor checklist
dev.env,dist/, or generated artifacts committed