test: pin the gitleaks allowlist against silent rot (Section 207) - #98
Merged
Conversation
PR #97 fixed the weekly full-history scan, but nothing prevents that fix from being silently undone. The push and pull_request gitleaks runs scan only the new commits, so a broken allowlist yields a green PR and does not surface until the following Monday. Section 207 asserts the invariants that keep the suppression safe: - The singular [allowlist] table is used and [[allowlists]] is not. gitleaks 8.24.3 silently ignores the plural array at global scope: it parses, warns about nothing, and suppresses nothing. Upstream docs show the plural form, so this is easy to "modernise" into a no-op. - No `paths` or `commits` entries, which would blind the scanner to whole files or whole commits instead of to two exact values. - useDefault stays true, so the allowlist is not layered over an empty ruleset that scans for nothing. - Exactly two reviewed entries, and no catch-all regex. - The workflow keeps its schedule trigger and fetch-depth: 0, without which nothing ever scans history at all. The section deliberately does not embed the two allowlisted literals; it counts entries instead, so the test file does not reintroduce the strings it exists to describe. Mutation-verified: violating all six config invariants and both workflow invariants at once fails 9 of the 10 asserts and the suite exits 1. The tenth ("file present") correctly still passes, since the file existed. Structural test count 5428 -> 5438; README badge synced by the suite.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #97.
Problem
#97 got the weekly full-history gitleaks scan back to green, but nothing stops that fix from being silently undone.
The workflow has two behaviours and only one of them can fail on history:
push/pull_request→ scans only the new commits. Green regardless of what is in history or what the allowlist does.schedule/workflow_dispatch→gitleaks detectover full history.So a broken allowlist produces a perfectly green PR and stays invisible until the following Monday's failure email. There is no pre-merge signal at all. That is precisely how the previous eleven-week red streak went unnoticed.
What this pins
New
Section 207: SECRET-SCANNER CONFIG INTEGRITYinTests/Run-Tests.ps1, following the Section 206 pattern:[allowlist]table is used[[allowlists]]is absentpaths =entrycommits =entryuseDefault = true'''.*'''silencing the scanner wholesaleschedule:fetch-depth: 0The plural-array assert is the important one. Most current gitleaks documentation shows
[[allowlists]], so "modernising" this file is an easy and completely silent way to break it.The section deliberately does not embed the two allowlisted literals — it counts entries instead. Embedding them would reintroduce the exact strings the config exists to describe into a new file.
Verification
Passing: all 10 asserts pass; suite green at 5438 tests, 0 failed, 0 skipped, exit 0.
Mutation-verified — a guard that cannot fail is decoration. Violating all six config invariants and both workflow invariants simultaneously (
[[allowlists]],paths,commits,useDefault = false, 4 entries, catch-all regex,schedule:stripped,fetch-depth: 1):9 of 10 fail and the suite exits 1. The tenth correctly still passes — the file did exist. Both mutated files were restored from git and verified byte-identical to master.
Also re-ran a local full-history gitleaks scan on this branch: 592 commits, no leaks found — the new test code introduces no findings of its own.
Tests/Run-Tests.ps1parses clean and is PSScriptAnalyzer-clean under the repo settings. README badge 5428 → 5438, synced by the suite's own self-heal rather than by hand.