Writing a Check

Writing a check

1. Pick or add a control in internal/catalog/catalog.go. Write the title as a narrow, checkable statement.
2. Add the detection:
   • single-line pattern: add a lineRule to codeRules in internal/adapters/builtin_code.go
   • structured file (YAML, JSON, XML): add logic to the matching adapter in builtin_static.go
   • new tool: add an externalTool in external.go with argument builder, accepted exit codes and a normalizer
3. Choose confidence honestly: Confirmed only when the observation itself decides the control; otherwise Hypothesis.
4. Write Message (why it matters), Fix (what to do) and, if useful, Retest.
5. Add a vulnerable case to internal/testfixture and assert the rule fires in internal/engine/engine_test.go. Add a safe analog that must not fire.
6. Add at least one vulnerable and one safe case for the rule to the benchmark corpus in internal/benchmark/corpus.go. go test ./internal/benchmark fails when a rule has no vulnerable or no safe case, or when it misses a vulnerable case or fires on a safe one. A rule is implemented_and_benchmarked only with at least 3 of each and precision and recall of at least 0.95. Semgrep rules live in rules/semgrep/ with their own fixtures.
7. Run gofmt -l ., go vet ./..., go test ./... and pandaward benchmark --strict. See Benchmark and Self-Test.

Never put real-looking secrets in source or fixtures. Assemble secret-shaped strings at runtime like testfixture.FakeStripe.