Skip to content

Revisit trufflehog CLI scan & non-blocking Hypatia once upstream toolchains are reliable #47

@hyperpolymath

Description

@hyperpolymath

Context

PR #41 stabilised three pre-existing CI checks. Two of the fixes are deliberate workarounds whose ideal end-state depends on external toolchains becoming reliable. This issue tracks reverting them when that happens. (A2ML has its own tracking issue: #44.)

1. trufflehog — re-adopt the action wrapper when its PR range-scan is reliable

Workaround (in .github/workflows/secret-scanner.yml): replaced trufflesecurity/trufflehog@<sha> with a pinned, deterministic full-history CLI scan:

trufflehog git "file://$GITHUB_WORKSPACE" --only-verified --fail --no-update

Why: the action wraps a Docker range-scan (--since-commit BASE --branch HEAD against ghcr.io/...:latest) that fails on PRs even with zero secrets. A full-history scan (trufflehog 3.95.3) confirmed 0 verified / 0 unverified secrets — there is no real leak; the failure was purely the wrapper.

  • Re-evaluate the official action once its PR-mode range-scan is reliable / pinnable
  • If re-adopted: restore uses: trufflesecurity/trufflehog@<pinned-sha>; otherwise periodically bump the pinned CLI version (currently v3.95.3)
  • trufflehog check stays green

2. Hypatia — restore strictness once the upstream build is reliable

Workaround (in .github/workflows/hypatia-scan.yml): the scan job is fully non-blocking — every fragile step has step-level continue-on-error: true, the scan step has a guaranteed-valid-JSON fallback, and HYPATIA_DIR was fixed (was expanding from the empty ${{ env.HOME }}/hypatia). Root causes also fixed: bogus cd scanner (no such subdir upstream) and an invalid setup-beam pin (elixir 1.19.4 / otp 28.3 not in the version index → repinned 1.17 / 27).

Why non-blocking: the scanner is cloned and built from the external hyperpolymath/hypatia repo and run with --exit-zero; build/runtime failures there are not this repo's content and must not gate merges (mirrors the #39 canary precedent).

  • When hyperpolymath/hypatia builds reliably (escript or bash fallback), consider tightening: drop continue-on-error from the non-external steps so genuine scanner regressions are visible
  • Keep continue-on-error only on the genuinely external clone/build steps
  • Revisit pinning setup-beam to exact patch versions once a known-good pair is chosen
  • Hypatia Neurosymbolic Analysis stays green

References

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions