Security hardening, case-path performance, and reliability fixes - #26
Merged
Conversation
…ion creds & plugin env, gRPC TLS Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ging, CI coverage & lint, hygiene Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…anic fixes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e, retry, SDK timeout/panic)
…mat, logging, CI coverage, hygiene)
…security engine The io.EOF sentinel for a missing response field is now a descriptive error, and the security engine runs scenarios concurrently, so the ordering-based sequenceTarget is replaced with a scenario-name-keyed target in the coverage test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…behavior in repo knowledge Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The repo enforces that all _test.go files live under tests/ (validateGoFileLayout
in internal/devtools). The security agent's in-package white-box tests broke
'make fmt' in CI. Relocated them as black-box tests under tests/{mcp,integrations,plugins}:
- MCP guard + path traversal tested via exported GuardMCPConfig/LoadConfigSource.
- Credential egress: extracted the pure policy into exported CredentialEgressAllowed
(used by applyAuth) so it is testable without a live HTTP server.
- Plugin env: exported BuildEntryEnv/BuildWASMEnv (pure functions over Entry).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Clears the two blocking semgrep findings introduced by the hardening work: - gRPC TLS credentials now pin MinVersion to TLS 1.2 (missing-ssl-minversion). - Retry backoff jitter uses crypto/rand instead of math/rand (math-random-used); randomness quality is immaterial for jitter, this just satisfies the scanner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A codebase-wide audit turned up security, performance, and reliability issues; this PR fixes them. Work was split across three cohesive areas (security, hot-path performance/correctness, CLI/tooling/hygiene) with disjoint file ownership and integrated here.
Static checks on the merged tree are green:
go build ./...,go vet ./..., fullgo test ./...all pass, and-raceis clean on the newly-concurrent engine/adapter paths.Security
cleanr_run/cleanr_generate_datasetaccepted inline configs with no sandbox, so a prompt-injected agent could runtype: clitargets orplugins/state_adapters/probes(all inheriting the full host env, including provider API keys). Newtoolkit.GuardMCPConfigrejects these on the MCP surface unlessCLEANR_MCP_ALLOW_EXECis set. The CLI path is unchanged.config_path/dataset_path/etc. are now confined to the working directory (no absolute paths, no..), and errors no longer echo file contents.api_key_envand have its value sent as a Bearer token to any URL. Provider-secret env vars (OPENAI/ANTHROPIC/AWS/…) are now only sent to an egress allowlist or loopback; every send/refusal is logged.os.Environ()— only declared vars are passed.grpc.plaintext: true.Performance & correctness
Configs never ranapplyDefaults, soTimeout()returned 0 and every request failed instantly withcontext deadline exceeded.Timeout()now falls back to a sane default.LoadEnginefixed. A load suite with zero scenarios hit a%len(scenarios)divide-by-zero — now guarded (a panic in the public SDK is an API bug).prompt_injection,security,token_optimization,release_policy,provenance) now run scenarios concurrently viarunBoundedByIndex, limit fromConfig.Concurrency(default 4), results written by index so ordering stays deterministic. Roughly an order-of-magnitude CI wall-clock win on large suites.Retry-After, never exceeds the request deadline).io.EOFsentinel.Deferred (documented):
llm_judgescenario-loop concurrency is left serial due to stateful sampling;drift/claim_trace/shadow_statestay serial by design.CLI, tooling & hygiene
SIGINT/SIGTERMhandled gracefully with a partial report on interrupt; trend/snapshot files written via temp-file + rename so an interrupt can't corruptcleanr.trends.yaml.htmlreport format now validates (it was implemented and CLI-documented but rejected by config validation).log/slog) with a-v/--debugflag at the CLI boundary../cleanr/...in addition to./internal/...(the attestation/signing path was previously unmeasured); threshold kept at a conservative floor with a note to ratchet up.default: nonetostandard+ errcheck/staticcheck/bodyclose.cleanr-devbinary and 936KBcoverage.out(both are build artifacts;makerebuildscleanr-devfrom source).Reviewer notes / behavior changes
grpc.plaintext: true. Loopback is unaffected.Config.Concurrencyfield (default 4 viaConfig.CaseConcurrency())..claude/knowledge/architecture-boundaries.md.Test plan
go build ./...go vet ./...go test ./...(all packages pass)go test -race ./cleanr/engines/... ./cleanr/adapters/... ./tests/engines/...(clean)🤖 Generated with Claude Code