Improve database credential redaction after betterleaks#1045
Merged
gtrrz-victor merged 7 commits intomainfrom Apr 28, 2026
Merged
Improve database credential redaction after betterleaks#1045gtrrz-victor merged 7 commits intomainfrom
gtrrz-victor merged 7 commits intomainfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR strengthens Entire CLI’s automatic transcript redaction by adding more secret detectors (beyond entropy + gitleaks) and expanding regression coverage to reduce both under- and over-redaction in common credential formats.
Changes:
- Add redaction for full private key blocks, credentialed URIs, database DSNs/URLs, and bounded password-like key/value pairs.
- Enhance JSON/JSONL redaction to support key-scoped replacements for structured credential fields.
- Expand unit tests and update security/privacy documentation to reflect the new detection layers.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| redact/redact.go | Adds new secret detectors (private keys, credentialed URIs, connection strings, bounded credentials) and extends JSONL replacement handling. |
| redact/redact_test.go | Adds extensive coverage for new redaction behaviors, including guardrails against over-redaction. |
| docs/security-and-privacy.md | Updates documentation to describe the expanded automatic secret redaction methods. |
Entire-Checkpoint: d271418e7c6f
Entire-Checkpoint: 7242e212bf5b
Entire-Checkpoint: 2dc818c76ce5
Contributor
Author
|
@BugBot review |
Entire-Checkpoint: de0e66c1ffc3
Contributor
Author
|
@BugBot review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 3115fad. Configure here.
Entire-Checkpoint: 93f541fa4dd4
Hot-path simplifications in redact/redact.go: replace per-replacement regex compilation with a manual JSON scan (replaceKeyedJSONValue), derive lowercase placeholder forms from RedactedPlaceholder, early-out on inputs without '=' before running connection-string rules, and single-pass key normalization in isCredentialJSONObject. Coverage and review follow-ups: broaden credentialValuePattern boundary so prefixed env vars (APP_DB_PASSWORD, PROD_MYSQL_PWD) match; add a Close goroutine to TestLogging_ConcurrentInitAndLog to exercise the close-during-write race; document why log() holds the read lock across l.Log; document why dispatch tests using newDispatchProgram cannot use t.Parallel; clarify docs/security-and-privacy.md that connection-string redaction is conditional on a real (non-placeholder) password. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 084f6db5e688
gtrrz-victor
approved these changes
Apr 28, 2026
This was referenced Apr 28, 2026
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Redact for database credential gaps that remain after #1043 switched redaction to Betterleaks.
DB_PASSWORD=...,PGPASSWORD=..., andREDIS_PASSWORD=...while preserving placeholders and already-redacted values.{"db":{"password":"...","host":"...","user":"..."}}without redacting unrelated fields that happen to share the same value.Scope After #1043
scheme://user:password@hostURI userinfo redaction is covered by the existingcredentialedURIPatterninredact/redact.go, so the DB-specific detector no longer owns userinfo URLs.Also Included
Two unrelated concurrency fixes that surfaced during this work:
cmd/entire/cli/logging/logger.go— holds the read lock acrossl.LogsoInit/Closecannot close the log writer mid-write. AddsTestLogging_ConcurrentInitAndLogas a regression guard (now exercisesClosetoo).cmd/entire/cli/dispatch_tui_test.go— dropst.Parallel()on two tests that mutate the package-levelnewDispatchProgramfactory; adds a doc comment on the var so future contributors don't reintroduce the race.Bundled here rather than split into separate PRs because both are small.
Testing
GOCACHE=/tmp/entire-go-cache go test ./redact -count=1mise run checkmise run lintNotes
mise run checkandmise run lintwere run outside the sandbox because golangci-lint fetches its schema, Go uses the user cache, and integration tests bind localhosthttptestservers.Note
Medium Risk
Medium risk because it changes always-on redaction behavior (could still under- or over-redact real user data) and adjusts shared logger state access patterns under concurrency.
Overview
Improves secret redaction coverage for database credentials.
redact.Stringnow adds always-on detectors for DB connection strings (JDBC, keyword DSNs, semicolon/ODBC-style, and DB URLs withpassword/pwdquery params) plus bounded credential assignments likeDB_PASSWORD=..., while explicitly skipping placeholders/already-redacted values.Makes JSON/JSONL redaction more field-aware. JSON replacement tracking now keys replacements by JSON field so skipped fields (e.g.,
session_id) don't get redacted due to value collisions elsewhere, and structured credential objects/keys (normalized variants like"DB Password") get their password values redacted without touching unrelated fields.Hardens CLI logging under concurrency. Removes helper getters and reads
logger/currentSessionIDunder the package RW lock insidelog, and adds a concurrentInit+log stress test. Docs are updated to describe the expanded redaction layers.Reviewed by Cursor Bugbot for commit 3115fad. Configure here.