Skip to content

Obfuscate secrets in a single scan - #3847

Merged
thomhurst merged 1 commit into
mainfrom
issue-3753-single-pass-obfuscation
Aug 4, 2026
Merged

Obfuscate secrets in a single scan#3847
thomhurst merged 1 commit into
mainfrom
issue-3753-single-pass-obfuscation

Conversation

@thomhurst

Copy link
Copy Markdown
Owner

Summary

  • replace per-secret full-input replacements with one SearchValues-driven forward scan
  • preserve longest-match precedence for overlapping patterns
  • avoid rescanning generated mask text

Validation

  • SecretObfuscatorCachingTests: 13 passed
  • SecretMasking*: 36 passed
  • ModularPipelines.slnx Release build: 0 warnings, 0 errors

Closes #3753

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review

Summary: Replaces the per-secret StringBuilder.Replace / manual IndexOf loops with a single forward scan over the input using a cached SearchValues<string>, finding the earliest match position and then resolving it against the (length-descending) secret list to preserve longest-match precedence.

Verification performed:

  • Checked out the PR branch and ran test/ModularPipelines.UnitTests filtered to SecretObfuscatorCachingTests — all 13 tests pass, including the two new tests (UsesLongestSecretWhenPatternsMatchAtSamePosition, DoesNotRescanMaskReplacement).
  • Traced the matching algorithm by hand against several overlapping/substring scenarios (shorter secret is a substring of a longer one at the same start index, secrets with different lengths starting at different indices, secrets equal in length). IndexOfAny(searchValues) locates the leftmost position where any secret starts, and the subsequent StartsWith scan (secrets pre-sorted longest-first via OrderByDescending(secret => secret.Length) in CreateSecretCache) always resolves to the longest secret matching at that position — this correctly reproduces the "longest match wins" behavior the old two-branch implementation aimed for, while also fixing the case-insensitive path's leftmost-match tie-breaking.
  • Confirmed the new single StringBuilder never re-scans previously appended mask text (it only walks the original input span and advances inputOffset past the matched secret), which is what the new DoesNotRescanMaskReplacement test targets and is a real behavioral improvement over the old case-sensitive path (StringBuilder.Replace per secret could, in principle, re-scan across replacement boundaries between separate secrets).
  • No CLAUDE.md violations found — the change is confined to src/ModularPipelines/Engine/SecretObfuscator.cs and its existing unit test file, doesn't touch generated code, and the updated XML doc comment on the class accurately reflects the new implementation.

Design note (non-blocking): ObfuscateMatches still does a linear StartsWith scan over secrets at every match position to determine which secret matched. For the typical secret-set sizes here (tokens/credentials registered per pipeline run) this is fine and is a clear net win over the old N-passes-over-the-whole-input approach. If secret counts ever grow large (hundreds+) and profiling shows this loop becoming hot, a follow-up could group secrets by first character (or build a small trie) to cut the per-match resolution cost — but that's speculative and not warranted by this PR's scope.

No issues found. Good, well-tested performance improvement.

@thomhurst

Copy link
Copy Markdown
Owner Author

Ubuntu pipeline failed only in the known retry timing race When_Retry_With_Timeout_Then_Honour_Overall_Timeout (expected one execution, observed two), unrelated to this PR's single-pass secret obfuscation. The root fix is pending in #3837. Triggered the single allowed failed-job rerun for workflow 30946543487.

@thomhurst
thomhurst merged commit 842e111 into main Aug 4, 2026
18 of 19 checks passed
@thomhurst
thomhurst deleted the issue-3753-single-pass-obfuscation branch August 4, 2026 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Perf: obfuscation does one full pass over the input per secret pattern instead of a single SearchValues-driven scan

1 participant