-
Notifications
You must be signed in to change notification settings - Fork 1
FAQ
Short answers with links to the canonical source. See also the README's Known limitations.
Is redact / scan safe to call from multiple threads? Yes. Compiled
patterns are immutable and shared (read-only after load); all per-scan mutable
state — NFA scratch and the lazy-DFA cache — lives in per-thread storage, so
concurrent scans never touch each other's state. For inputs above a few KB,
redact releases the GVL around the built-in scan, so a large redaction on
one thread doesn't block other Ruby threads. A thread's state is freed when the
thread exits, so many short-lived scanning threads don't leak memory.
Can I register custom patterns from a running server? Yes.
add_pattern, remove_pattern, and clear_custom_patterns! are thread-safe —
the shared custom-pattern array is guarded by a mutex that writers take around the
mutation and redact/scan take around their custom-pattern loop. Registration
is rare, so the lock is uncontended in practice.
Full detail: README Thread safety · C Engine Internals.
- Base64 attachments (inline PDFs, images, audio) and URL-referenced files — the sensitive bytes are encoded or remote, so the patterns can't see them. This applies everywhere, including transparent RubyLLM mode.
- AWS Secret Key (40-char base64) is a broad match and can false-positive inside base64-encoded blobs (embedded images, binary). It's kept because missing a real secret is worse than an occasional over-redaction.
-
Hash keys —
redact_deep/redact_jsononly touch values. Keys, and non-string scalars (Integer, Float, nil, Boolean), pass through unchanged. - Your org's internal IDs and people's names — until you register them. See Custom Patterns & Name Patterns.
Several national IDs share a digit length (11 digits: PESEL, Norwegian Fødselsnummer, Belgian National Number). They're separate catalogue slots for clarity, but in practice any 11-digit boundary-delimited number is redacted. This is deliberate — the engine leans toward redacting when uncertain. See Pattern Catalogue.
Custom patterns use the same POSIX ERE engine as the built-ins. Use [0-9]
for \d, [[:space:]] for \s, and so on. Unsupported constructs raise
DataRedactor::InvalidPatternError at registration time, never mid-redaction.
Full table: Custom Patterns & Name Patterns.
Longest-match-wins. When two patterns match overlapping spans, the engine keeps the longest span; equal-length ties go to the lower pattern index. This biases toward redacting more — a 40-char secret is redacted whole rather than leaking the bytes past a shorter prefix match. A rare edge: two secrets abutting with no separator can leave the second unredacted if a boundary-wrapped pattern finds no word boundary between them. Real text almost always separator-delimits secrets. See Known limitations.
No. redact, redact_deep, and redact_json always return a copy. (The Rails
filter_parameters adapter is the one exception — it mutates string values in
place via String#replace because Rails' filter contract requires it.)
Use the ⭐ RubyLLM Integration — per-call, or transparent
install! mode that scrubs every request automatically. For the Anthropic/OpenAI
SDKs directly, see Integration Guides.
- API reference (method signatures, params): GitHub Pages — this wiki does not duplicate it.
- Release history: CHANGELOG.
- Why decisions were made / prototypes: DONE.md and the research log.
Guides
Reference
Elsewhere