FAQ

FAQ

Short answers with links to the canonical source. See also the README's Known limitations.

Thread safety

Is redact / scan safe to call from multiple threads? Yes. Compiled patterns are immutable and shared (read-only after load); all per-scan mutable state — NFA scratch and the lazy-DFA cache — lives in per-thread storage, so concurrent scans never touch each other's state. For inputs above a few KB, redact releases the GVL around the built-in scan, so a large redaction on one thread doesn't block other Ruby threads. A thread's state is freed when the thread exits, so many short-lived scanning threads don't leak memory.

Can I register custom patterns from a running server? Yes. add_pattern, remove_pattern, and clear_custom_patterns! are thread-safe — the shared custom-pattern array is guarded by a mutex that writers take around the mutation and redact/scan take around their custom-pattern loop. Registration is rare, so the lock is uncontended in practice.

Full detail: README Thread safety · C Engine Internals.

What is not redacted

• Base64 attachments (inline PDFs, images, audio) and URL-referenced files — the sensitive bytes are encoded or remote, so the patterns can't see them. This applies everywhere, including transparent RubyLLM mode.
• AWS Secret Key (40-char base64) is a broad match and can false-positive inside base64-encoded blobs (embedded images, binary). It's kept because missing a real secret is worse than an occasional over-redaction.
• Hash keys — redact_deep / redact_json only touch values. Keys, and non-string scalars (Integer, Float, nil, Boolean), pass through unchanged.
• Your org's internal IDs and people's names — until you register them. See Custom Patterns & Name Patterns.

Duplicate digit patterns

Several national IDs share a digit length (11 digits: PESEL, Norwegian Fødselsnummer, Belgian National Number). They're separate catalogue slots for clarity, but in practice any 11-digit boundary-delimited number is redacted. This is deliberate — the engine leans toward redacting when uncertain. See Pattern Catalogue.

Why can't I use \d, \b, or lookahead in a custom pattern?

Custom patterns use the same POSIX ERE engine as the built-ins. Use [0-9] for \d, [[:space:]] for \s, and so on. Unsupported constructs raise DataRedactor::InvalidPatternError at registration time, never mid-redaction. Full table: Custom Patterns & Name Patterns.

How does overlap resolution work?

Longest-match-wins. When two patterns match overlapping spans, the engine keeps the longest span; equal-length ties go to the lower pattern index. This biases toward redacting more — a 40-char secret is redacted whole rather than leaking the bytes past a shorter prefix match. A rare edge: two secrets abutting with no separator can leave the second unredacted if a boundary-wrapped pattern finds no word boundary between them. Real text almost always separator-delimits secrets. See Known limitations.

Does redact mutate my input?

No. redact, redact_deep, and redact_json always return a copy. (The Rails filter_parameters adapter is the one exception — it mutates string values in place via String#replace because Rails' filter contract requires it.)

How do I redact everything I send to an LLM?

Use the ⭐ RubyLLM Integration — per-call, or transparent install! mode that scrubs every request automatically. For the Anthropic/OpenAI SDKs directly, see Integration Guides.

Where's the API reference / changelog / research?

• API reference (method signatures, params): GitHub Pages — this wiki does not duplicate it.
• Release history: CHANGELOG.
• Why decisions were made / prototypes: DONE.md and the research log.