Search and Rules

Search and Rules

Finding every occurrence of something, and deciding about all of them at once, at the scope of a document, a batch, or everything you upload. Shipped in 1.9.0 (#142).

Canonical sources: docs/editor.md §14–15 · docs/api.md (search, rules) · docs/security-internals.md §7 · code: lib/redaction/patterns.ts, search.ts, rules.ts, owner-rules.ts.

---

1. One pattern compiler for everything

Search, rules and Hush all compile the same pattern (lib/redaction/patterns.ts):

type PatternSpec = { kind: "literal" | "regex", pattern: string, matchCase: boolean, wholeWord: boolean }

Kind	Matches
literal	the text, case-insensitively unless matchCase
regex	an RE2 pattern (RE2JS): linear-time, ^/$ match at line breaks

"Whole word" is Unicode-aware (so café works), not RE2's ASCII \b. The same compiler runs in the browser to validate as you type, so the browser and the server cannot disagree about what is allowed. That is what lets "Redact all" promise the number on its button.

The defences, in order

A RegEx that backtracks catastrophically ((a+)+$ against a long line of as) would be a denial of service with a text box in front of it.

Defence	Value	Why
Linear-time engine	RE2JS	Matching is linear whatever the pattern. Backreferences and lookaround do not compile. This is the real defence; the rest are belt and braces.
Length limit	PATTERN_MAX_LENGTH = 500	bounds compile cost; keeps a pattern reviewable
No empty matches	refused at compile	a pattern that can match nothing matches between every character pair
Time budget	RULE_BUDGET_MS = 5000 per document	measured on the matching alone, so slow storage cannot trip it
Match cap	RULE_MATCH_LIMIT = 10,000 per document (search: 50,000)	\w+ is safe to run and would still redact every word

A pattern over budget fails whole with 422. Nothing is written, because a rule applied partway is worse than none.

Patterns travel in request bodies, never in query strings. A search term is usually the exact value being redacted, and URLs are what access logs, proxies and browser history keep.

---

2. Search

Ctrl/⌘+F or / opens a find strip docked across the top of the canvas. It replaces the browser's find on this screen, which cannot read a PDF page drawn on a canvas and cannot see pages that are not loaded. On the one screen where "not found" matters most, the browser would say it with confidence and be wrong.

The strip: the query, with match case / whole word / RegEx toggles and an "N / M" counter inside the field; previous / next; a split Redact this | Redact all M button; the results toggle; this batch; close.

It runs on the server, over the normalized model a page at a time, so a 400-page PDF is searched without 400 pages reaching the browser. There are three requests, each made only when needed:

Request	When	Answer
Summary (POST /api/documents/:id/search)	once per question, debounced	hits per page, matching cells, total
Page hits (same, with page)	once per page shown, per question	{ start, end } offsets on that page
Batch (POST /api/batches/:id/search)	when "this batch" is on	a count per document (null + a note while a document is still processing, never 0)

Every answer carries the key of the question it answers, so a slow answer to "Smi" cannot be drawn over the answer to "Smith". A spreadsheet cell counts once however often it matches, because a cell is redacted whole. The total is therefore exactly what "Redact all" would write.

• Results panel: a column docked beside the canvas (not floating over the text it lists) with the first 200 hits in context, grouped by page or by cell. Clicking a hit makes it current and moves there. With "this batch" on, a second tab lists every document with its count and a link.
• Rail badges: each thumbnail shows its page's hit count.
• Moving: Enter / Shift+Enter step through hits in page order (then cells), wrapping at either end. The first hit shown is the first one at or after the page you were on.
• Highlighting: amber, never red or black, because red means a suggestion and black a decision, and a hit is neither. Text flow uses the CSS Custom Highlight API over the drawn text (without splitting spans, so a search never changes what a click does). PDF, image and OCR hits resolve to rectangles through boxesForRange, the same resolution a redaction gets. Spreadsheets highlight whole cells.
• From a hit to a decision: Redact this makes the current hit a manual redaction and moves on. Redact all opens the rule dialog with the search filled in.
• Esc closes the strip and clears the highlights; the query is kept for next time.

Rate limit: whole-document searches use search (ANONIFY_RATE_LIMIT_SEARCH, 60/min demo, 600/min self-hosted); page hits use read.

---

3. Rules

A rule is a pattern, a category, a scope and a kind. It is searched against the normalized model on the server, never through a model, and every match is written as an accepted redaction carrying the rule's id. Removing a rule removes exactly what it created.

Scope	Reaches	Stored as
Document	every match in this document	GlobalRule on the document
Batch	every document in the batch, including ones still processing (they pick it up in carryDecisions)	BatchRule, materialised as a GlobalRule per document
Global	this document, and every document you upload from now on	OwnerRule, materialised per document

The rule dialog

The one way a rule is made or changed. It opens from Redact all in search, the Rules panel, a Hush proposal, or a learned-shape offer.

1. As you type, the pattern is checked in the browser by the server's compiler.
2. It is previewed (POST /api/documents/:id/rules/preview): the count and the first 20 matches in context. For a batch rule, the count in every other document too, and the document that would fail if it goes over budget.
3. The confirm button names the number of redactions it will write. Nothing is written until it is pressed.

Every document a rule reaches is planned within budget before anything is written, then the rule and all its redactions are written in one transaction. A rule is never partly applied: a pattern that runs out of budget in the fifth file leaves the first four untouched.

The Rules panel

The inspector's Rules tab lists every rule in scope for the open document, grouped by scope, each with its kind, category and reach ("12 here · 40 in 4 documents"). Each rule can be:

• Switched off: its redactions go and it stays listed, so switching it back on is one click. A batch or global rule that is off is not carried into documents that finish later.
• Edited through the dialog, with a fresh preview. The edit replaces what the rule redacted everywhere it reached, in one transaction. A match you had rejected comes back accepted, because the edit is a new decision and its preview showed every match.
• Removed after a confirmation that says how many redactions go with it, in how many documents.
• Improved with Hush (RegEx rules only): Hush looks at what the rule matched and what was rejected, tests a tighter pattern with compare_patterns, and proposes it as an update_rule card. See Hush Assistant.

The inspector's per-suggestion Everywhere and Whole batch buttons still make literal rules at those scopes without the dialog.

Global rules

• The pattern is sealed with the master key. It is document content kept longer than any document, so it is never stored in the clear. A document's copy of it dies with that document.
• It expires 30 days after it was last used (applied, edited, switched on). The cleanup sweep prunes expired ones. At most 200 per owner.
• It belongs to your browser session. The session cookie now renews whenever you upload, create, edit or import a rule, or talk to Hush, so a rule in use never outlives the cookie that reaches it. Clearing cookies, a private window or another browser is a new session that cannot reach your rules. Export them before then.
• Export / Import (in the panel's Global section): GET /api/rules/export produces { "format": "anonify.rules", "version": 1, "rules": [{ kind, pattern, matchCase, wholeWord, category, enabled }] }, with patterns and options only (no counts, ids or dates), because the file travels to other people and instances. POST /api/rules/import compiles each rule, reports the ones this instance refuses, skips duplicates, and applies the rest to future uploads. This is how a team shares one rule set, and how anyone keeps rules for longer than 30 days.

Carried rules can fail a document

A batch or global rule that goes over budget while being carried into a newly processed document fails that document as rule-too-broad. The document still opens for review, and a Retry after narrowing or switching off the rule produces a different result. See Failure Codes.

---

4. The API

Method	Path	Does
POST	/api/documents/:id/search	summary, or one page's hits
POST	/api/batches/:id/search	count per document
GET	/api/documents/:id/rules	every rule reaching this document, with reach
POST	/api/documents/:id/rules/preview	count and samples, nothing written
POST	/api/documents/:id/rules	create at document / batch / global scope
PATCH	/api/documents/:id/rules	switch off/on, or rewrite, everywhere it reached
DELETE	/api/documents/:id/rules	remove, with its redactions
PATCH / DELETE	/api/batches/:id/rules	the same, from the batch page
GET	/api/rules	your global rules, unsealed for you
GET	/api/rules/export · POST /api/rules/import	the JSON rule file

Details, bodies and status codes: API Reference.

---

See also

• Review Workspace · Hush Assistant · Batches and Mailboxes · Data Model