Supported Formats

Supported Formats

Eleven formats: ten documents plus one container. Each format hides sensitive text somewhere different, so each pipeline is built around the specific way that format can betray you. Every one of them produces the same normalized model and answers the same question at export time: is the string actually gone?

The register that defines them lives in one place, lib/documents/formats.ts. The upload allow-list, the file picker's accept, the refusal message, the quota mapping and the export output types all derive from it, and the table is typed so that adding a kind without registering it does not compile. A guard test also requires an adversarial case for every format.

Canonical source: docs/pipelines.md.

---

At a glance

Format	What gets removed	How it is removed	Verified by	Quota unit	Read by
PDF	Text on any page, including scanned pages	Redacted pages rasterised with boxes burned in; other pages keep vector text	extracted text, raw byte scan, no annotations, zero spans on redacted pages	pages	ranged (pdf.js range transport)
Word (.docx)	Text nodes, in place	In-place run surgery: styles, numbering, relationships survive; headers, footers, footnotes, comments reviewable	every .xml / .rels part	pages	ranged zip, XML parts only
Excel (.xlsx)	Cells, rows, columns	Cells rewritten; formulas referencing a redacted address dropped; hidden sheets read	every cell of every sheet + every part	filled cells	ranged zip, one sheet at a time
PowerPoint (.pptx)	Slides, speaker notes, layouts, master	The same OOXML surgery across all four	the whole package	slides	ranged zip, XML parts only
Email (.eml)	Headers, every body, HTML (as markdown), link targets, quoted replies, filenames, nested messages	Byte-range replacement in the original; attachments become child documents	our parser and postal-mime; attachments by checksum	KiB of decoded text	forward scan, attachments never held
Mailbox (.mbox)	Nothing: a container	Expands into one document per message; comes back as a rebuilt mailbox	conservation on split; re-split and byte-compare on rebuild	nothing of its own	streamed split
CSV	Cells	Parsed grid, rewritten cell by cell	re-parse, every cell	filled cells	streamed
TSV	Cells	Same grid model as CSV	re-parse, every cell	filled cells	streamed
Plain text (.txt)	Offsets	Offset replacement	the exact text	pages of text	streamed
Rich text (.rtf)	Parsed text	Parsed through an atom map, not searched	re-parse (must still open)	pages of text	whole (state machine)
Images (PNG, JPEG, WebP)	Pixels	Pixels replaced, file re-encoded; EXIF and GPS removed	pixel sampling, metadata	one per image	whole (decoded raster)

Every export is re-opened and read the way an adversary would. A surviving value fails the export rather than shipping (lib/redaction/validation.ts).

flowchart TB
    SRC["Sealed source: layer A"] --> KIND["formats.ts: the sniffed kind chooses the pipeline"]
    KIND --> PDF["pdf: pages, glyph boxes, OCR for scans"]
    KIND --> OOX["docx · pptx · xlsx: OOXML package surgery"]
    KIND --> GRD["csv · tsv: a parsed grid"]
    KIND --> TXT["txt · rtf: offsets through an atom map"]
    KIND --> EML["eml: headers, every body, nested messages"]
    KIND --> IMG["png · jpeg · webp: pixels + OCR + metadata"]
    KIND --> MBX["mbox: a container that expands into a batch"]
    PDF --> NORM["One normalized model + page index"]
    OOX --> NORM
    GRD --> NORM
    TXT --> NORM
    EML --> NORM
    IMG --> NORM
    NORM --> ACC["Accepted redactions only"]
    ACC --> OUT["Format-native export, generated from the original source"]
    OUT --> VER{"Re-opened adversarially"}
    VER -- "an accepted value survived" --> REF["Refused: nothing stored, no link"]
    VER -- clean --> DEL["Checksummed, sealed, signed short-lived download"]

Loading

What they share. Every pipeline produces NormalizedDocument { pages, sheets?, regions? }. A page is a flat text stream plus spans whose offsets index into it, and a span's id is its source address (word/header1.xml#p3r1, ppt/slides/slide2.xml#p0r1, text#1042, part:0.1/body#248). That single convention is why one detector, one inspector and one review flow serve ten formats, and why the exporter parses an address instead of searching for the value. See Streaming IO for how each format is read without holding the whole file.

---

PDF: the format that cannot be faked

A PDF page is a content stream of drawing operators. Drawing a black rectangle over text adds another operator, and the text underneath is still selectable, searchable and extractable. This is the single most common way redaction is done wrong, and it is why "add a black box" was never an option here.

Extraction. pdf.js text items become spans with text, offsets, font details and a bounding box flipped once from PDF's bottom-up space into top-down (flipping in two places and forgetting one is how boxes end up mirrored). Character positions come from the font's own widths, reaching below the baseline for descenders (1.5.0, GHSA-h3hv-rpqw-7p84). PDFs using the standard fonts without embedding them, or needing the bundled CMaps, render and extract correctly on the server (fixed in 1.5.0).

Scanned pages. A page with fewer than 16 characters of text is flagged, not treated as empty (a scanned page is not a blank page). It is rasterised and read with OCR, and the words become spans with real geometry, so a scan is reviewable exactly like a born-digital page. A page the recogniser cannot read stays flagged rather than presented as read and empty. Images inside a PDF are analysed too.

Export.

page has accepted redactions?
  ├── no  → copy the page through untouched (vector text survives)
  └── yes → render to raster at 2× → paint the boxes (padded 1.5 pt) → rebuild the page from that image

The trade, stated plainly: a redacted page loses its text layer. Only affected pages are rasterised, so a 100-page document with one redacted page keeps 99 pages of real text. Preserving the text layer via content-stream surgery is #40, and nearly right would mean leaking.

Verified by: extracted text; a raw byte scan with whitespace, parens and escapes stripped (so glyph-spaced text cannot slip through); the absence of /Annots and /Redact; re-extraction showing zero spans on redacted pages.

---

DOCX: surgery, not reconstruction

The text nodes that carry the characters are edited in place, so styles, numbering and relationships survive byte-identical. Rebuilding the document would be easier and would destroy everything that made it a Word document. A value straddling several runs (Word splits at every formatting change) gives each run exactly the slice it contributed.

Headers, footers, footnotes and comments are extracted and reviewable, not merely swept on the way out, because you cannot decide about a value you were never shown (#14). Addresses are part-qualified (word/header1.xml#p0r0). Page breaks and "page break before" are honoured, so a DOCX paginates (#31, #34).

Verified by: concatenating every .xml and .rels part and searching it; confirming the package still opens with its heading, table and bold run.

---

XLSX: the format with the most hiding places

Cells, rows or columns are rewritten, and any formula still referencing a redacted address is dropped, because a cached result is a second copy of the value. Hidden and very hidden sheets are read and marked (a banner and a tab marker), with counts of hidden rows and columns. Very hidden is a different problem because Excel's own menu cannot unhide it. A hidden worksheet is exactly where a value goes to survive a redaction pass (#15).

The model judges columns, not cells (headers plus five samples), producing one column-level suggestion. Workbooks are streamed a sheet at a time through exceljs's own machinery, never spilling plaintext to temp files (Streaming IO §5). Cells are charged as filled cells, not used-range area.

Verified by: every cell of every sheet including hidden ones; every XML part; asserting <f>B2</f> is gone; the shared-string table directly.

---

PPTX: four places, one of them on the screen

The same OOXML surgery across the slides, the speaker notes, the layouts and the master. Three of those four are never on screen, and a deck that redacts only its slides ships the other three. Notes, layouts and masters are processed with the slide they belong to and charged as slides.

Verified by: the whole package, not the visible slide text, and reopening the export as an OOXML package to confirm every part is still there.

---

EML: a tree, not a text file with a header

A message is a header block, the same body twice (text/plain and HTML), whatever it carried, and all of that again for every message forwarded inside it. Every header, every text part, the HTML body, quoted replies, attachment filenames and nested messages are reviewable, and the export replaces byte ranges in the original, so untouched parts come out identical to the byte.

HTML bodies are analysed as markdown (since 1.1.1, #101). An atom map links each visible character back to the markup bytes that produced it. Structural atoms (which carry no source characters and are never removed) now contribute markdown instead of a bare newline:

<h2>Attendees</h2>                    ## Attendees
<ul><li>Alice</li></ul>               - Alice
<blockquote>On Tue…                   > On Tue…
<tr><td>Alice</td><td>PM</td></tr>    | Alice | PM |
<b>John Smith</b>                     **John Smith**

This costs nothing at export, because the exporter still cuts byte ranges from the original markup. Three consequences: whitespace in markup collapses (a single space stays literal, so John Smith stays one range); link targets are reviewable ([text](url), so an address reachable only through mailto: is shown, not silently swept); and layout tables stay layout (pipes only for real multi-cell rows).

The message folds into sections: headers, each body, attachments, and nested messages at increasing depth. Section labels are metadata now, not text written into the reviewed stream (where nobody could redact them and the model read them as content). See Review Workspace for how folding works, including why a section with undecided suggestions always opens.

Streamed. A forward scanner reads the message a line at a time and never holds attachment bodies, and it matches the old recursive parser byte for byte across 3,000 generated messages (Streaming IO §6).

Attachments are documents. An attachment in a supported format becomes a child document of its own, is reviewed and redacted, and is substituted back into the message, verified by checksum. A message with attachments therefore arrives as a batch. See Batches and Mailboxes §3.

Resource limits are its own, independent of the upload ceiling, and fail closed: MIME depth (20), parts (1000), decoded text (16MB), header block (1MB), attachments (200), nested messages (5). Raising the size a file may be must never be the same decision as allowing unlimited complexity. Going over is too-complex, never a truncated message presented as complete.

Verified by two parsers. A redaction system checking its own work with its own parser is grading its own homework. postal-mime is the independent reader: if it cannot parse the export, the export failed.

---

MBOX: a mailbox is a batch

A mailbox is hundreds of documents. It expands into a batch, one document per message, each exactly like the same .eml uploaded on its own, and decisions carry across all of them: "this recurring name is a colleague" answered once, applied to nine hundred messages. The mailbox itself ends at status expanded, is charged nothing, and is never exported as a file. Instead, the batch download rebuilds it (since 1.10.0): its messages' verified exports in order, separator lines written fresh with no envelope sender, verified again by re-splitting. Splitting rules, limits, verification by conservation, and the rebuild: Batches and Mailboxes §4 and §6.

---

CSV / TSV: a grid, parsed rather than split

Parsed into a grid and rewritten cell by cell, so a value next to a comma inside a quoted field cannot shift every row after it. Reviewed in the same grid as a workbook, and read streamed.

Verified by: re-parsing the export as a grid and reading every cell. A value hiding in a field the parser cannot reach would not be found by searching the file as one blob, and an export that no longer parses is a failure in its own right. A file holding control characters is refused as not text.

---

TXT: offsets, and nothing invented

Addressed by offset into the source; paged for review; read streamed. Verified by: the exact resulting text.

---

RTF: visible text is not contiguous

Parsed rather than searched, because a word processor splits a value across formatting groups and escapes, and the string is often not in the file at all. RTF shares its atom map with the HTML inside an email: "the visible text is not contiguous in the source" is the same problem twice.

Verified by: re-parsing the export, which also proves it still opens. A file that does not open throws there, rather than passing for want of a match.

---

Images: pixels, or it did not happen

Pixels are replaced and the file is re-encoded. EXIF and GPS go too. OCR reads any text in the image (words become hit targets, and text suggestions are placed on the pixels they cover). A vision pass proposes regions: faces as the whole head with generous bounds, on a 0–1000 grid (which is what makes local models like Qwen3-VL work, since 1.6.2).

Style	Guarantee
Solid fill (default)	The source pixels are gone.
Blur	Source pixels destroyed in the export, but heavily blurred content can in principle be attacked.
Pixelate	Same caveat as blur.

Text is always filled solid regardless of style, and the report records which style was used.

Verified by: sampling the redacted region's mean colour; checking untouched areas are unchanged and the output differs from the source; checking EXIF is absent and the copyright string is not in the bytes.

OCR engines. Tesseract (local, per-word boxes) or Mistral (hosted, per-block boxes). A redaction over a Mistral block covers the whole block, because a proportional slice of a wrapped paragraph would be a rectangle in the wrong place. See Configuration §OCR.

---

Not supported yet

Format	Status
PST (Outlook archives)	Planned: #90. The same container→batch shape as MBOX, behind an opaque binary format with no native Node parser.

tests/adversarial.test.ts includes cases for unsupported files, so they are refused rather than half-read. HTML email bodies were on this list until 1.1.1.

---

See also

• Redaction Engine · Export and Anonymisation · Batches and Mailboxes · Streaming IO · Security Model