Troubleshooting

Troubleshooting

Failure-code reference: Failure Codes. Configuration reference: Configuration. Upgrade notes: Upgrading.

---

Setup and install

pnpm install fails on the Node or pnpm version

Node 22+ and pnpm 11+ are required and enforced (engines + engine-strict), so an unsupported runtime fails here with a clear message rather than somewhere confusing later.

The app refuses to start

• ENCRYPTION_KEY is malformed. assertMasterKey() checks at boot that it decodes to 32 bytes (64 hex chars or base64). Watch for YAML turning an all-digits key into a number.
• The streaming budget cannot fit. ANONIFY_STREAM_MEMORY_BUDGET must give every concurrently processing document at least two chunks. Raise it, lower ANONIFY_BATCH_PROCESSING, or lower ANONIFY_ENCRYPTION_CHUNK_SIZE.
• A limit override is malformed. Batch-concurrency and size overrides throw rather than being ignored. Fix the value the error names.

A port is already in use

Every Compose port binds to 127.0.0.1. Set APP_PORT, POSTGRES_PORT, RUSTFS_PORT or RUSTFS_CONSOLE_PORT in .env. (The old MINIO_* names were renamed in 1.3.0.)

pnpm setup then pnpm dev says "port in use"

You probably started the whole Compose stack and tried to run the app on the host. The app container already holds port 3000. Either use docker compose up -d alone, or start only postgres rustfs rustfs-init and run pnpm dev. Setup's finishing step keeps these apart for you.

After upgrading, object storage looks empty or RustFS fails to start

The 1.3.0 move from MinIO to RustFS reuses the anonify_minio-data volume and imports it in place, one way. Check docker compose logs rustfs rustfs-init, and that rustfs-volume-init ran (it hands the volume to UID 10001). If you used MinIO SSE, tiering or custom bucket features, the in-place import cannot read those objects: restore your backup and follow the API-copy path in Upgrading.

---

Uploads and processing

Uploads are accepted but documents sit at queued

Two different causes:

1. No workflow worker (self-hosted). Durable runs need @workflow/world-postgres and the worker started by instrumentation.ts. Check that WORKFLOW_TARGET_WORLD is set for self-hosted (and unset on Vercel), and look for the startup line "message":"workflow worker started". If it never appears, nothing is polling.
2. Waiting for admission. Each owner may process ANONIFY_BATCH_PROCESSING documents at once (3 demo, 6 self-hosted), and the rest queue by design and start as slots free. If a run was killed mid-flight (a crash or a deploy), its slot may never be released by an event. The /api/cron/cleanup endpoint re-admits such documents (admitStalled), but pnpm cleanup does not. Use the endpoint (the Compose scheduler, or a curl with CRON_SECRET), or press Retry.

The upload panel says "larger than the 25 MB demo limit"

The real ceiling is MAX_UPLOAD_BYTES = 50 MiB, and the check uses that number. The "25 MB" in the message text is stale wording, so files up to 50 MiB are accepted.

Vercel: "Content type mismatch" on upload

Fixed long ago (#51): the upload token no longer constrains the browser's guessed MIME type, and ingest sniffs the real bytes. If you see it, your deployment predates that fix.

Processing failures

The message always comes from a fixed table, and the code tells you what to do. Full table: Failure Codes. The common ones:

Code	What to do
extension-mismatch	Magic bytes decide, not the filename. Check the file is what its name says.
corrupt-source	Upload again. If it recurs, your storage is returning different bytes than it was given.
too-complex	Over an EML parser, attachment-expansion or mailbox limit. Not retryable: the same bytes hit the same limit. An administrator can raise the limits.
quota	Resets at midnight UTC. Self-hosted installs have no quotas. If you see this on your own machine, ANONIFY_PROFILE is probably demo.
configuration	Missing configuration. Commonly OCR_PROVIDER=mistral with no MISTRAL_API_KEY (a key under another name such as MISTRAL_AI_KEY is named in the log). Fix it and restart.
rule-too-broad	A batch or global rule matched too much of this document. Narrow or switch it off in the Rules panel, then Retry. The document still opens for review meanwhile.
storage / database / timeout / unknown	Transient: Retry is offered.

Small .xlsx files sometimes fail to process

Known bug. loadWorkbook in lib/documents/xlsx/extract.ts copies the bytes with Buffer.from(...) and then slices .buffer at the original view's offset. For a workbook under 4 KB, Node places the copy in its shared buffer pool, so the slice can land on the wrong bytes, and the file is intermittently read as a corrupt zip. Which files fail varies from run to run. Workaround: retry, or re-save the workbook so it is larger. (Found by the benchmark corpus; see Testing and Benchmarks.)

The progress bar stopped moving

The SSE stream reconnects with startIndex = last + 1 and resumes; the run itself continues regardless. A closed tab does not cancel processing. If a batch-export stream cannot be held open, the client falls back to reading the row every 5 s.

---

Detection

No suggestions, or far fewer than expected

Check, in this order:

1. The usage panel. A model pass that fell short writes document.ai.degraded with the reason (no model configured, unsupported model, rate limit, empty balance, bad key, spend cap), and the panel says so. Pattern-only results are never silently presented as a model's.
2. Is a model configured and verified? pnpm ai status. Without one, names, prose addresses and roles are not found (the patterns do not look for them). A non-Gateway provider needs AI_MODEL_CAPABILITIES, written by pnpm setup / pnpm ai verify. Changing the model or AI_BASE_URL invalidates it.
3. Which preset? The editor header says what was looked for. A narrow preset means a short list.
4. The daily spend cap (ANONIFY_AI_DAILY_SPEND_USD). At 100%, the contextual pass is skipped until the next UTC day.
5. Non-English documents. Detectors and prompts are English-shaped, and the failure is silent (#43).

Images get no suggestions with a local model

Use a vision model (e.g. qwen3-vl) verified for images. Before 1.6.2, local vision models ignored the 0–1 coordinate request and failed validation; regions are now asked for on a 0–1000 grid. A model verified text-only skips image analysis by design, and says so.

pnpm ai verify / setup says verification failed

It names the failed check and the reason. The common ones:

Says	Usually means
credentials rejected (401/403)	wrong or missing key; for Azure discovery, a management credential is needed as well as the resource key
not found (404)	a model this account cannot use, or AI_BASE_URL missing /v1
refused (400)	the model or endpoint has no JSON-schema output or image input; try --text-only, or another model
out of credit (402) / rate-limited (429)	billing or quota at the provider
unreachable	nothing listening (Ollama not running), DNS, TLS, or a two-minute timeout
answered badly	the model returned non-JSON or off-schema output; pick a stronger model

For Ollama in Docker on Linux, Ollama must listen on an interface the container can reach (OLLAMA_HOST). For a ChatGPT sign-in, run pnpm ai login where DATABASE_URL reaches the instance's database; if ENCRYPTION_KEY changed, pnpm ai status reports that the token cannot be opened.

The spend cap does not seem to apply

It needs prices: AI_MODEL_PRICES for every model used (or the legacy pair for the selected one). Without them the cap cannot be computed, and the log says so. A ChatGPT sign-in costs $0 unless you record a price.

---

Review

Clicking a word redacts the whole line

That was the behaviour before 1.9.0. Since then a click takes the token under the pointer. In browsers without the CSS Custom Highlight API, highlights still fall back to whole-line marking (the redaction itself is exact).

Search says "no matches" but I can see the text

Search runs on the server over the normalized text. Check match case, whole word and RegEx. In RegEx mode ^/$ match at line breaks, and backreferences and lookaround are refused (RE2). Scanned pages are searchable only through their OCR text. A whole-document search over its budget returns 422, so make the pattern more specific.

A rule won't save: "over budget"

A rule may spend 5 s and make 10,000 matches per document. Over that it fails whole and writes nothing, so narrow it (anchor it, add whole-word, drop \w+-style breadth). A batch preview names the document that would fail.

My global rules disappeared

Global rules belong to the browser session. A new browser, a private window or cleared cookies is a new session. They also expire 30 days after last use. Export them (Rules → Global → Export) to keep them, and import on the other side.

Hush is unavailable, or stopped mid-answer

GET /api/assistant reports the reason: not-configured (no model), unsupported (the model is not verified for structured output), or budget (the spend cap). A run also stops when your daily assistantTokens allowance is spent (429 allowance), at 12 steps, or at 80 messages. The panel says which. Search, rules and learned offers keep working.

Undo overwrote other decisions

Fixed in 1.5.0: undo now saves only the redactions it changed. Before that, Ctrl+Z in a tab with stale decisions could overwrite the whole document's decisions.

---

Export and download

"The export did not pass verification and was not saved"

This is the gate working. An accepted value was still recoverable from the artifact, so nothing was written and no link was issued. Your source is untouched. If you can reproduce it on a synthetic file, that is a valuable report, and it goes to private reporting first (Security Model §13).

Exports with a second copy fail with "could not be generated"

Fixed in 1.5.0. Upgrade.

My old PDFs' boxes cover whole lines

PDFs analysed before 1.5.0 fall back to whole-run boxes (the fix for GHSA-h3hv-rpqw-7p84). Re-upload them for tight boxes.

The download link stopped working

Tokens live five minutes. Reopen the export dialog or the batch button for a fresh link. A download that breaks off partway means the bytes failed their checksum as they streamed. That is deliberate: you never get a complete file that failed its check.

The vault is gone

For a single export the vault is shown once and stored nowhere, by design, so it cannot be recovered. A batch export's vault is inside the archive. See Export and Anonymisation §5.

Restore returns 422

PDFs and images cannot be restored (their surrogates are pixels). Pseudonymised values never come back (no mapping exists). Also check X-Vault-Matches: the wrong vault produces plausible nonsense, not an error.

A batch download is a single file, not a zip

Since 1.10.0 the default output=original returns one upload as itself (e.g. inbox-redacted.mbox). Choose Processed files or Both for a zip. A download with a vault is always a zip.

A batch export is missing documents

Each skipped document is named with a reason ("had not finished processing", "failed verification and was withheld", "hit the export allowance", "did not fit in the archive", "could not be exported", "not reached"). Messages left out of a rebuilt mailbox are listed in batch-report.json. Before 1.10.0, batches over 50 documents were cut to the first 50, so upgrade if you see that.

A download with a non-Latin name fails

Fixed in 1.10.0.

---

Retention and cleanup

Documents are never deleted

The most common self-hosting mistake: nothing outside Vercel reads vercel.json. Run pnpm cleanup from cron, or docker compose --profile scheduler up -d. On Vercel Hobby, the daily cron is backed up by the sweep.yml Action (set SWEEP_URL and CRON_SECRET as repo secrets). See Deployment §3.

The scheduler returns 401 every cycle

CRON_SECRET is not set, and production refuses unauthenticated sweeps. It says so at startup and on every refusal.

The sweep failed with "No record was found for a delete"

Fixed in 1.8.0: a document removed while the sweep ran now counts as deleted, and one un-purgeable document no longer stops the run.

Extending retention does not push the expiry out as far as expected

By design: the new expiry is computed from creation and capped at 72 h, so renewing converges on the ceiling.

---

Deployment

The Vercel deploy fails outright

Two Hobby limits fail the deploy: a maxDuration over 300 s, and a cron more frequent than daily. The build can also fail on the 250 MB step-function bundle (it is ~228 MB with OCR); a new dependency in that bundle is the usual cause. Set OCR_PROVIDER=mistral at build time.

A large scanned PDF fails late on Vercel

2 GB, one core and 300 s per step. The run fails cleanly. Lower MAX_UPLOAD_BYTES to refuse such files up front.

Link previews point at localhost:3000

Set ANONIFY_PUBLIC_URL and rebuild (it is resolved at build time).

---

Development

pnpm typecheck says "Cannot find name 'RouteContext'"

Use pnpm typecheck (it runs next typegen first), not bare tsc.

The OCR tests are skipped

They need the pinned Tesseract model cached. Run pnpm ocr:warm once.

pnpm test:db deleted rows I cared about

TEST_DATABASE_URL must be a throwaway database, never DATABASE_URL.

A limit I set is being ignored

pnpm rate-limit show shows each limit and where it came from: profile → ANONIFY_RATE_LIMIT_<NAME> → CLI, later wins.

Everything stopped working after I regenerated .env

If ENCRYPTION_KEY changed, every stored document (and any ChatGPT sign-in) is permanently unreadable. There is no recovery. pnpm setup offers to reuse the existing secrets for exactly this reason.

---

Still stuck

• Search the issues.
• Open a bug report with the failure code, the format, the version, and a synthetic file that reproduces it.
• A value that survived an export goes to private reporting instead.