PDF Titles

PDF titles

A browser tab shows the title stored inside a PDF. Most stored PDFs have no useful title, so the tab shows the file name. The PDF title job writes the document's public display name into the stored PDF so the tab shows it. This page is for operators: what the job does, when it skips a document, how to run it on test, and how to undo it for one document.

The archive backup the job depends on is in Document-Backup. Putting a document back is also in Document-Backup-Restore.

What the job changes

The title goes in as an appended incremental update: new bytes added after the end of the file. The update revises Info with only /Title changed (Chromium reads this). When the file has an XMP packet, it also writes the XMP stream with dc:title (Firefox prefers this) at a new object number, at or above the original's next free number. The Catalog is revised so only /Metadata changes, and it points at the new stream. The original /Metadata object is never rewritten, because it may also serve as another object. The API refuses any other shape of update. The original bytes stay an exact prefix of the stored object. Page content is never changed. After titling, every route that serves the file (view, download, bulk zip, sealed link) serves the titled bytes.

The API records the result on the document row in the field pdfTitle: the original length and sha256, the title, the titled length and sha256, a status and a reason. The status is titled, skipped, restored or needs-review. Source: src/controllers/nosql/pdf-title.js, pdf-title/titler.py.

What is skipped

A skipped document is left as it is. The reason is stored on its pdfTitle record, except where the table says otherwise.

Reason	Meaning
not-from-eagle	The row id is not a 24-character lowercase hex Mongo ObjectId. Only rows that came from Eagle are titled. Not recorded.
sealed	The document is sealed, by DEMI or by Eagle.
not-public	The document is not public level.
no-public-name	The public display name is withheld, so there is no title.
not-pdf	The type and extension are not PDF.
no-file	The row has no stored file.
empty	The stored object has 0 bytes.
over-cap	The stored object is over 256 MiB.
shared-key	Another row stores the same s3Key.
encrypted	The PDF is encrypted. The reader reports original-encrypted.
signed	The PDF has /ByteRange or an AcroForm with /SigFlags.
unreadable	pypdf cannot read the file.
check-qpdf-source-errors	qpdf reports errors in the original. Warnings alone do not skip.
no-report	Ten leases ended with no report from the worker. See the end of Write sequence.

The titler also skips with no-title, xmp-direct, xmp-unreadable, xmp-unrecognised, no-startxref, bad-startxref and info-unsafe. info-unsafe means Info is a stream, is the Catalog, is written directly in the trailer, or holds a value that is not a string (other than /Trapped). If one of its own output checks fails, the reason is check-prefix, check-reopen, check-pages, check-title, check-qpdf-open, check-qpdf, check-qpdf-pages or check-qpdf-title. The worker skips with prefix-mismatch and increment-too-large. The API refuses an increment that is not the titler's own with reasons that start original- (it could not read the original's structure) or tail- (the added bytes are not the expected update).

The work list (GET /api/documents/pdf-title/pending) leaves out documents that are not from Eagle, sealed, not public, not PDF, over the cap by fileSize, sharing an s3Key, leased, or under review.

Write sequence

The worker holds no storage credential. It asks the API for permission one document at a time.

1. Lease. POST /api/documents/:id/pdf-title/lease checks eligibility and the size cap. It then runs the archive backup gate. The lease lasts 10 minutes. Only one lease per document is possible; a second worker gets 412 lease-race or 409 lease-held.
2. Per-document backup. The API copies the stored object to pdf-title-backup/<key> and returns a 300-second download link for the copy. A backup that already exists is replaced only if it equals the live object; otherwise the row goes to needs-review with orphan-backup-differs.
3. The worker reads the backup, builds the original plus one update, and checks it with pypdf and qpdf (prefix unchanged, same page count, title reads back).
4. Commit. POST /api/documents/:id/pdf-title/commit sends the new length and hashes. The API re-checks that the stored object is unchanged, hashes the backup itself, and on first titling records the original length and sha256 once. It returns an upload link for exactly the leased key, valid 120 seconds, with the headers to send (Content-Type, Content-MD5).
5. The worker sends one PUT with Content-Type and Content-MD5. The MD5 is signed into the link, so the store refuses any other bytes. The PUT carries no If-Match, because the NRS store answers 412 to it on objects stored years ago, even when the ETag matches.
6. Report. PUT /api/documents/:id/pdf-title ends the lease. Until the upload link has expired, the API answers 409 put-window-open. After that the API reads the store and checks the written object: size, ETag, content type, whole-file sha256, sha256 of the original prefix, and the appended tail. Only an object that passes every check is recorded titled. The unchanged original releases the lease. Anything else is copied back from the backup and set to needs-review.
7. The per-document backup is deleted, by version, only when the outcome is final, so after the PUT window closes. A row in needs-review keeps its backup.

POST /api/documents/pdf-title/sweep settles expired leases the same way. The worker calls it before a live run.

A lease can end with no word from the worker, for example when the worker process is killed for memory on a large PDF, or when a large PDF runs out of time. The API counts these releases in pdfTitle.releases. A report with a skip or a failure reason does not count, and a title clears the count. On the tenth such release the row is recorded as a skip with reason no-report (needs-review for a restore), so the file is not offered again. The cap is 10 because a large PDF can run out of time on many leases before it is titled: in prod, files of 98 to 238 MB ended 10 to 54 leases this way. A file that crashes the worker every time still stops after 10 runs.

The sweep response counts these skips in outcomes.noReport, and the API's [pdf-title] sweep log line carries the same count. They are also counted in outcomes.skipped.

Archive backup gate

Before any lease, assertBackedUp (src/helpers/backup-check.js) must find the document's original in the Azure archive account (Document-Backup). The lease is refused with 409 and one of these reasons:

Reason	Meaning
backup-unconfigured	BACKUP_ACCOUNT or BACKUP_CONTAINER is empty.
no-backup	No blob for the key. A titled document with no recorded original ETag also gives this.
size-mismatch	The blob size differs from the object.
md5-mismatch	The blob checksum differs from the object ETag. The row is also set to needs-review.
not-archive	The blob is not in the Archive tier.

The gate checks the live object on first titling, and the recorded original on later titlings.

The API reads the archive account and container from the app settings BACKUP_ACCOUNT and BACKUP_CONTAINER (src/config.js, backupAccount and backupContainer). Bicep sets them from the params backupAccountName (default empty) and backupContainerName (default originals) in azure/main.bicep, passed to azure/modules/api-function-flex.bicep. On test, azure/main.test.bicepparam sets backupAccountName to eaglebaktestvymaysch2agd. With the account unset, every lease is refused with backup-unconfigured.

Run the worker on test

The worker is pdf-title/run.py. On test and prod it also runs on a schedule as the app demi-pdf-title-<env> (see "Where it runs"). Run it by hand for a dry run or a single document set.

1. Install the pinned libraries: pip install -r pdf-title/requirements.txt. qpdf comes through pikepdf.

2. Check the API has DEMI_PDF_TITLE_WORKER_PRINCIPALS set to a comma-separated list of key registry ids, and that the id of the worker's key is in it. The list has no default. A principal on DEMI_EAGLE_MIRROR_PRINCIPALS must not be on it. The key also needs the write role.

3. Set the API settings in the shell. Never write the key into a file in the repo.

   • DEMI_API_URL: the base that /documents hangs off, for example https://<host>/api.
   • DEMI_API_KEY: the worker's key. It is sent to the API only.

4. Dry run. This lists the work and writes nothing:

   python3 pdf-title/run.py
   

5. Live run:

   python3 pdf-title/run.py --live --max-rows 20
   

Flags:

Flag	Default	Meaning
--live	off	Write. Without it the run only lists rows.
--max-rows N	20	Rows taken from the work list.
--concurrency N	4	Documents in flight.
--max-minutes M	30	No new row starts after this.
--timeout S	60	Seconds per request.

Exit code 0 means no failures. 1 means a row failed or needs review. 2 means the API settings are missing. The work list is the resume point, so run again after a stop. The final log line counts outcomes. Roll out with 20 documents first, then the full backfill.

Restore one document

Restore cuts the stored object back to its recorded original length. It checks first that those bytes hash to the recorded original sha256, and refuses if they do not. It works only for a document with status titled. It uses the same lease flow, so the backup gate applies.

1. Set DEMI_API_URL and DEMI_API_KEY as above.

2. Dry run:

   python3 pdf-title/restore.py --id <docId> --project <projectId>
   

3. Restore:

   python3 pdf-title/restore.py --id <docId> --project <projectId> --live
   

   Exit code 0 means the document was restored. The row status becomes restored.

To check a file offline, take the recorded originalLength and originalSha256 from the row:

python3 pdf-title/restore.py --file titled.pdf --length <originalLength> --sha256 <originalSha256> --out original.pdf

--out must not exist. A wrong hash, or a file shorter than the length, exits 1 and writes nothing.

Restore drill on test: on 2026-10-07 (UTC), restore.py --id <id> --live restored document 58868f2be036fb0105767ea5. It ran on demi-devbox-test through az vm run-command. DEMI_API_URL was the test API. DEMI_API_KEY was read on the devbox from demi-kv-test secret pdf-title-worker-api-key, so no key left the vault network. The --id path imports run.py and titler.py, so they must sit beside it, with pypdf and pikepdf from pdf-title/requirements.txt. The devbox has no python3-venv; python3-pip was installed on 2026-10-07 and the libraries went into a temp dir with pip install --target. Result: result=restored after about 2.5 minutes. The served file went from 88612 bytes with one /Title to 84031 bytes with none, equal to the record's fileSize, a byte-exact prefix of the titled copy. The worker re-titles it on a later tick.

If a document is in needs-review, its backup is still in pdf-title-backup/<key> and the row holds the reason. A person decides what to do; the job does not retry it.

Test rollout ticks

Measured 2026-10-07, UTC. Dry-run ticks were clean. After BACKUP_ACCOUNT was set at 20:39, the live ticks were:

Tick	Titled	Skipped
20:40	19	1
20:50	20	0
21:00	18	2

There were no refusals and no API exceptions. Every lease and commit request returned 200 or 201. A tick takes about 3 minutes. The API traces a skip as "lease ended with no write"; the cause is not logged.

To read a tick, query Log Analytics demi-logs-test, table AppTraces:

• AppRoleName demi-pdf-title-test: the work list, sweep and done: lines.
• demi-api-fc-test: messages that start [pdf-title] (lease issued, write signed, titled, backup deleted).

Pass-rate check

scripts/pdf-title-check/pass_rate.py runs the titler and the API's own checks over a folder of real PDFs, with an ASCII and a non-ASCII title, and writes a summary and a CSV. Run it before each rollout. Sample PDFs stay outside the repo.

python3 scripts/pdf-title-check/pass_rate.py <folder of PDFs> --csv results.csv

Exit code 0 means every check passed, 1 a check failed, 2 the run broke. The acceptance rules and a reference run are in scripts/pdf-title-check/README.md.

Worker key on test

The worker's registry key is minted on the devbox demi-devbox-test. The break-glass admin-api-key is read from demi-kv-test on that box, so no key leaves the vault network.

1. POST /admin/api-keys with the body {"name":"pdf-title-worker test","roles":["demi-service-write"],"allowWrite":true}. The response has id and key.
2. Store the key in demi-kv-test as the secret pdf-title-worker-api-key. The Key Vault policy Enforce-GR-KeyVault rejects az keyvault secret set without --content-type and --expires. Use 89 days, because the key itself expires in 90 days.
3. Add the key id to DEMI_PDF_TITLE_WORKER_PRINCIPALS on the API. The worker app reads the secret as a Key Vault reference in DEMI_API_KEY.
4. To revoke, call DELETE /admin/api-keys/<id>.

Where it runs

Environment	State
Test	demi-pdf-title-test runs run.py on a timer every 10 minutes with PDF_TITLE_MAX_ROWS 20 and PDF_TITLE_LIVE on. It is defined in bicep (Operating-DEMI-in-Azure). The full backfill proceeds at 20 documents per tick.
Prod	demi-pdf-title-prod runs run.py live (PDF_TITLE_LIVE true) on a timer every 10 minutes (0 */10 * * * *). It sets PDF_TITLE_MAX_ROWS 20, PDF_TITLE_MAX_MINUTES 5, PDF_TITLE_CONCURRENCY 4 and DEMI_API_URL https://demi-api-fc-prod.azurewebsites.net/api. The prod corpus is about 61,000 documents, so the backfill at 20 documents per tick takes about three weeks. The prod archive backup is complete (Document-Backup) and the API's backup gate is on, so a title write is refused for a document with no backup copy. Restore with Document-Backup-Restore.