Skip to content

Document Content Search

Daniel Truong edited this page Sep 15, 2026 · 6 revisions

Document Content Search

Full-text search of document contents, the text inside PDFs, not just titles and metadata. The DocumentChunk dataset: 1,128,576 chunks in the prod index, measured 2026-08-26.

Backends by environment are on Eagle Search: prod serves DocumentChunk from the eagle-chunks index on demi-search-prod; test from demi-api over Cosmos.

The UI is the search/content route in eagle-public on develop; prod v2.7.28 does not ship it, so nothing in prod queries the dataset yet. On the react line the same search is a scope of the documents tab on the unified search page, /search?record=documents&scope=inside, still behind the CONTENT_SEARCH flag; see Eagle Search.

Where the text comes from

Eagle DEMI extracts the PDFs and owns the chunk corpus. The eagle-chunks index holds a copy of that corpus; no extractor writes into it. Eagle's own object store does not hold most of the corpus in dev, so the backlog cannot be re-extracted there.

What a search returns

A chunk is a passage, so one long report can fill a page of results on its own. Results are grouped by document in application code, Azure AI Search having no result collapsing, with each document's best chunk fixing its rank.

Two things the UI does not claim:

  • The result count is passages, not documents. Azure cannot return a distinct-document count, so the number is labelled rather than invented.
  • There are no "jump to page N" links. pageNumber is a passage sequence number the chunker increments per block; it has no relation to the PDF's pages.

Access control

A chunk is visible only if its document and the document's project are both public. eagle-search stamps chunk read[] from the parent document at write time and re-stamps on unpublish; DEMI derives it from the parent document at query time. Details on Eagle Search.

OCR

Scanned documents with no text layer are parked, not retried, and stay unsearchable. Track them by the count of extractionMethod: 'none'; the fraction is unmeasured.

See also

Clone this wiki locally