Home

DEMI — Document Extraction & Machine Intelligence

Documentation for BC EAO's Digital File Library (DFL) and AI Classifier Tool (AICT), built in the eagle-demi repository.

DEMI is EPIC's central data store, not a sandbox. demi-api is the master directory for projects, documents, chunks and administrative boundaries. It holds a merged model across epic.track (authoritative for projects) and eagle (richer EA-process data), and runs on Azure Functions (Flex Consumption), Cosmos DB for NoSQL and Azure AI Search, all private-endpoint only.

Track project ids are the master project ids — see Identity-Model. eagle-api pushes every record it writes into DEMI, and ENGAGE pushes its engagements into DEMI. Nothing in DEMI writes Eagle's MongoDB; DEMI's sync-out writes the ENGAGE comment periods into Eagle through eagle-api's HTTP API — see Sync-Architecture and ADR-014-Sync-Out.

Environments

Test (c4b0a8-test-rg) holds the full estate, including the React demo frontend. Prod (rg-demi-prod, subscription c4b0a8-prod) holds the backend: demi-api-fc-prod (Flex Consumption, behind API gateway demi-apim-prod), demi-cosmos-prod and demi-search-prod. There is no Azure dev estate.

eagle-public's search, test and prod, is served by demi-api through SEARCH_API_PATH: /demi-search. eagle-search is retired; /eagle-search now renders a sentinel page. Which surface reads which route is in Sources-and-Status; the estate itself is in Azure-Environments.

Why

Three developer briefs from April 2026 define the scope. They are the requirements source and are kept frozen under briefs/.

Digital File Library. Staff cannot efficiently find, file or retrieve EA documents. 65,000+ documents in EPIC have limited metadata, no full-text search and no structured filing system. The outcome wanted: documents findable in seconds, classified by type, topic and ORCS code, with visibility controls that enforce who sees what, and a digital path forward for a 27,000-record paper backlog.

AI Classifier Tool. Manual metadata entry for every upload is slow and inconsistent — staff skip fields, use wrong categories, or leave documents untagged. The outcome wanted: automatic metadata suggestions that staff review and confirm rather than replace, consistent across the whole corpus.

EPIC integration. DFL and AICT have to work with the existing EPIC platform. The briefs assumed they would live inside eagle-api on a single MongoDB. What was built is a standalone service with its own database, which other EPIC frontends consume.

Start here

Page	What it covers
Environment-Reality-and-Operational-Gotchas	What exists in Azure, app settings, and the traps. Read this before trusting anything else
Operating-DEMI-in-Azure	Deploy the API and frontend, wire Front Door and CORS, verify, what CI deploys
Operating-DEMI-Data-Plane	Reach Cosmos and Search from the devbox, probe a change, read audit and cost
Edge-Ban-Detector	The hourly edgeBan timer: judge, escalation, modes, kill switch, policy, soak and promote
CI-Workflows	Every GitHub Actions workflow: trigger, path filters, identity and Azure roles
CI-Deploy-Workflows	eagle-demi staging and production deploy workflows: steps, identity, rollback
CI-Admin-Frontend-Workflows	eagle-demi-admin PR checks and test and production deploys
Release-Process	How a version is cut, tagged, deployed to production and rolled back
Frontend	React shell: screens, routes, backend calls, sign-in gate, styling
Architecture	Containers, the merge model, authorization, deletion semantics, storage
Seeding	Filling the Cosmos containers from Track and Eagle, and the gates that stop a bad seed
Sources-and-Status	Every data source and served surface, each in one state
Search-and-Retrieval	Retrieval overview; links to Search-Index-Reference, Search-Query-Construction, Search-Measurements
Extraction-Pipeline	How a PDF becomes searchable chunks
Extraction-Quality	Corpus coverage, word-joining, tiling, and what the quality metrics miss
Identity-Model	Which id means what, across Track, Eagle and DEMI
Sync-Architecture	How records get into DEMI, and what happens on delete
Eagle-Mirror-Reference	Mirrored kinds, refusal codes, parked retry, drift classes, stored report, migration re-push rule
ENGAGE-Rows-Reference	ENGAGE-owned comment periods: field ownership, join rules, tombstones, reconcile-engage report, sync-out settings
Public-Read-Backfill	Filling and repairing the comment period, comment, notification and list containers
Document-Intake	Upload link, Defender malware scan, release to the object store, scan cap
Connecting-an-Application-to-DEMI	How another application gets a credential
Recent-Uploads	GET /api/documents/recent-uploads — the eagle-public homepage panel
Reports	The comment export and BCGW CSV reads
User-Data	Saved map areas and preferences, /me routes, the userdata container
Azure-Environments	Subscriptions, estates, Bicep, and what it costs
Document-Backup	The Azure archive backup of document originals: account, access, run, check
Document-Backup-Restore	Reverting documents to the archive backup, and the restore drill
PDF-Titles	The PDF title job: what it writes, what it skips, how to run it on test and restore one document
Secret-Sync	Key Vault, the secret-sync Function app, required secret names
BC-Gov-Azure-Landing-Zone	The platform rules that block deploys
Business-Rules	BR-01…BR-09 — hold states, sensitivity ladder, ORCS mapping

Decisions

Each ADR page owns its own status detail.

ADR	Status
ADR-001-Typesense-for-DFL-Search	Superseded by Azure AI Search
ADR-002-Async-Processing-MongoDB-Queue	Superseded — change streams do not exist on the Cosmos Mongo API
ADR-003-Rule-Based-Classification	Accepted
ADR-004-Read-ACL-Authorization-Model	Accepted
ADR-005-Cosmos-Full-Text-Search-Ruled-Out	Rejected — fuzzy is a silent no-op
ADR-006-AI-Summarization-over-BM25	Accepted
ADR-007-Service-to-Service-Credentials	Accepted
ADR-008-Audit-and-Usage-Analytics-Store	Accepted
ADR-009-Track-Feed	Proposed
ADR-010-Attribute-Level-Access-Control	Accepted
ADR-011-Public-Read-Surface	Accepted
ADR-012-Public-Config-Surface	Accepted
ADR-013-Stored-Project-Summaries	Proposed
ADR-014-Sync-Out	Accepted
ADR-015-Malware-Scanning-on-Intake	Accepted

History

Cosmos-Migration-Record — the move off the MongoDB API, the Typesense removal, and the Phase 8 teardown.

Cost

Azure-Environments owns the spend figures and the per-service breakdown.