Home

DEMI — Document Extraction & Machine Intelligence

Documentation for BC EAO's Digital File Library (DFL) and AI Classifier Tool (AICT), built in the eagle-demi repository.

DEMI is EPIC's central data store, not a sandbox. demi-api is the master directory for projects, documents, chunks and administrative boundaries. It holds a merged model across epic.track (authoritative for projects) and eagle (richer EA-process data), and runs on Azure Functions (Flex Consumption), Cosmos DB for NoSQL and Azure AI Search, all private-endpoint only.

Track project ids are the master project ids — see Identity-Model. eagle-api pushes every record it writes into DEMI, and ENGAGE pushes its engagements into DEMI. Nothing in DEMI writes Eagle's MongoDB; DEMI's sync-out writes the ENGAGE comment periods into Eagle through eagle-api's HTTP API — see Sync-Architecture and ADR-014-Sync-Out.

Environments

Test (c4b0a8-test-rg) holds the full estate, including the React demo frontend. Prod (rg-demi-prod, subscription c4b0a8-prod) holds the backend: demi-api-fc-prod (Flex Consumption, behind API gateway demi-apim-prod), demi-cosmos-prod and demi-search-prod. There is no Azure dev estate.

eagle-public's search, test and prod, is served by demi-api through SEARCH_API_PATH: /demi-search. eagle-search is retired; /eagle-search renders a sentinel page. Which surface reads which route is in Sources-and-Status; the estate itself is in Azure-Environments.

Why

Three developer briefs are the requirements source and stay frozen: Developer-Brief-Digital-File-Library, Developer-Brief-AI-Classifier-Tool and Developer-Brief-EPIC-System-Integration.

Start here

Page	What it covers
Environment-Reality-and-Operational-Gotchas	What exists in Azure, app settings, and the traps. Read this before trusting anything else
Operating-DEMI-in-Azure	Deploy the API and frontend, wire Front Door and CORS, verify, what CI deploys
Operating-DEMI-Data-Plane	Reach Cosmos and Search from the devbox, probe a change, read audit and cost
Edge-Ban-Detector	The hourly edgeBan timer: judge, escalation, modes, kill switch, policy, soak and promote
CI-Workflows	Every GitHub Actions workflow: trigger, path filters, identity and Azure roles
CI-Deploy-Workflows	eagle-demi staging and production deploy workflows: steps, identity, rollback
CI-Admin-Frontend-Workflows	eagle-demi-admin PR checks and test and production deploys
Release-Process	How a version is cut, tagged, deployed to production and rolled back
Frontend	React shell: screens, routes, backend calls, sign-in gate, styling
Architecture	Containers, the merge model, authorization, deletion semantics, storage
Seeding	Filling the Cosmos containers from Track and Eagle, and the gates that stop a bad seed
Sources-and-Status	Every data source and served surface, each in one state
Search-and-Retrieval	Retrieval overview; links to Search-Index-Reference, Search-Query-Construction, Search-Measurements
Search-Query-Construction	How GET /search builds a query and routes a read
Deep-Search-Queries	Deep Search: passages, chunk filters, degraded modes, the chunk gate
Native-List-Reads	Native list endpoints and mirror list reads: store choice, filters, paging
Chunk-Parent-Fields	The parent-document fields copied onto each chunk, and how the copy stays current
Chunk-Parent-Field-Backfill	Rolling the chunk parent fields into an environment, flags, repair
Extraction-Pipeline	How a PDF becomes searchable chunks
Extraction-Quality	Corpus coverage, word-joining, tiling, and what the quality metrics miss
Identity-Model	Which id means what, across Track, Eagle and DEMI
Sync-Architecture	How records get into DEMI, and what happens on delete
Eagle-Mirror-Reference	Mirrored kinds, refusal codes, parked retry, drift classes, stored report, migration re-push rule
ENGAGE-Rows-Reference	ENGAGE-owned comment periods: field ownership, join rules, tombstones, reconcile-engage report, sync-out settings
Public-Read-Backfill	Filling and repairing the comment period, comment, notification and list containers
Document-Intake	Upload link, Defender malware scan, release to the object store, scan cap
Connecting-an-Application-to-DEMI	How another application gets a credential
Recent-Uploads	GET /api/documents/recent-uploads — the eagle-public homepage panel
Reports	The comment export and BCGW CSV reads
User-Data	Saved map areas and preferences, /me routes, the userdata container
API-Gateway	The APIM gateway, its two APIs, the Front Door edge secret, caller IP
Azure-Environments	Subscriptions, estates, Bicep, and what it costs
Document-Backup	The Azure archive backup of document originals: account, access, run, check
Document-Backup-Restore	Reverting documents to the archive backup, and the restore drill
PDF-Titles	The PDF title job: what it writes, what it skips, how to run it on test and restore one document
Secret-Sync	Key Vault, the secret-sync Function app, required secret names
BC-Gov-Azure-Landing-Zone	The platform rules that block deploys
Business-Rules	BR-01…BR-09 — hold states, sensitivity ladder, ORCS mapping

Decisions

Each ADR page owns its own status detail.

ADR	Status
ADR-001-Typesense-for-DFL-Search	Superseded by Azure AI Search
ADR-002-Async-Processing-MongoDB-Queue	Superseded — change streams do not exist on the Cosmos Mongo API
ADR-003-Rule-Based-Classification	Accepted
ADR-004-Read-ACL-Authorization-Model	Accepted
ADR-005-Cosmos-Full-Text-Search-Ruled-Out	Rejected — fuzzy is a silent no-op
ADR-006-AI-Summarization-over-BM25	Accepted
ADR-007-Service-to-Service-Credentials	Accepted
ADR-008-Audit-and-Usage-Analytics-Store	Accepted
ADR-009-Track-Feed	Accepted
ADR-010-Attribute-Level-Access-Control	Accepted
ADR-011-Public-Read-Surface	Accepted
ADR-012-Public-Config-Surface	Accepted
ADR-013-Stored-Project-Summaries	Proposed
ADR-014-Sync-Out	Accepted
ADR-015-Malware-Scanning-on-Intake	Accepted

Cost

Azure-Environments owns the spend figures and the per-service breakdown.