Repository navigation
CI Workflows
Reference for every GitHub Actions workflow that touches digitalspace/eagle-demi and
digitalspace/eagle-demi-admin. How to deploy by hand, verify a deploy, and what each script does
at the Azure end is Operating-DEMI-in-Azure. How a version number gets cut and shipped is
Release-Process.
| Workflow | File | Trigger | Concurrency group |
|---|---|---|---|
| PR Validation | pr.yaml |
pull_request → main
|
pr-<PR number>, cancels in progress |
| Deploy DEMI API to Azure staging | azure-deploy-staging-api.yaml |
push to main under API paths, or workflow_dispatch
|
deploy-azure-staging-api, queues |
| Deploy DEMI Frontend to Azure staging | azure-deploy-staging-frontend.yaml |
push to main under frontend/**, or workflow_dispatch
|
deploy-azure-staging-frontend, queues |
| Deploy DEMI secret sync to Azure staging | azure-deploy-staging-secret-sync.yaml |
push to main under src/secret-sync/** (plus the shared logger/config paths), or workflow_dispatch
|
deploy-azure-staging-secret-sync, queues |
| Deploy DEMI PDF title worker to Azure staging | azure-deploy-staging-pdf-title.yaml |
push to main under pdf-title/** (not pdf-title/test_*.py), scripts/package-pdf-title.sh or the workflow file, or workflow_dispatch
|
deploy-azure-staging-pdf-title, queues |
| Deploy DEMI to Azure production | azure-deploy-prod.yaml |
workflow_dispatch with a vX.Y.Z tag |
deploy-azure-prod, queues |
| Tag and draft release | draft-release.yaml |
push to main, or workflow_dispatch
|
draft-release, queues |
| CodeQL Advanced | codeql.yml |
pull_request → main, weekly (25 1 * * 0 UTC) |
codeql-<ref>, cancels in progress |
| PR checks (eagle-demi-admin) | pr.yaml |
pull_request → main
|
pr-<PR number>, cancels in progress |
| Deploy to Azure test (eagle-demi-admin) | deploy-test.yaml |
push to main under app source paths, or workflow_dispatch
|
deploy-azure-test, queues |
| Deploy to Azure production (eagle-demi-admin) | deploy-prod.yaml |
workflow_dispatch with a vX.Y.Z tag |
deploy-azure-prod, queues |
The two staging deploys and CodeQL share a repository but not a workflow: a React-only change
under frontend/** never touches the Flex app, and an API change never re-publishes the bundle.
Each declares its own concurrency group so the two cannot block each other.
Runs on every pull request against main. Four jobs, all read-only (permissions: contents: read at the workflow level, so a PR from Dependabot or a fork keeps the same token scope as one
from a maintainer):
-
Test & Lint API —
yarn lint,yarn test, then the extractor's own Python suite (extractor/test_extract.py), which checksdecide()againstextraction-host/worker.pyover 3,000 generated inputs. Node 22 (the Flex host's runtime), Python 3.11,requests==2.34.2pinned for the extractor test dependency. -
Test & Build Frontend —
yarn --cwd frontend lint,test,build, then a browser smoke test of the build. Node 24, job timeout 15 minutes. Playwright is not a frontend dependency: the job installs a pinned version and headless Chromium into$RUNNER_TEMP, and passes its path to the smoke script inPLAYWRIGHT_MODULE. The smoke step serves the build withvite previewon127.0.0.1:4173, waits for it to answer, then runsfrontend/scripts/e2e-smoke.mjs. The script opens 12 screens and 8 legacy redirects with Keycloak off and every API answer served fromfrontend/parity/fixtures/. Each page must answer 200 with no console errors, each screen must show its heading, and each redirect must land on its target; any miss fails the job. -
search-schema-change — fires only when the diff touches
azure/search/**or a*_SELECTline insrc/search/ai-search.js. Probes the test API (scripts/search-schema-probe.sh) with the branch's index definitions, and requires the PR description to carry the lineSearch-Schema: applied test— the author's statement that they already ranscripts/demi-devbox.sh apply --env test. Production is still widened by hand at release time; this gate exists so forgetting it on test is visible before the PR merges. -
Validate Bicep Template —
az bicep buildonmain.bicepandmodules/extractor.bicep(no Azure login: compiles locally, and login is deliberately absent so this job cannot deploy anything even by mistake), thenaz bicep build-paramson bothmain.test.bicepparamandmain.prod.bicepparamwith one placeholder,DEVBOX_SSH_PUBLIC_KEY. Every credential the app reads (admin-api-key,docling-api-key,minio-access-key,minio-secret-key,track-client-secret,role-sync-client-secretand the rest) lives indemi-kv-<env>— see Secret-Sync — and the Bicep carries no parameter for any of them, so there is nothing left to placeholder for those.DEVBOX_SSH_PUBLIC_KEYis not a credential; it still callsreadEnvironmentVariablewith no default, so an unset export fails compilation (BCP427) rather than silently resolving to an empty string. CI supplies an obviously-fake value only to prove the rest of the param file still resolves; this job has no Azure login and reaches nothing.
The staging deploys (API, frontend, secret sync, PDF title worker) and the production deploy, with their steps, identities, Azure roles and rollback, are in CI-Deploy-Workflows.
See Release-Process — this workflow is what cuts the version.
Static analysis across three language matrices (actions, javascript-typescript, python),
each build-mode: none, with the security-extended query suite. Runs on every pull request to
main and on a weekly schedule (25 1 * * 0 UTC — Sunday 01:25); no push: trigger. Findings
post to the repository's Security tab, not to the PR itself. permissions: security-events: write
plus packages: read and actions: read for private CodeQL packs.
The PR checks and the test and production deploys for eagle-demi-admin, with their identities and
Azure roles, are in CI-Admin-Frontend-Workflows.
None of the four identities in CI-Deploy-Workflows and CI-Admin-Frontend-Workflows hold
anything at resource-group scope. demi-cicd-test and
demi-cicd-prod each hold Website Contributor on one Function App; the two admin/frontend
identities hold two Storage roles on one storage account. That is what makes "CI cannot run an ARM
deployment" a fact about the credential rather than a convention about the workflow file. Bicep
changes go through scripts/deploy-infra.sh by hand — see Azure-Environments and
Operating-DEMI-in-Azure.
Measured 2026-09-05, staging runs:
- API deploy: about 213 s end to end.
az functionapp deployment source config-zipagainst Flex Consumption accounts for 190–193 s of that — Azure-side OneDeploy (upload, validate, extract, restart); the call is not passed--no-wait.yarn install --immutabletakes about 8 s. Measured 2026-09-05 after pruning: the deploy step took 188 s (run 33955143467), against 190 to 193 s before. Zip size does not move the Flex deploy time; the floor is Azure-side provisioning. The zippackage-api.pyproduces, measured 2026-09-05: 12,728 files / 21.7 MB, pruned from 23,368 files / 33.4 MB beforeyarn workspaces focus --productionand the extra exclusions (.yarn,*.d.ts,LICENSE*,CHANGELOG*,local.settings.json,*.pem,*.key,*.p12,*.pfx,.npmrc). - Frontend deploy: 45–58 s end to end, dominated by the React build and the three-pass blob upload.
The sed-then-grep pairs in both frontend workflows, and the 10×20s registration polls in both
API/extractor deploy jobs, exist because each failure mode was seen live, not designed against on
paper:
-
2026-08-11 — a staging frontend build shipped with
env.jsstill pointing at the dev Keycloak realm and the dev API. The bundle looked healthy; every sign-in went through the wrong realm. The rewrite-then-assert pattern in both frontend workflows is the guard. -
2026-08-13 — an infra
what-ifrun with an unset secret param resolved to an empty string and overwroteADMIN_API_KEYandDOCLING_API_KEYon the live app;what-ifmasks@secure()values as*******in both the before and after, so nothing in the output showed it. Only MinIO's credential survived, because OpenShift held an authoritative copy elsewhere. The Bicep no longer carries a parameter for any of those values — every credential the app reads comes fromdemi-kv-<env>by reference instead, so a template run cannot blank one this way again. See Secret-Sync.pr.yaml's Bicep validation job still fails the build (BCP427) rather than a quoted empty string on the one parameter that remains,DEVBOX_SSH_PUBLIC_KEY, andscripts/deploy-infra.shstill requiresCONFIRM_PROD=yesbefore an apply. -
Kudu status stuck at 1 for 30+ minutes against a healthy 30 s — an oversized deploy package
(a reconvergent symlink walk enumerating the same directory through every path to it, or a
.claude/worktrees/*checkout caught by a naivedistprune) produced a 202 MB zip that hung the deploy.scripts/package-api.py'swalk()bounds the traversal to one symlink hop per path for this reason — see its docstring for the two broader rules that were tried and both lost content. -
scripts/wait-for-function.sh's 10×20s poll after everyconfig-zipexists because a Flex publish reporting success is not the same fact as the HTTP or queue trigger being registered — the host can load an app that registers nothing, and every route then 404s (or, for the extractor, the queue never drains) with no other signal. - The staging API deploy's
/healthsmoke test exists for the fact one layer up: a route registering is not the same fact as it serving. It runs after the wait-for-function poll, not instead of it.
Read from the live repository, 2026-09-05:
-
maincarries no branch protection and no ruleset. PR Validation runs on every pull request but nothing blocks a merge on it failing. - The default
GITHUB_TOKENpermission at the repository level is write.pr.yamlnarrows its own token tocontents: readat the workflow level — every job in it only reads the checkout, so the narrower scope costs nothing and keeps Dependabot's PRs running with the same token shape as a maintainer's. -
prodenvironment: see CI-Deploy-Workflows.
Planned, not yet applied: the default GITHUB_TOKEN permission moving to read-only at the
repository level, and a ruleset on main requiring the PR checks to pass before merge.
- CI-Deploy-Workflows — eagle-demi staging and production deploys
- CI-Admin-Frontend-Workflows — eagle-demi-admin PR checks and deploys
- Operating-DEMI-in-Azure — deploy by hand, restart, verify
- Release-Process — how a version is cut, tagged and shipped
- Azure-Environments — subscriptions, Bicep, cost
- Environment Reality & Gotchas
- Architecture
- Seeding — filling Cosmos from Track and Eagle
- Sources and Status — every source, one state
- Search & Retrieval — overview
- Search Index Reference
- Search Query Construction
- Search Measurements
- Extraction Pipeline
- Extraction Quality — coverage, word-joining, tiling
- Identity Model — which id means what
- Attribute-Level Access — field-visibility levels, catalog
- Sync Architecture — how records get in
- Eagle Mirror Reference — mirrored kinds, refusals, drift classes
- ENGAGE Rows Reference — ENGAGE-owned comment periods, reconcile
- Public-Read Backfill — filling and repairing the mirror containers
-
Short Links —
/s/{code}redirect and/api/links -
Recent Uploads —
/api/documents/recent-uploads, eagle-public homepage panel - Reports — comment export and BCGW CSV reads
-
User Data — saved map areas, preferences,
/meroutes -
Bulk Download —
/api/bulk-downloads, zip jobs - Bulk Download: Operations — enable, poison-queue runbook
- Document Intake — upload link, malware scan, release
- Connecting an Application to DEMI
- Azure Environments
- Document Backup — archive copy of originals, run and check
- Document Backup: Revert and Drill — put documents back from the archive
- PDF Titles — public name written into stored PDFs, run, restore
- Secret Sync — Key Vault, secret-sync Function app, required names
-
Operating DEMI in Azure — deploy, verify
- Operating the DEMI Data Plane — devbox access, probes, audit and cost reads
- Edge ban detector — hourly timer, modes, kill switch, soak and promote
-
CI Workflows — every GitHub Actions workflow, trigger, identity, roles
- CI Deploy Workflows — eagle-demi staging and production deploys
- CI Admin Frontend Workflows — eagle-demi-admin checks and deploys
- Release Process — cutting, tagging, shipping and rolling back a version
- Runbook: Search Outage — index/app schema drift, diagnose, fix, roll back
- BC Gov Landing Zone
- Business Rules
- EPIC.search Business Rules — where DEMI differs from EPIC.search
- 001: Typesense for DFL Search — superseded
- 002: Async Processing (MongoDB Queue) — superseded
- 003: Rule-Based Classification
- 004: Read ACL Authorization Model
- 005: Cosmos Full-Text Search Ruled Out
- 006: AI Summarization over BM25
- 007: Service-to-Service Credentials
- 008: Audit and Usage Analytics Store
- 009: Track Feed — proposed
- 010: Attribute-Level Access Control
- 011: Public Read Surface
- 012: Public Config Surface
- 013: Stored Project Summaries — proposed
- 014: Sync-out to Eagle
- 015: Malware scanning on intake