Skip to content

CI Workflows

Daniel Truong edited this page Oct 7, 2026 · 13 revisions

CI Workflows

Reference for every GitHub Actions workflow that touches digitalspace/eagle-demi and digitalspace/eagle-demi-admin. How to deploy by hand, verify a deploy, and what each script does at the Azure end is Operating-DEMI-in-Azure. How a version number gets cut and shipped is Release-Process.

Workflows

Workflow File Trigger Concurrency group
PR Validation pr.yaml pull_request → main pr-<PR number>, cancels in progress
Deploy DEMI API to Azure staging azure-deploy-staging-api.yaml push to main under API paths, or workflow_dispatch deploy-azure-staging-api, queues
Deploy DEMI Frontend to Azure staging azure-deploy-staging-frontend.yaml push to main under frontend/**, or workflow_dispatch deploy-azure-staging-frontend, queues
Deploy DEMI secret sync to Azure staging azure-deploy-staging-secret-sync.yaml push to main under src/secret-sync/** (plus the shared logger/config paths), or workflow_dispatch deploy-azure-staging-secret-sync, queues
Deploy DEMI PDF title worker to Azure staging azure-deploy-staging-pdf-title.yaml push to main under pdf-title/** (not pdf-title/test_*.py), scripts/package-pdf-title.sh or the workflow file, or workflow_dispatch deploy-azure-staging-pdf-title, queues
Deploy DEMI to Azure production azure-deploy-prod.yaml workflow_dispatch with a vX.Y.Z tag deploy-azure-prod, queues
Tag and draft release draft-release.yaml push to main, or workflow_dispatch draft-release, queues
CodeQL Advanced codeql.yml pull_request → main, weekly (25 1 * * 0 UTC) codeql-<ref>, cancels in progress
PR checks (eagle-demi-admin) pr.yaml pull_request → main pr-<PR number>, cancels in progress
Deploy to Azure test (eagle-demi-admin) deploy-test.yaml push to main under app source paths, or workflow_dispatch deploy-azure-test, queues
Deploy to Azure production (eagle-demi-admin) deploy-prod.yaml workflow_dispatch with a vX.Y.Z tag deploy-azure-prod, queues

The two staging deploys and CodeQL share a repository but not a workflow: a React-only change under frontend/** never touches the Flex app, and an API change never re-publishes the bundle. Each declares its own concurrency group so the two cannot block each other.

PR Validation

Runs on every pull request against main. Four jobs, all read-only (permissions: contents: read at the workflow level, so a PR from Dependabot or a fork keeps the same token scope as one from a maintainer):

  • Test & Lint API — yarn lint, yarn test, then the extractor's own Python suite (extractor/test_extract.py), which checks decide() against extraction-host/worker.py over 3,000 generated inputs. Node 22 (the Flex host's runtime), Python 3.11, requests==2.34.2 pinned for the extractor test dependency.
  • Test & Build Frontend — yarn --cwd frontend lint, test, build, then a browser smoke test of the build. Node 24, job timeout 15 minutes. Playwright is not a frontend dependency: the job installs a pinned version and headless Chromium into $RUNNER_TEMP, and passes its path to the smoke script in PLAYWRIGHT_MODULE. The smoke step serves the build with vite preview on 127.0.0.1:4173, waits for it to answer, then runs frontend/scripts/e2e-smoke.mjs. The script opens 12 screens and 8 legacy redirects with Keycloak off and every API answer served from frontend/parity/fixtures/. Each page must answer 200 with no console errors, each screen must show its heading, and each redirect must land on its target; any miss fails the job.
  • search-schema-change — fires only when the diff touches azure/search/** or a *_SELECT line in src/search/ai-search.js. Probes the test API (scripts/search-schema-probe.sh) with the branch's index definitions, and requires the PR description to carry the line Search-Schema: applied test — the author's statement that they already ran scripts/demi-devbox.sh apply --env test. Production is still widened by hand at release time; this gate exists so forgetting it on test is visible before the PR merges.
  • Validate Bicep Template — az bicep build on main.bicep and modules/extractor.bicep (no Azure login: compiles locally, and login is deliberately absent so this job cannot deploy anything even by mistake), then az bicep build-params on both main.test.bicepparam and main.prod.bicepparam with one placeholder, DEVBOX_SSH_PUBLIC_KEY. Every credential the app reads (admin-api-key, docling-api-key, minio-access-key, minio-secret-key, track-client-secret, role-sync-client-secret and the rest) lives in demi-kv-<env> — see Secret-Sync — and the Bicep carries no parameter for any of them, so there is nothing left to placeholder for those. DEVBOX_SSH_PUBLIC_KEY is not a credential; it still calls readEnvironmentVariable with no default, so an unset export fails compilation (BCP427) rather than silently resolving to an empty string. CI supplies an obviously-fake value only to prove the rest of the param file still resolves; this job has no Azure login and reaches nothing.

eagle-demi deploy workflows

The staging deploys (API, frontend, secret sync, PDF title worker) and the production deploy, with their steps, identities, Azure roles and rollback, are in CI-Deploy-Workflows.

Tag and draft release

See Release-Process — this workflow is what cuts the version.

CodeQL Advanced

Static analysis across three language matrices (actions, javascript-typescript, python), each build-mode: none, with the security-extended query suite. Runs on every pull request to main and on a weekly schedule (25 1 * * 0 UTC — Sunday 01:25); no push: trigger. Findings post to the repository's Security tab, not to the PR itself. permissions: security-events: write plus packages: read and actions: read for private CodeQL packs.

eagle-demi-admin workflows

The PR checks and the test and production deploys for eagle-demi-admin, with their identities and Azure roles, are in CI-Admin-Frontend-Workflows.

Why infrastructure has no CI path

None of the four identities in CI-Deploy-Workflows and CI-Admin-Frontend-Workflows hold anything at resource-group scope. demi-cicd-test and demi-cicd-prod each hold Website Contributor on one Function App; the two admin/frontend identities hold two Storage roles on one storage account. That is what makes "CI cannot run an ARM deployment" a fact about the credential rather than a convention about the workflow file. Bicep changes go through scripts/deploy-infra.sh by hand — see Azure-Environments and Operating-DEMI-in-Azure.

Measured timings

Measured 2026-09-05, staging runs:

  • API deploy: about 213 s end to end. az functionapp deployment source config-zip against Flex Consumption accounts for 190–193 s of that — Azure-side OneDeploy (upload, validate, extract, restart); the call is not passed --no-wait. yarn install --immutable takes about 8 s. Measured 2026-09-05 after pruning: the deploy step took 188 s (run 33955143467), against 190 to 193 s before. Zip size does not move the Flex deploy time; the floor is Azure-side provisioning. The zip package-api.py produces, measured 2026-09-05: 12,728 files / 21.7 MB, pruned from 23,368 files / 33.4 MB before yarn workspaces focus --production and the extra exclusions (.yarn, *.d.ts, LICENSE*, CHANGELOG*, local.settings.json, *.pem, *.key, *.p12, *.pfx, .npmrc).
  • Frontend deploy: 45–58 s end to end, dominated by the React build and the three-pass blob upload.

What each guard defends against

The sed-then-grep pairs in both frontend workflows, and the 10×20s registration polls in both API/extractor deploy jobs, exist because each failure mode was seen live, not designed against on paper:

  • 2026-08-11 — a staging frontend build shipped with env.js still pointing at the dev Keycloak realm and the dev API. The bundle looked healthy; every sign-in went through the wrong realm. The rewrite-then-assert pattern in both frontend workflows is the guard.
  • 2026-08-13 — an infra what-if run with an unset secret param resolved to an empty string and overwrote ADMIN_API_KEY and DOCLING_API_KEY on the live app; what-if masks @secure() values as ******* in both the before and after, so nothing in the output showed it. Only MinIO's credential survived, because OpenShift held an authoritative copy elsewhere. The Bicep no longer carries a parameter for any of those values — every credential the app reads comes from demi-kv-<env> by reference instead, so a template run cannot blank one this way again. See Secret-Sync. pr.yaml's Bicep validation job still fails the build (BCP427) rather than a quoted empty string on the one parameter that remains, DEVBOX_SSH_PUBLIC_KEY, and scripts/deploy-infra.sh still requires CONFIRM_PROD=yes before an apply.
  • Kudu status stuck at 1 for 30+ minutes against a healthy 30 s — an oversized deploy package (a reconvergent symlink walk enumerating the same directory through every path to it, or a .claude/worktrees/* checkout caught by a naive dist prune) produced a 202 MB zip that hung the deploy. scripts/package-api.py's walk() bounds the traversal to one symlink hop per path for this reason — see its docstring for the two broader rules that were tried and both lost content.
  • scripts/wait-for-function.sh's 10×20s poll after every config-zip exists because a Flex publish reporting success is not the same fact as the HTTP or queue trigger being registered — the host can load an app that registers nothing, and every route then 404s (or, for the extractor, the queue never drains) with no other signal.
  • The staging API deploy's /health smoke test exists for the fact one layer up: a route registering is not the same fact as it serving. It runs after the wait-for-function poll, not instead of it.

GitHub repository settings

Read from the live repository, 2026-09-05:

  • main carries no branch protection and no ruleset. PR Validation runs on every pull request but nothing blocks a merge on it failing.
  • The default GITHUB_TOKEN permission at the repository level is write. pr.yaml narrows its own token to contents: read at the workflow level — every job in it only reads the checkout, so the narrower scope costs nothing and keeps Dependabot's PRs running with the same token shape as a maintainer's.
  • prod environment: see CI-Deploy-Workflows.

Planned, not yet applied: the default GITHUB_TOKEN permission moving to read-only at the repository level, and a ruleset on main requiring the PR checks to pass before merge.

See also

DEMI Wiki

Home

Current system

Azure

Domain

ADRs

History

Related

Clone this wiki locally