Repository navigation
ADR 009 Track Feed
Accepted 2026-10-16. Option A (DEMI pulls) is what shipped. The nightly syncTrackTeams timer reads
GET /api/v1/projects and mirrors it into DEMI, and the syncTrackProjects timer polls the same
list every five minutes (eagle-demi PRs #627, #628), so a Track edit reaches DEMI within five
minutes. Option B (Track pushes) is not built; Track does not call PUT /api/projects/<trackProjectId>.
The timers are in Sync-Architecture § The Track side. The Context below is the state when the ADR
was proposed (2026-08-24).
Track (epictrack-api) owns the master project registry, and the Track project id is the master
project id across EPIC. DEMI stores it that way: a DEMI project's id is its trackProjectId
verbatim, and createProject refuses a project without one.
DEMI has no live connection to Track. Its entire Track dataset is a checked-in file,
src/data/track_projects_enriched.json, 382 records, snapshot 2026-07-29, read off disk by
src/seed/sources.js. Every project created, renamed, re-phased or withdrawn in Track since that
snapshot is invisible to DEMI, and DEMI serves eagle-public's test search. The rule is fail-closed,
so the symptom is absence rather than error: a project created after the snapshot never appears, and
nothing logs that it is missing.
Track runs in the Gold namespaces c8b80a-{dev,test,prod}; the API is
https://epictrack-api-c8b80a-<env>.apps.gold.devops.gov.bc.ca, and /ops/healthz is the
unauthenticated liveness path (every /api/v1 route answers 500, not 401, to a missing or bad
token). c72cba-* holds a dormant clone whose DeploymentConfigs sit at 0 replicas; its hostnames
resolve to the wildcard router and answer 503.
Ask Track for one of two directions. Option B is the recommendation: it costs Track one HTTP call, and it needs nothing built in DEMI and no credential from Track.
DEMI calls GET /api/v1/projects on epictrack-api on a schedule (nightly is enough; the data changes
at human speed) and reconciles. Track must supply:
- The epictrack-api base URL in each environment.
- A service credential: a Keycloak client in the
eao-epicrealm with theviewrole and audienceepictrack-web, issued to DEMI as a service account rather than a person's token. This is the critical path; the rest is a morning's work. - Confirmation of the response contract. DEMI assumes the endpoint is unpaginated and serves
epic_guidper project. - Whether deletes are observable. Without a tombstone or an is-deleted flag, reconciliation can only infer a delete from absence.
Track calls PUT /api/projects/<trackProjectId> on DEMI on create, on update, and on whatever Track
considers a withdrawal or delete. The path id is the Track project id because that is DEMI's own
project id. The route is authenticated and write-gated (src/http/routes.js, authMiddleware +
requireWrite), and DEMI mints Track a scoped, revocable, expiring key — see
Connecting-an-Application-to-DEMI.
No write access to Track: nothing in DEMI writes upstream, by design. No database access: an HTTP contract in either direction is enough. No real-time guarantee: minutes or hours of lag is fine, months is not.
- Master ids follow Track within five minutes. Before 2026-10-16 the id space was a snapshot, so a stale Track file was a stale primary key space, not a stale side table.
- The feed makes absence observable. A DEMI project Track no longer lists is counted as
orphanedand retained, never deleted; an Eagle project with no Track row is retained and flagged. -
src/data/track_projects_enriched.jsonis the seed's offline fallback only. - Option A adds two scheduled jobs and a Track client credential to rotate. Option B would have added neither, and moved the failure mode to Track missing a call.
- Sync-Architecture — every sync path, including this gap
- Identity-Model — which id means what, and the Eagle-only project rule
- Connecting-an-Application-to-DEMI — how DEMI issues a service key
- Environment Reality & Gotchas
- Architecture
- Seeding — filling Cosmos from Track and Eagle
- Sources and Status — every source, one state
- Search & Retrieval — overview
- Search Index Reference
- Search Query Construction
- Search Measurements
- Extraction Pipeline
- Extraction Quality — coverage, word-joining, tiling
- Identity Model — which id means what
- Attribute-Level Access — field-visibility levels, catalog
- Sync Architecture — how records get in
- Eagle Mirror Reference — mirrored kinds, refusals, drift classes
- ENGAGE Rows Reference — ENGAGE-owned comment periods, reconcile
- Public-Read Backfill — filling and repairing the mirror containers
-
Short Links —
/s/{code}redirect and/api/links -
Recent Uploads —
/api/documents/recent-uploads, eagle-public homepage panel - Reports — comment export and BCGW CSV reads
-
User Data — saved map areas, preferences,
/meroutes -
Bulk Download —
/api/bulk-downloads, zip jobs - Bulk Download: Operations — enable, poison-queue runbook
- Document Intake — upload link, malware scan, release
- Connecting an Application to DEMI
- Azure Environments
- Document Backup — archive copy of originals, run and check
- Document Backup: Revert and Drill — put documents back from the archive
- PDF Titles — public name written into stored PDFs, run, restore
- Secret Sync — Key Vault, secret-sync Function app, required names
-
Operating DEMI in Azure — deploy, verify
- Operating the DEMI Data Plane — devbox access, probes, audit and cost reads
- Edge ban detector — hourly timer, modes, kill switch, soak and promote
-
CI Workflows — every GitHub Actions workflow, trigger, identity, roles
- CI Deploy Workflows — eagle-demi staging and production deploys
- CI Admin Frontend Workflows — eagle-demi-admin checks and deploys
- Release Process — cutting, tagging, shipping and rolling back a version
- Runbook: Search Outage — index/app schema drift, diagnose, fix, roll back
- BC Gov Landing Zone
- Business Rules
- EPIC.search Business Rules — where DEMI differs from EPIC.search
- 001: Typesense for DFL Search — superseded
- 002: Async Processing (MongoDB Queue) — superseded
- 003: Rule-Based Classification
- 004: Read ACL Authorization Model
- 005: Cosmos Full-Text Search Ruled Out
- 006: AI Summarization over BM25
- 007: Service-to-Service Credentials
- 008: Audit and Usage Analytics Store
- 009: Track Feed — proposed
- 010: Attribute-Level Access Control
- 011: Public Read Surface
- 012: Public Config Surface
- 013: Stored Project Summaries — proposed
- 014: Sync-out to Eagle
- 015: Malware scanning on intake