feat(api): hot-reload webhook for the public data branch#70
Merged
Conversation
themightychris
added a commit
that referenced
this pull request
May 19, 2026
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Captures the contract for POST /api/_internal/reload-data: bearer-auth, optional body, cheap ancestry pre-check, lock-protected reconcile, in-place state rebuild. Plan tracks #65. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /api/_internal/reload-data — closes the loop on #65. Hidden from the OpenAPI doc; bearer-auth via CFP_DATA_RELOAD_SECRET with constant- time compare; 503 when the secret is unset so the deployment surface stays uniform. Two layers of no-op coverage: 1. Cheap pre-check via `git merge-base --is-ancestor` — handles the self-trigger from push-daemon-emitted pushes without acquiring the data-repo lock or hitting the network. 2. After a reconcile, outcome === 'in-sync' short-circuits the rebuild. Otherwise the reconcile state machine (#66) runs under the data-repo lock and the helper at `store/memory/reload.ts` re-opens the public store (gitsheets caches a dataTree per Sheet, so the snapshot has to be replaced after a fast-forward), builds a fresh InMemoryState, mutates the live Maps in place, swaps the public-store reference, reloads the FTS index in a single SQLite transaction, and invalidates the facet cache. If the rebuild throws after the swap has begun, the route logs loudly and returns 500 so the operator knows a restart is warranted. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds CFP_DATA_RELOAD_SECRET to the env-var table in deploy.md and a "Hot-reload webhook" section to runbook.md covering manual triggering and the response shapes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
91333f4 to
10abee7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #65. Builds on #66 (PR #68) — the data-repo reconciliation moved into the Node process, which this PR consumes via
fastify.reconcileDataRepo.Summary
POST /api/_internal/reload-data— hidden from the OpenAPI doc, bearer-auth viaCFP_DATA_RELOAD_SECRET, constant-time compare, 503 when the secret is unsetgit merge-base --is-ancestorshort-circuits the self-trigger case (push-daemon's own commits arriving via the webhook) without acquiring the data-repo lockoutcome === 'in-sync'short-circuits the rebuild; otherwise the in-memory state + FTS index are rebuilt in place against the post-fast-forward treeplans/hot-reload-webhook.md; spec section inspecs/behaviors/storage.md; operator notes indocs/operations/runbook.mdDecisions worth flagging
Storecaches adataTreeper Sheet. A fast-forward merge updates the working tree but the cached snapshot stays stale, soqueryAll()returns the pre-merge records. The reload helper re-opens the public store andStore.swapPublics it. The transact path is unaffected —repo.transactbuilds a fresh workspace from the parent commit each call.fastify.inMemoryState. Service classes captured the state object at boot; replacing it would orphan them. The helper builds a fresh state to a local variable, thenclear()+set()every Map in a tight synchronous block on the live object. Failure during the build leaves the running state untouched; failure during the FTS reload is loud and the route 5xx's so the operator restarts.Authorizationheader gets 401 before the route checks whether the secret is configured. This way unauthenticated probes can't tell whether the env var is set; only callers that pass some bearer token receive a 503-vs-401 distinction (and a sealed-secret operator can't pose as an unauthenticated probe).Operator follow-up
This PR intentionally doesn't include the GitHub Actions workflow that calls the webhook — that lives on the
codeforphilly-datarepo. Drop the following intocodeforphilly-data/.github/workflows/notify-deployments.yml:```yaml
name: Notify deployments
on:
push:
branches: [published]
jobs:
notify:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
target:
- { name: sandbox, url: 'https://next-v2.codeforphilly.org/api/_internal/reload-data' }
# - { name: prod, url: '...' } when prod stands up
steps:
- name: Trigger reload on ${{ matrix.target.name }}
env:
CFP_DATA_RELOAD_SECRET: ${{ secrets.CFP_DATA_RELOAD_SECRET }}
run: |
curl --fail-with-body -sS -X POST "${{ matrix.target.url }}"
-H "Authorization: Bearer $CFP_DATA_RELOAD_SECRET"
-H "Content-Type: application/json"
-d "$(jq -nc --arg sha "${{ github.sha }}" '{branch:"published", commitHash:$sha}')"
```
The
CFP_DATA_RELOAD_SECRETrepository (or environment) secret oncodeforphilly-datamust match the cluster-side sealed Secret. Per environment, the sealed Secret lives incfp-sandbox-cluster/codeforphilly-ng.secrets/(or production equivalent) — not in this repo. Generate withopenssl rand -base64 48, then seal into the GitOps repo and mirror the same plaintext into the data repo's secret.Test plan
npm run -w apps/api type-check— cleannpm run -w apps/api test— 240 tests pass (including 7 new ininternal-reload.test.ts)npm run lint— cleanCFP_DATA_RELOAD_SECRET, mirror it into thecodeforphilly-datasecret, push a no-op commit onpublished, verify the workflow's curl logs the 200 noChanges response and the API logshot-reload short-circuit: commit already in local HEAD.legacy-import→publishedchange, verify the workflow's curl logs the 200outcome: fast-forwardedresponse and the new records are visible immediately without a pod restart.