Repository navigation
Runbooks
Index of the operational runbooks. Each row names a runbook, says when you reach for it, and whether the source describes it as a one-time step per deployment or a recurring one (blank where the source does not say). Hub operations live on this wiki's Runbooks pages; desktop/README.md (the Electron client) and frontend/README.md (the web app) keep their own. For the first deploy see Getting Started and Deploy and Upgrade; for symptom-first help see Troubleshooting; for the private network as a whole see Private Network.
| Runbook | When you need it | Cadence |
|---|---|---|
| Install, test, deploy | Root install, tests, the frontend build every cdk command needs, bootstrapping (including us-east-1) and the first manual hub deploy; also the Lambda concurrency raise a fresh account needs |
|
| Setup checklist | What the in-app setup card checks off (admin, branding, workload account, team, custom email) and how it is dismissed or reopened | |
| Theme wizard | How admins restyle the portal after deploy (/branding.json merged over config.json) and the by-name site bucket migration on the first deploy that includes it |
| Runbook | When you need it | Cadence |
|---|---|---|
| Add a user (and make one an admin) | Command-line recipe: create a user in the pool, get a token, register an account and role, vend credentials and grant a user through the admin API | recurring |
| First admin: created by the deploy |
firstAdmin in config makes the deploy create the first admin and send the invite; re-sending after the 7-day expiry, what prod: false means for the pool, and the SES sandbox decision to make before the first deploy |
one-time |
| Invite emails | What an invite contains with and without a custom web domain, the 7-day expiry, and the live invite check to run after any deploy that touches sign-in config | recurring |
| Federated sign-in (SSO) | Register the portal at the customer's identity provider (Entra worked example), put the client secret in Secrets Manager, map the customer's own identifier (a GID) as a claim, rotate the secret, remove a provider, and what the desktop app needs | one-time per provider |
| Service users: machine access to the API | What a service user is (a client-credentials app client with scopes: aws, network, shares, admin.read.all, admin.write.all), how its auth and revocation work, how to treat its secret, and the headless vend-and-join sequence |
|
| Add one (admin UI) | Create a service user, capture the shown-once client ID and secret, grant it roles and call the API with a token | recurring |
| Rotate a secret | Replace-with-overlap rotation on a suspected leak, a holder leaving, a lost secret or a policy demand; remove the old client once the new one shows as used | recurring |
| Cost note (per clone) | Cognito's usage-based machine-to-machine pricing |
| Runbook | When you need it | Cadence |
|---|---|---|
| What a deployment is turning on | What a plugin is, the two permission levels, and why nobody — core admin included — holds plugin access until it is granted | |
| Turning it on | The three-deploy rollout, and the build step a registered plugin needs before synth | one-time |
| Granting, and what a dormant grant is | Grant and revoke from the Users and Groups screens, user expiry versus standing group access, and why a dormant grant is kept rather than deleted | recurring |
| Disabling and removing | What disable, remove and reinstall each do to grants, and the two-deploy order for switching the feature off | |
| What revocation promises | Why a completed revocation bites on the next request, and what it does not stop |
| Runbook | When you need it | Cadence |
|---|---|---|
| Cross-account: trust a second account | Bootstrap and deploy Roles-<account> into a workload account; fixed role names and the destroy-then-redeploy rule on an "already exists" error; the 12 h session-length cap |
|
| Sub-pinning: role-scoped trust | Why each role's trust pins a role-scoped sub, and the load-bearing order: CredsStack first, then every Roles-<account>, then vend end to end |
See Access Model for how grants turn into vended credentials.
| Runbook | When you need it | Cadence |
|---|---|---|
| Signing-key rotation (issuer) | Introduce, wait 10+ minutes, flip, retire: rotating the issuer's signing key through issuer.keyVersions and signerVersion; retiring orphans a KMS key under prod: true
|
recurring |
| Issuer on S3 + CloudFront | What the 0.1.8 issuer migration changes (certificate to us-east-1, WebCertStack on every deployment) and what it leaves alone (issuer URL, keys, every workload account's trust) | one-time |
| Zero-downtime sequencing | The order CloudFormation applies the migration in and what a mid-way failure rolls back to | |
| Before pushing it | Preconditions: us-east-1 bootstrapped, a clean current tree, no key rotation in the same deploy | one-time |
| Verify before any client gets 0.1.8 |
dig and curl checks of the discovery document and JWKS, a real vend, then the ACM CNAME, old certificate, orphaned log group and Security Hub follow-ups |
one-time |
| Republish by hand | Invoke the publisher Lambda with a Create-shaped event if the issuer bucket was ever emptied | |
| Rollback |
git revert the migration commit and push; CloudFormation rebuilds the API Gateway path |
|
| Restore from backup (PITR) | Restore a table to a side table, verify counts, copy rows back into the live table; a restored table comes back with TTL and PITR disabled |
| Runbook | When you need it | Cadence |
|---|---|---|
| Email deliverability: Cognito via SES | The email block: SES identity plus DKIM, MAIL FROM and DMARC records, the sandbox limitation, requesting production access and checking SPF/DKIM/DMARC |
one-time |
| Custom web domain | The web block: certificate in us-east-1, CloudFront alias, Cognito custom auth domain (a child of the app domain), first-deploy timing and cutover notes |
one-time |
| Runbook | When you need it | Cadence |
|---|---|---|
| Security posture: WAF + Security Hub CSPM | Entry point: how a stock deployment relates to the Foundational Security Best Practices control set | |
| What deploys (default on) | The three web ACLs, the per-IP and per-token rate rules, managed rule groups in COUNT, logging, cost, the waf config, 403/429 triage and the hand cleanup when turning WAF off |
|
| Accepted findings | CloudFront.17 cannot pass; suppress it by workflow status per hub account (or disable the control / add an automation rule in an administrator account) | one-time |
| FSBP coverage | Control-by-control status: by construction, with prod: true, opt-in knobs, unbuilt cost knobs, accepted by design, account-level; upgrade notes for existing deployments |
Background in Security Posture.
Network stacks are deployed by hand, never from CI. Concepts are on Private Network; procedures on Runbooks-Network.
| Runbook | When you need it | Cadence |
|---|---|---|
| Adding a VPC to the network (ConnectorStack) | Declare the VPC in its site, allow the account on the site's connector minter, deploy NetworkStack then Connector-<account>; add split DNS if the VPC has private names |
recurring |
| Which network runbook do I need? | Decision list: first migration to sites, adding a non-colliding VPC, or adding a site for a colliding CIDR | |
| One-time migration: single plane to sites | Moving a pre-sites deployment onto per-site control planes; re-enrolls every device and connector once | one-time |
| Adding a SITE (a new control plane) | Only when a new VPC's CIDR collides with an existing site; site ids are permanent; deploy NetworkStack, push promptly, then the site's connectors | |
| More security-model notes | Reconciler freshness (rules render every 5 minutes), two-keyed reachability (route plus security group), Headscale state on EFS | |
| Split DNS (private names over the VPN) | Per-site splitDns maps a private domain to its VPC resolver; one config entry plus a NetworkStack redeploy |
|
| Removing a SITE | Delete the site from config and redeploy NetworkStack, then delete its API-key secret by hand, the one manual VPN step left | |
| Headscale admin ops | ECS Exec into a site's Headscale task, how the deploy mints the infra user and admin API key, and rotating that key with -c headscaleKeyRotation=<stamp>
|
| Runbook | When you need it | Cadence |
|---|---|---|
| Debugging | Where the API logs are and the per-request JSON trail; cached 401s at CloudFront and how to tell an API Gateway 401 from the app's | |
| Tear down | Lift termination and deletion protection, destroy in reverse order, then clean up retained tables, web ACLs, EFS recovery points and test users |
| Runbook | When you need it | Cadence |
|---|---|---|
| Package (distribution) |
npm run dist builds the one public binary; where the app name, bundle id and runtime strings come from; what unsigned builds can and cannot do |
recurring |
| Code signing & notarization (macOS) | How the desktop-build workflow signs, notarizes and Gatekeeper-verifies, why it is pinned to macos-15, the one-time Apple enrollment behind the five repository secrets, and adding those secrets to a repository that cuts releases |
one-time (setup) |
| Windows installer | How the Windows installer is signed, what a user may click through on first launch | recurring |
| Passkeys (per platform) | Passkey support on Windows, signed macOS builds and Linux, plus the Windows verification checklist |
| Runbook | When you need it | Cadence |
|---|---|---|
| Build + deploy |
npm run build then cdk deploy WebStack; where WebStack looks for a bundle and the "Web assets not found" failure |
recurring |
| E2E (Playwright) | The suite against a live deployment: setup once per machine (e2e users, .env.e2e, Chromium), the commands, and re-approving screenshot baselines |
Release mechanics (packing the tarball, cutting a desktop build) are on Releasing.
Source: wiki/ in FNTIO/aws-infrastructure-client, mirrored by the Wiki workflow. Edit the files there; changes made on this site are overwritten by the next sync.
Start here
Use
How it works
Operate
Contribute