Operations and Troubleshooting

Day-two operation: the commands you actually use, where the stack listens, where the logs are, and what the common failures look like. Installing is Installation; upgrading has its own page at Upgrades and Rollback.

One rule before anything else

Bring the box up with forge-deploy --up, not bare docker compose up. A bare compose brings up every service in the file set — including components you deliberately removed from this box, and profile-gated extras you never wanted — and resurrects them on every subsequent restart. forge-deploy compose <args> is the scope-aware passthrough when you genuinely need compose itself.

Everyday commands

Command	What it does
forge-deploy	The guided console — reads the machine, reports in plain language, offers a menu. This is the operator-facing surface; everything below is tooling
forge-deploy --status	Deployed tag and container health for this box, and it flags drift between the running tag and the recorded deploy
forge-deploy --up	Start only this box's components
forge-deploy --logs	Tail the deploy history log
forge-deploy --list	Recent versions paired with their build SHA
forge-deploy --components	Show or set which components this box runs
forge-preflight	Read-only doctor: validates the things that have actually bitten deployments and fails loud with the fix. Changes nothing
forge-deploy --recover	Detects common failure modes and an incomplete setup, then fixes and resumes in place, keeping your data

forge-preflight before a change and --recover after one that went wrong are the two habits worth forming. --recover is not destructive; the destructive one is --fresh-start, which wipes containers, database, files and config after a typed confirmation.

Where the stack listens

Every published port binds to 127.0.0.1 by default. Nothing is reachable from the network until you deliberately widen it, which is why a fresh install is safe on an untrusted LAN and also why "I can't reach it from my laptop" is the most common first question.

Each service takes a *_BIND and a *_PORT variable in .env — UI_BIND/UI_PORT, API_BIND/API_PORT, POSTGRES_BIND/POSTGRES_PORT, MINIO_BIND/MINIO_API_PORT and MINIO_CONSOLE_PORT, plus the optional profiles (AI, TTS, crash reporting, document signing, the demo and test SPAs). Widening is per service: set the bind to the interface you want, or leave it on loopback and put a reverse proxy in front, which is what --lan, --public and --cohost set up for you.

Two rules worth stating plainly:

• Widen the UI, not the database. There is rarely a reason to expose Postgres or the MinIO console beyond loopback; if you need remote database access, tunnel to it.
• A split install is the exception — the API box must reach Postgres and object storage, and the UI box must reach the API. Wire those with --remote-db and --remote-api rather than by hand, and firewall the widened ports to the peer box. See Hardening a Production Install.

Health and logs

The API publishes a composite health report at /api/v1/health covering PostgreSQL, Hangfire, object storage and SignalR, and that is what the container health check polls. /api/v1/version is anonymous and returns the running version and git commit — the fastest way to confirm what is actually deployed. Both are described for programmatic use on API Access.

Where	What
forge-deploy --logs	Deploy history — what was deployed, when, and what happened
/var/log/forge-deploy.log	The same log on disk
/etc/forge	Deploy state: current and prior tag per component, deploy timestamps
forge-deploy compose logs -f <service>	Live container logs

Read the API's startup block first. On boot it emits a labelled database-lifecycle summary saying whether it found an existing schema or applied a fresh one. That single block separates a stale volume from a code bug faster than anything else, and it is the first thing to paste into an issue.

Optional profiles add a log viewer and a crash reporter if you want them; neither is required and neither is on by default.

Common failures

"relation … does not exist" on startup, right after an upgrade. The schema reconcile did not run. It is gated on a flag that ships off, and on a split install it reports success while doing nothing because the database is on another box. See Upgrades and Rollback — this is the single most common upgrade failure.

The app loads but real-time features are dead. Usually CORS. The policy allows credentials, so origins must be listed explicitly in CORS_ORIGINS — wildcards are not an option, and SignalR's negotiate checks the origin even on same-host paths. A missing entry shows up as broken live updates rather than an obvious error.

Unexpected 429s. You are not in Development. The rate limiter is a no-op in the shipped default environment and switches on with ASPNETCORE_ENVIRONMENT=Production, which is the setting you want in production — see Hardening a Production Install.

Everything works but you never entered credentials. The mock posture. Check the integration readiness report before believing any integration is live — Configuration and Integrations.

An upgrade fails immediately with a backup error. The pre-reconcile backup runs inside the backup sidecar and the deploy fails closed without it. If a container pruner removed that sidecar, restore it and retry — Backup and Restore.

A container is unhealthy but the app seems fine. Check which one: the composite health report degrades if object storage or Hangfire is unreachable even when the web UI still renders. forge-deploy --status names it.

When you need to file something

Include the running version from /api/v1/version, the API's startup database-lifecycle block, forge-deploy --status, and the relevant container log. File it in the repo that owns the component — see Home. Security problems go privately, not into a public issue — Contributing.