Improve self-hosting flow

[tyler-dane]

Context

Homelabbers running Compass on-premises have reported friction with the current self-hosting setup. The main pain points: no prebuilt images on Docker Hub (requires building from source), no semver pinning, healthcheck logic baked into the compose file instead of maintainer-controlled, install.sh doing too much (cloning the full repo just to get two files), containers running as root, and no official volume for backend log writes.

---

Files to Create or Modify

1. .github/workflows/publish-images.yml — CREATE

New GitHub Actions workflow triggered on v*.*.* tag push.

• Uses docker/login-action@v3, docker/setup-buildx-action@v3, docker/build-push-action@v6
• Strips v prefix from tag → bare semver (e.g. 1.2.3), also derives minor (1.2)
• Pushes three tags per image: X.Y.Z, X.Y, latest
• Builds and pushes:
  • switchbacktech/compass-backend from self-host/Dockerfile.backend
  • switchbacktech/compass-mongo from self-host/Dockerfile.mongo
  • switchbacktech/compass-web from self-host/Dockerfile.web with build-args:
    • BASEURL=http://localhost:3000/api (localhost default for local installs)
    • GOOGLE_CLIENT_ID=compass-self-host-placeholder.apps.googleusercontent.com
    • COMPASS_BUILD_REF=<bare-semver>
• Secrets required in GitHub repo settings (out-of-band): DOCKERHUB_USERNAME, DOCKERHUB_TOKEN
• Match existing workflow conventions: actions/checkout@v6, permissions: contents: read

Note on web build-args: BASEURL and GOOGLE_CLIENT_ID are baked into the web bundle at compile time. The published image ships with localhost defaults. Users who need custom values (e.g., a different API domain or real Google credentials) must rebuild the web image locally — the commented-out build: blocks in docker-compose.yml support this. This limitation is pre-existing and should be noted in the env-vars doc.

2. self-host/docker-compose.yml — REWRITE

Switch from local builds to Docker Hub images

Replace all build: blocks with image: switchbacktech/compass-<service>:${COMPASS_VERSION:-latest}. Keep the original build: blocks as commented-out alternatives for users who need local builds.

Healthchecks — no changes in this effort

Healthcheck removal is deferred to a separate cleanup effort because the mongo healthcheck also initializes the replica set on first boot. All existing healthcheck: blocks and condition: service_healthy depends_on conditions stay as-is.

Add volumes for writable paths

• compass_backend_logs:/app/logs — backend's only runtime write path (Winston logs)
• compass_mongo_configdb:/data/configdb — mongo entrypoint writes keyfile here; needs to be a named volume so it survives container recreation and works under read-only filesystem

Add both to the top-level volumes: block alongside the existing two.

Add read-only filesystem for backend and web

read_only: true
tmpfs:
  - /tmp   # backend only; Bun uses /tmp at runtime

Web container has zero runtime writes, so no tmpfs needed.
Mongo, supertokens, and supertokens-db are third-party images with opaque write paths — leave them without read_only.

Rename COMPASS_REF → COMPASS_VERSION

Replace ${COMPASS_REF:-main} with ${COMPASS_VERSION:-latest} in the web service build-args comment. Add COMPASS_VERSION to the env var pass-through on the backend service if needed.

3. self-host/Dockerfile.backend — MODIFY

Add a non-root compass user in the runtime stage:

RUN addgroup --system --gid 1001 compass \
 && adduser --system --uid 1001 --ingroup compass --no-create-home compass

COPY --from=build --chown=compass:compass /app/build/backend ./
RUN mkdir -p /app/logs && chown compass:compass /app/logs

USER compass

Linux capabilities needed: None. Port 3000 is above 1024 — no CAP_NET_BIND_SERVICE required. All chown calls happen at build time (as root during image build), not at container startup.

4. self-host/Dockerfile.web — MODIFY

Same non-root user pattern in the runtime stage:

RUN addgroup --system --gid 1001 compass \
 && adduser --system --uid 1001 --ingroup compass --no-create-home compass

COPY --from=build --chown=compass:compass /app/build/web ./build/web
COPY --from=build --chown=compass:compass /app/self-host/serve-web.ts ./self-host/serve-web.ts

USER compass

Linux capabilities needed: None. Port 9080 is above 1024.

5. self-host/mongo-entrypoint.sh — NO CHANGES

RS initialization stays in the mongo healthcheck (deferred cleanup). Only the new compass_mongo_configdb named volume mount makes /data/configdb writable — the entrypoint script itself doesn't change.

Note on Dockerfile.mongo: No changes needed. The official mongo:8.0 image already runs mongod as the mongodb user (uid 999). CAP_CHOWN is in Docker's default capability set — no --cap-add flags needed.

6. self-host/.env.example — MODIFY

• Rename COMPASS_REF=main → COMPASS_VERSION=latest
• Add # generate with: openssl rand -hex 32 comment on each secret field
• Ensure all var names stay in sync with the Zod schema in packages/backend/src/common/constants/env.constants.ts

7. self-host/install.sh — SIMPLIFY (~200 lines target, currently 947)

With Docker Hub images, the installer no longer needs to clone the repo or download a full archive. It only needs docker-compose.yml and a generated .env.

Remove entirely:

• download_with_git(), download_with_archive(), download_app()
• replace_app_dir(), backup_app(), restore_app_backup(), finish_app_replacement()
• All APP_DIR, TMP_APP, TMP_ARCHIVE_DIR, APP_BACKUP variables
• COMPASS_ARCHIVE_URL, COMPASS_REPO_URL, INSTALL_SOURCE logic
• validate_compass_ref() (git-ref validation no longer applies)

Replace with:

download_compose_file() {
  base_url="https://raw.githubusercontent.com/SwitchbackTech/compass/${COMPASS_VERSION}/self-host"
  curl -fsSL "${base_url}/docker-compose.yml" -o "$COMPOSE_FILE" \
    || fail "Could not download docker-compose.yml for version ${COMPASS_VERSION}."
}

Update directory layout:

~/compass/
  .env
  .compass-self-host    (marker)
  compass               (helper — now written inline via heredoc)
  docker-compose.yml    (downloaded directly, no app/ subdir)

Keep: secret generation, .env writing, port availability checks, Docker prerequisite check, health-poll loop for installer readiness gate (this is separate from Docker healthchecks — it's just the installer waiting for the stack to come up before printing success), the compass helper script (written inline via heredoc instead of copied from repo).

Update start_stack(): Use docker compose up -d (no --build).

8. self-host/compass — MODIFY

• Update COMPOSE_FILE from $APP_DIR/self-host/docker-compose.yml → $INSTALL_DIR/docker-compose.yml
• Rewrite update command:

  docker compose pull
  docker compose up -d

  Remove update_repo() and git-related logic entirely.
• Remove validate_compass_ref() and the INSTALL_SOURCE=git guard on update.

9. docs/Self-Hosting/environment-variables.md — CREATE

New reference doc grouping all env vars by category. For each var: name, required/optional/build-time, default, purpose, valid values (where constrained).

Sections:

• Compose Metadata (COMPOSE_PROJECT_NAME, COMPASS_VERSION)
• Port Bindings (WEB_PORT, PORT)
• Runtime Behavior (NODE_ENV, TZ, LOG_LEVEL)
• URLs (FRONTEND_URL, BASEURL [build-time note], CORS, GCAL_WEBHOOK_BASEURL)
• MongoDB (MONGO_INITDB_ROOT_USERNAME, MONGO_INITDB_ROOT_PASSWORD, MONGO_REPLICA_SET_KEY, MONGO_URI)
• SuperTokens Postgres (SUPERTOKENS_POSTGRES_USER, SUPERTOKENS_POSTGRES_PASSWORD, SUPERTOKENS_POSTGRES_DB)
• SuperTokens Core (SUPERTOKENS_URI, SUPERTOKENS_KEY)
• Internal Tokens (TOKEN_COMPASS_SYNC, TOKEN_GCAL_NOTIFICATION)
• Google OAuth (GOOGLE_CLIENT_ID [build-time note], GOOGLE_CLIENT_SECRET, CHANNEL_EXPIRATION_MIN)

Include: a note that BASEURL and GOOGLE_CLIENT_ID are baked into the web image at build time, so changing them after pulling from Docker Hub requires rebuilding the web image locally (see the commented-out build: blocks in docker-compose.yml).

10. docs/Self-Hosting/README.md — MODIFY

Add link to the new environment-variables.md in the navigation section. Add a note about the health endpoint (so maintainers know it exists regardless of whether Docker healthchecks are enabled):

Health endpoint: GET /api/health returns {"status":"ok","timestamp":"..."} (200) or {"status":"error","timestamp":"..."} (500) based on MongoDB connectivity. No authentication required. Useful for monitoring probes, load balancer checks, or readiness scripts.

---

Build Sequence

1. Dockerfiles — Add non-root users to Dockerfile.backend and Dockerfile.web. Verify docker build succeeds for both from repo root.
2. docker-compose.yml — Switch to Hub images, add compass_backend_logs and compass_mongo_configdb volumes, add read_only: true + tmpfs on backend and web.
3. .env.example — Rename COMPASS_REF → COMPASS_VERSION, add secret generation comments.
4. install.sh — Simplify: remove git/archive download, add download_compose_file().
5. compass helper — Update COMPOSE_FILE path, rewrite update command.
6. Documentation — Create environment-variables.md, update README.md.
7. GitHub Actions — Create publish-images.yml. Add DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets in GitHub repo settings (manual step).

---

Verification

• docker build -f self-host/Dockerfile.backend . && docker build -f self-host/Dockerfile.web . — both succeed; docker inspect shows user is compass (uid 1001), not root
• Fresh docker compose up -d with only docker-compose.yml + .env (no repo clone) — all services start and become healthy
• curl http://localhost:3000/api/health returns {"status":"ok",...}
• docker exec compass-backend-1 touch /test fails (read-only filesystem), docker exec compass-backend-1 touch /app/logs/test succeeds (volume is writable)
• Stop and restart the stack — mongo comes up with RS already initialized (state persists in named volume)
• ./compass update — pulls new images and restarts cleanly
• Push a test v0.0.1-test tag and verify all three images appear on Docker Hub with :0.0.1-test, :0.0, and :latest tags

Use Case

This'll make setting up compass on-prem easier

Additional Context

No response

[tyler-dane]

How does the new GitHub Action affect day-to-day dev?

It doesn't — zero friction. The workflow only fires when a maintainer pushes a semver git tag (e.g. git tag v1.2.3 && git push origin v1.2.3). Normal PRs, branch pushes, and commits never trigger it. The only new step in a release flow is "push a tag." Developers working on features don't touch it at all.

Why three tags per image?

This is the standard Docker Hub convention for semver and it directly fulfills the homelabber feedback about "easier Docker updates":

Tag	Meaning	Who uses it
1.2.3	Exact pin	Users who want reproducible, auditable deploys
1.2	Minor-pinned	Users who want patch fixes automatically but not minor changes
latest	Always newest	Users who just want the current stable release

A user pinned to 1.2 in their .env (COMPASS_VERSION=1.2) gets 1.2.1, 1.2.2, etc. automatically via ./compass update without having to edit their config file. Without the minor tag they'd have to manually change 1.2.0 → 1.2.1 every time.

Docker Hub cost impact?

Docker Hub's free tier allows unlimited pulls for public images and unlimited public repositories. switchbacktech/compass-* would be public. No cost for pulls or storage of images — Docker Hub only charges for private repos or very high-volume commercial usage. Pushing images from GitHub Actions also costs nothing on the Docker Hub side (it counts against GitHub Actions minutes, which the existing workflows already use). Net cost: $0 change.

Why a non-root compass user?

The homelabber feedback explicitly listed this: "Create a non-root user to run your app otherwise security becomes cumbersome." Here's why it matters in practice:

• Container escape mitigation: If a remote code execution vulnerability exists in the app, the attacker gets a user with no privileges (uid 1001, no sudo, no capabilities) instead of root. Root inside a container can still cause real damage — mounting host paths, accessing other containers via the Docker socket if it's exposed, or exploiting kernel vulnerabilities.
• Host filesystem protection: The read_only: true container setting (also in the plan) requires non-root to be effective — a root process can bypass many read-only restrictions.
• It's a Docker best practice: Every major Docker security guide (CIS Benchmark, Docker official docs, OWASP Container Security) lists "don't run as root" as a baseline requirement. Homelabbers who harden their containers (AppArmor, seccomp, capability dropping) expect this.

The complexity cost is minimal: four lines in each Dockerfile (addgroup, adduser, chown, USER). The resulting images are functionally identical for end users.

[tyler-dane]

Cleanup ENV values and naming:

• CORS in the .env is renamed to ORIGINS_ALLOWED inside the backend — the schema splits it on commas, so the env key and the code name differ.
• TZ. Is this necessary to have as an ENV? The backend will only work with it as utc, so why even expose it?
• EMAILER_ key removal is part of Replace EMAILER envs and code with background job #1730