Deployment and Operations

Deployment and Operations

Compose services

The reference stack includes:

• PostgreSQL for Dagster storage;
• Prometheus Pushgateway;
• Slackpipe gRPC code server;
• optional Slackquery gRPC code server;
• Dagster webserver and daemon;
• a workspace-file reload watcher.

Ports bind to loopback by default in public examples. Use an authenticated reverse proxy or trusted private network for remote access.

Start

cp .env.example .env
cp deployment.env.example deployment.env
chmod 600 .env deployment.env
docker compose --env-file deployment.env up -d --build

Review code-location load status before launching ingestion.

First run

1. Launch <workspace_slug>_ingest_once.
2. Watch extraction logs and storage consumption.
3. Verify all blocking checks.
4. Inspect canonical row counts and latest timestamps.
5. Start recurring schedules deliberately.

Routine response

Start with Dagster instance health, code locations, active/queued runs, schedule and sensor ticks, then asset checks. For extraction errors, classify auth, network, Slackdump, lock, disk, and source-integrity failures separately.

Metrics

Slackpipe pushes latest-run gauges grouped by bounded workspace/stage labels. Metrics include success, completion lag, source/canonical counts, mutations, deletions, error state, and selected timestamp observations.

Upgrade discipline

• Pin and checksum Slackdump releases in the image.
• Test schema and orchestration changes before deployment.
• Stop schedules for destructive maintenance.
• Preserve the previous image and data backups.
• Re-run blocking checks after code-location reload.

See the repository's docs/operations.md for the concise runbook.