Docker and Local Development

Canonical source: docs/08-runtime-devops/docker-guide.md — this page is rendered from it on every publish; edit the source.

Docker Development Guide

Overview

ClawAI runs as a fully containerized development environment using split Docker Compose files. The complete profile contains 21 backend services, their owned databases, shared infrastructure, a reverse proxy, the frontend, and optional local-AI runtimes.

All containers share the claw-network bridge network and read environment variables from a single root .env file.

---

Container Inventory

Infrastructure

Container	Image	Host Port	Internal Port	Purpose
claw-pg-auth	pgvector/pgvector:pg16	5441	5432	PostgreSQL for auth service
claw-pg-chat	pgvector/pgvector:pg16	5442	5432	PostgreSQL for chat service
claw-pg-connector	pgvector/pgvector:pg16	5443	5432	PostgreSQL for connector service
claw-pg-routing	pgvector/pgvector:pg16	5444	5432	PostgreSQL for routing service
claw-pg-memory	pgvector/pgvector:pg16	5445	5432	PostgreSQL for memory service
claw-pg-files	pgvector/pgvector:pg16	5446	5432	PostgreSQL for file service
claw-pg-ollama	pgvector/pgvector:pg16	5447	5432	PostgreSQL for ollama service
claw-pg-images	pgvector/pgvector:pg16	5448	5432	PostgreSQL for image service
claw-pg-file-generations	pgvector/pgvector:pg16	5449	5432	PostgreSQL for file generation service
claw-pg-workspace	pgvector/pgvector:pg16	5450	5432	PostgreSQL for workspace service
claw-pg-agent	pgvector/pgvector:pg16	5451	5432	PostgreSQL for agent service
claw-pg-research	pgvector/pgvector:pg16	5452	5432	PostgreSQL for research service
claw-pg-payments	pgvector/pgvector:pg16	5453	5432	PostgreSQL for payment service
claw-pg-llamacpp	pgvector/pgvector:pg16	5454	5432	PostgreSQL for llama.cpp service
claw-pg-thread-generation	pgvector/pgvector:pg16	5457	5432	PostgreSQL for thread generation
claw-pg-threads	pgvector/pgvector:pg16	5458	5432	PostgreSQL for threads service
claw-pg-plugin	pgvector/pgvector:pg16	5459	5432	PostgreSQL for plugin service
claw-mongodb	mongo:7	27018	27017	MongoDB for audit, client/server logs
claw-redis	redis:7-alpine	6380	6379	Redis cache and session store
claw-rabbitmq	rabbitmq:3-management-alpine	5672/15672	5672/15672	Message broker with management UI
claw-ollama	ollama/ollama:latest	11434	11434	Local AI runtime (LLM inference)

Backend Services (21 containers)

Container	Dockerfile	Host Port	Purpose
claw-auth-service	apps/claw-auth-service/Dockerfile.dev	4001	Authentication, users, RBAC
claw-chat-service	apps/claw-chat-service/Dockerfile.dev	4002	Threads, messages, AI execution
claw-connector-service	apps/claw-connector-service/Dockerfile.dev	4003	Cloud provider management
claw-routing-service	apps/claw-routing-service/Dockerfile.dev	4004	Intelligent model routing
claw-memory-service	apps/claw-memory-service/Dockerfile.dev	4005	Memory and context packs
claw-file-service	apps/claw-file-service/Dockerfile.dev	4006	File upload and chunking
claw-audit-service	apps/claw-audit-service/Dockerfile.dev	4007	Audit logs and usage tracking
claw-ollama-service	apps/claw-ollama-service/Dockerfile.dev	4008	Ollama model management
claw-health-service	apps/claw-health-service/Dockerfile.dev	4009	Aggregated health checks
claw-client-logs-service	apps/claw-client-logs-service/Dockerfile.dev	4010	Frontend log ingestion
claw-server-logs-service	apps/claw-server-logs-service/Dockerfile.dev	4011	Backend log aggregation
claw-image-service	apps/claw-image-service/Dockerfile.dev	4012	AI image generation
claw-file-generation-service	apps/claw-file-generation-service/Dockerfile.dev	4013	AI file generation
claw-workspace-service	apps/claw-workspace-service/Dockerfile.dev	4014	Workspace connectors and actions
claw-agent-service	apps/claw-agent-service/Dockerfile.dev	4015	Desktop agent
claw-research-service	apps/claw-research-service/Dockerfile.dev	4016	Research orchestration
claw-llamacpp-service	apps/claw-llamacpp-service/Dockerfile.dev	4017	Local llama.cpp runtime
claw-payment-service	apps/claw-payment-service/Dockerfile.dev	4018	Subscriptions and payments
claw-threads-service	apps/claw-threads-service/Dockerfile.dev	4019	Public Thread publications
claw-thread-generation-service	apps/claw-thread-generation-service/Dockerfile.dev	4020	Durable Thread generation jobs
claw-plugin-service	apps/claw-plugin-service/Dockerfile.dev	4021	Plugin and MCP catalog

Proxy and Frontend (2 containers)

Container	Image	Host Port	Purpose
claw-nginx	nginx:alpine	4000	Reverse proxy routing to all services
claw-frontend	Custom (Next.js)	3000	Web UI (Turbopack hot reload)

---

Starting and Stopping

Full Stack (Single Compose File)

# Start everything
./scripts/claw.sh up -d

# Stop everything
./scripts/claw.sh down

# Stop and remove volumes (DESTRUCTIVE - deletes all data)
./scripts/claw.sh down -v

Using the Management Script

The scripts/claw.sh script provides granular control with separate compose files for databases, services, and Ollama:

# Start all (dev mode, default)
./scripts/claw.sh up

# Stop all
./scripts/claw.sh down

# Check status
./scripts/claw.sh status

# Production mode
./scripts/claw.sh --prod up

Starting Individual Services

# Rebuild and start a single service
./scripts/claw.sh service:rebuild chat-service

# Start only infrastructure (databases, Redis, RabbitMQ)
./scripts/claw.sh up -d pg-auth pg-chat pg-connector pg-routing pg-memory pg-files pg-ollama pg-images pg-file-generations mongodb redis rabbitmq

# Restart a single service
./scripts/claw.sh restart auth-service

---

Container Dependencies and Startup Order

Docker Compose health checks enforce the correct startup order:

Layer 1 (no dependencies):
  pg-auth, pg-chat, pg-connector, pg-routing, pg-memory, pg-files,
  pg-ollama, pg-images, pg-file-generations, mongodb, redis, rabbitmq, ollama

Layer 2 (depends on Layer 1 being healthy):
  auth-service      → pg-auth, redis, rabbitmq
  chat-service      → pg-chat, redis, rabbitmq
  connector-service → pg-connector, redis, rabbitmq
  routing-service   → pg-routing, redis, rabbitmq
  memory-service    → pg-memory, redis, rabbitmq
  file-service      → pg-files, redis, rabbitmq
  audit-service     → mongodb, redis, rabbitmq
  ollama-service    → pg-ollama, redis, rabbitmq
  image-service     → pg-images, redis, rabbitmq
  client-logs-service → mongodb, redis, rabbitmq
  server-logs-service → mongodb, redis, rabbitmq
  health-service    → redis, rabbitmq

Layer 3 (depends on all services being healthy):
  nginx → all 21 backend services

Layer 4:
  frontend → nginx (service_started, not health check)

Each PostgreSQL service uses pg_isready health checks (10s interval, 5 retries). Backend services use wget against their /api/v1/health endpoint (15s interval, 10 retries, 45s start period). RabbitMQ uses rabbitmq-diagnostics check_running. Ollama uses ollama list with a 30s start period to allow model auto-pulling.

---

Volume Management

Named Volumes

Volume	Container	Mount Point	Purpose
pg-auth-data	claw-pg-auth	/var/lib/postgresql/data	Auth database files
pg-chat-data	claw-pg-chat	/var/lib/postgresql/data	Chat database files
pg-connector-data	claw-pg-connector	/var/lib/postgresql/data	Connector database files
pg-routing-data	claw-pg-routing	/var/lib/postgresql/data	Routing database files
pg-memory-data	claw-pg-memory	/var/lib/postgresql/data	Memory database files
pg-files-data	claw-pg-files	/var/lib/postgresql/data	Files database files
pg-ollama-data	claw-pg-ollama	/var/lib/postgresql/data	Ollama database files
pg-images-data	claw-pg-images	/var/lib/postgresql/data	Images database files
pg-file-generations-data	claw-pg-file-generations	/var/lib/postgresql/data	File generations DB files
mongo-data	claw-mongodb	/data/db	MongoDB data files
redis-data	claw-redis	/data	Redis persistence
rabbitmq-data	claw-rabbitmq	/var/lib/rabbitmq	RabbitMQ state and queues
ollama-data	claw-ollama	/root/.ollama	Downloaded AI models
file-storage-data	claw-file-service	/data/files	Uploaded user files

Host-Mounted Volumes (Hot Reload)

Backend services mount their src/ and prisma/ directories from the host for hot reload:

./apps/claw-<service>/src    → /app/apps/claw-<service>/src
./apps/claw-<service>/prisma → /app/apps/claw-<service>/prisma

The frontend mounts src/ and public/:

./apps/claw-frontend/src    → /app/src
./apps/claw-frontend/public → /app/public

Nginx mounts its config read-only:

./infra/nginx/nginx.conf → /etc/nginx/nginx.conf:ro

Cleaning Up Volumes

# List all ClawAI volumes
docker volume ls | grep claw

# Remove all volumes (DESTRUCTIVE)
./scripts/claw.sh down -v

# Remove a specific volume
docker volume rm docker-compose-dev_pg-auth-data

---

Hot Reload Matrix

Change	Action Required	Downtime
Source code (src/)	Auto-detected by node --watch (backend) or Turbopack (frontend)	None
Prisma schema	Rebuild container (prisma migrate deploy runs in entrypoint)	~30s
package.json deps	Rebuild container (docker compose up -d --build <service>)	~60s
Docker Compose config	./scripts/claw.sh up -d (recreate)	~10s
.env values	Restart containers (docker compose restart <service>)	~5s
Shared packages	Rebuild shared package, then restart dependent services	~30s
Nginx config	Restart nginx (docker compose restart nginx)	~2s
Dockerfile changes	Rebuild (docker compose up -d --build <service>)	~60s

How Hot Reload Works

Backend services using Prisma (auth, chat, connector, routing, memory, file, ollama, image) run docker-entrypoint.dev.sh which executes prisma migrate deploy then starts the app with node --watch. Source file changes trigger automatic restart.

Backend services without Prisma (audit, health, client-logs, server-logs) run npm run dev directly.

Frontend runs npm run dev with Next.js Turbopack, providing instant HMR (Hot Module Replacement) for React components, styles, and pages.

---

Viewing Logs

# Follow logs for a specific service
./scripts/claw.sh logs -f chat-service

# Follow logs for multiple services
./scripts/claw.sh logs -f auth-service chat-service routing-service

# Last 100 lines from a service
./scripts/claw.sh logs --tail=100 ollama-service

# All service logs (very verbose)
./scripts/claw.sh logs -f

# View logs for infrastructure
./scripts/claw.sh logs -f rabbitmq
./scripts/claw.sh logs -f pg-auth

# Using docker directly
docker logs claw-chat-service --tail=50 -f

---

Rebuilding Containers

# Rebuild a single service (no cache)
./scripts/claw.sh build --no-cache auth-service
./scripts/claw.sh up -d auth-service

# Rebuild and restart in one command
./scripts/claw.sh service:rebuild auth-service

# Rebuild all services
./scripts/claw.sh build --no-cache
./scripts/claw.sh up -d

# Force recreate without rebuild (useful for env changes)
./scripts/claw.sh up -d --force-recreate auth-service

---

Network Topology

All containers are connected to the claw-network bridge network. Services communicate using Docker DNS hostnames that match the service name in docker/docker-compose.dev.services.yml / docker/docker-compose.dev.databases.yml:

claw-network (bridge)
  |
  +-- Infrastructure
  |     pg-auth:5432, pg-chat:5432, pg-connector:5432, pg-routing:5432
  |     pg-memory:5432, pg-files:5432, pg-ollama:5432, pg-images:5432
  |     pg-file-generations:5432, pg-workspace:5432, pg-agent:5432
  |     pg-research:5432, pg-llamacpp:5432, pg-payments:5432
  |     pg-threads:5432, pg-thread-generation:5432, pg-plugin:5432
  |     mongodb:27017, redis:6379, rabbitmq:5672
  |     ollama:11434
  |
  +-- Backend Services
  |     auth-service:4001, chat-service:4002, connector-service:4003
  |     routing-service:4004, memory-service:4005, file-service:4006
  |     audit-service:4007, ollama-service:4008, health-service:4009
  |     client-logs-service:4010, server-logs-service:4011
  |     image-service:4012, file-generation-service:4013
  |     workspace-service:4014, agent-service:4015, research-service:4016
  |     llamacpp-service:4017, payment-service:4018
  |     threads-service:4019, thread-generation-service:4020
  |     plugin-service:4021
  |
  +-- Proxy + Frontend
        nginx:80 (host 4000), frontend:3000

Nginx resolves service hostnames using Docker's internal DNS (127.0.0.11) with 5s TTL, allowing it to survive service restarts without becoming stale.

Inter-service HTTP calls use the *_SERVICE_URL environment variables (e.g., CONNECTOR_SERVICE_URL=http://connector-service:4003).

Local and distributed nginx both expose payment webhooks, customer payments, customer billing, and admin billing. Distributed deployments set CLAW_PAYMENT_ORIGIN; neither configuration exposes /api/v1/internal/payments or any other internal payment contract.

Scraping sidecars (claw-scrapers network, ADR-121)

research-service's fetch escalation chain can use three optional sidecars, each behind its own compose profile in docker-compose.{dev,prod}.services.yml: crawl4ai (~4 GB), flaresolverr (~1.5 GB) and firecrawl (API + Playwright service + private Redis/RabbitMQ/Postgres, 8-12 GB). CLAW_SCRAPER_PROFILES in .env (e.g. crawl4ai,flaresolverr) decides which exist; scripts/claw.sh adds them to COMPOSE_PROFILES, and down/status always include all three. They publish no port, have no nginx route, and sit only on the compose-managed claw-scrapers bridge — research-service is the one ClawAI container on both networks, so a sidecar cannot reach any other service or database. A running sidecar is still unused until its tier is enabled DB-level (PATCH /api/v1/research/fetch-strategies/<KIND>); see skills/add-a-fetch-strategy.md. crawl4ai also needs CRAWL4AI_API_TOKEN in .env: without it Crawl4AI 0.9.x binds only its own loopback, so research-service gets ECONNREFUSED while the healthcheck (which curls localhost inside the container) still reports healthy.

---

Native tools inside service images (ffmpeg, multimodal batch 7)

claw-file-service installs Debian's ffmpeg (which provides ffprobe) in BOTH Dockerfile.dev and the Dockerfile runner stage, with a build-time ffmpeg -version && ffprobe -version check so a missing binary fails the build rather than the first video upload. It is used for video probe, audio-track extraction, the thumbnail and on-demand frames — always through common/utilities/media-process.utility.ts (argument arrays, shell: false, SIGKILL budget, protocol/format whitelists, per-job temp dir under os.tmpdir(), removed in finally). No env var: the binaries are on PATH. Adding it grows the image by roughly 100 MB of shared libraries. An old container reports TOOL_UNAVAILABLE on every video until it is rebuilt (a restart keeps the old image). Verify:

docker exec claw-file-service sh -c "ffmpeg -version | head -1; ffprobe -version | head -1"

Published service ports (production)

In docker/docker-compose.prod.services.yml every backend service port (4001-4021) is published on 127.0.0.1 only ('127.0.0.1:4003:4003'), so the internet cannot reach an internal route such as /api/v1/internal/*. Services reach each other by container name on the Docker network, health checks run inside the container, and nginx proxies by container name, so nothing needs the host port. Only nginx (80, 443, 4000) and the frontend (3000) stay public. Host-side tools (curl, Playwright, smoke scripts) still work on the server itself; from another machine use an SSH tunnel. The dev compose files are unchanged. In production a placeholder INTER_SERVICE_AUTH_TOKEN (anything containing change-me) now stops the service at boot.

Resource Requirements

Minimum Development Requirements

Resource	Minimum	Recommended
RAM	12 GB	16 GB
CPU	4 cores	8 cores
Disk	20 GB	40 GB

Per-Component Estimates

Component	RAM Usage	Disk Usage
13 PostgreSQL (idle)	~1.8 GB	~500 MB
MongoDB	~300 MB	~100 MB
Redis	~50 MB	~10 MB
RabbitMQ	~200 MB	~50 MB
Ollama (5 models)	~2-6 GB	~10 GB
21 NestJS services	~2.6 GB	~500 MB
Nginx	~10 MB	~5 MB
Next.js frontend	~500 MB	~300 MB

Ollama is the largest consumer. Model sizes: gemma3:4b (3.3 GB), llama3.2:3b (2.0 GB), phi3:mini (2.2 GB), gemma2:2b (1.6 GB), tinyllama (637 MB). Models are pulled on first startup and cached in the ollama-data volume.

---

Common Docker Issues and Solutions

Port Conflicts

Symptom: Bind for 0.0.0.0:XXXX failed: port is already allocated

Solution: Check what is using the port and either stop it or change the port in .env:

# Find process using a port (Linux/macOS)
lsof -i :4001

# Windows
netstat -ano | findstr :4001

Container Restart Loop

Symptom: A service container keeps restarting (status shows Restarting).

Solution:

1. Check logs: docker logs claw-<service> --tail=100
2. Common causes: database not ready, missing env vars, Prisma migration failure
3. Ensure infrastructure is healthy: ./scripts/claw.sh ps
4. Try rebuilding: ./scripts/claw.sh service:rebuild <service>

Health Check Failures

Symptom: Nginx or dependent services fail to start because upstream services are unhealthy.

Solution:

1. Check which services are unhealthy: ./scripts/claw.sh ps
2. Inspect health check output: docker inspect --format='{{json .State.Health}}' claw-auth-service
3. Start period is 45s for services; wait for initial startup to complete

Ollama Model Pull Timeout

Symptom: Ollama container stays unhealthy for a long time on first run.

Solution: This is expected on first startup. The 5 models total ~10 GB and may take 10-30 minutes to download. Monitor progress:

docker logs claw-ollama -f

Database Volume Corruption

Symptom: PostgreSQL container fails with "database system was not properly shut down".

Solution:

# Stop the container
./scripts/claw.sh stop pg-auth

# Remove the volume
docker volume rm docker-compose-dev_pg-auth-data

# Restart (will recreate from scratch; run prisma migrate)
./scripts/claw.sh up -d pg-auth auth-service

Out of Disk Space

Symptom: Containers fail to start or build, Docker daemon errors.

Solution:

# Check Docker disk usage
docker system df

# Remove unused images, containers, and build cache
docker system prune -a

# Remove dangling volumes
docker volume prune

Shared Package Changes Not Reflected

Symptom: A service does not pick up changes made in packages/shared-*.

Solution: Shared packages are copied at build time, not mounted. Rebuild the affected service:

./scripts/claw.sh service:rebuild <service>

Nest can't resolve dependencies of ... RabbitMQService

Cause: A consumer injects RabbitMQService, but the service's AppModule does not configure RabbitMQModule.forRootAsync. Add the root module using the service's RABBITMQ_URL from .env; the variable is shared across services and is already declared in .env.example.

ERR_MODULE_NOT_FOUND for packages/shared-*/dist/<file> (no .js extension)

Symptom: A dev container crashes at boot with Cannot find module '/app/packages/shared-<x>/dist/<file>' imported from /app/packages/shared-<x>/dist/index.js, while prod and CI are fine.

If npm run build reports Cannot find module '@claw/shared-<x>' inside a service image instead, check that the workspace is declared in the service package.json and built in both of that service's Dockerfiles.

Cause: Root .dockerignore copies the host's packages/*/dist into the image on purpose, and that service's Dockerfile.dev never runs npm run build for shared-<x>. The container is running whatever stale dist the host had — here one emitted before tsc-alias -f added .js extensions, which ESM refuses to resolve. Rebuilding the image without fixing the Dockerfile copies the same stale dist again.

Fix: add RUN cd packages/shared-<x> && npm run build to that service's Dockerfile.dev (and Dockerfile if missing), then the full stop → rm → rmi → service:rebuild cycle. tools/__tests__/dockerfile-shared-package-completeness.test.mjs fails on the gap; see rule 14 §6 and add-a-shared-package-to-a-service. Hit 2026-09-02 on payment-service; nine Dockerfile.dev files had the gap.

After pulling that fix: claw.sh up reuses existing images, so the nine stale dev images stay stale until rebuilt. One invocation, safe in parallel:

./scripts/claw.sh service:rebuild audit-service connector-service file-service llamacpp-service memory-service ollama-service payment-service routing-service server-logs-service

A dev container compiling new source against an OLD build config

Symptom: one service is unhealthy for days while everything else is fine. Its log ends in nodemon app crashed, and above that a wall of TS2307: Cannot find module '../../../common/enums/…' on imports that plainly exist, often mixed with TS2834: Relative import paths need explicit file extensions … when '--moduleResolution' is 'node16'. git log shows nobody touched those files.

Cause: the dev compose mounted src, package.json and prisma — and not tsconfig.json, tsconfig.build.json or tools/. So a dev container recompiles today's source using the build config baked into its image. When the repo moved every workspace to moduleResolution: "bundler", services whose images were rebuilt afterwards picked it up and services whose images were not kept compiling bundler-style extensionless imports under Node16, where they are illegal. The same gap hides a second failure one step later: Cannot find module '/app/tools/typescript/copy-generated-prisma.mjs', a build script added after the image was built.

Confirm it in one command — the container's config, not the host's:

docker exec claw-<name> sh -c "grep moduleResolution /app/apps/claw-<name>/tsconfig.json"
docker exec claw-<name> sh -c "ls /app/tools/typescript/"

If that disagrees with the file on the host, this is it.

Fix: docker/docker-compose.dev.services.yml now mounts tsconfig.json, tsconfig.build.json and ../tools read-only into every service, so the build config and the build scripts come from the working tree like the source does. Recreating the container is enough — no rebuild:

./scripts/claw.sh up -d claw-<name>

Hit 2026-09-10: research-service had been down 22 hours and ollama-service and llamacpp-service were down with the same cause, all three on 1.63.0 images. /research/* returned 502 the whole time, which silently disabled every web search and fetch in chat. A crash-looping dev container is a config-drift suspect before it is a code suspect — check the container's config against the host's before reading the source.

Root overrides must equal the workspace pin (react-hook-form, 2026-10-03)

A dependency pinned in overrides (root package.json) at a different version than the workspace that uses it (react-hook-form 7.88.0 vs 7.89.0) leaves a stale nested copy in package-lock.json (apps/claw-frontend/node_modules/react-hook-form). The dev image then has the package only under the app, so a hoisted dependency such as @hookform/resolvers fails with Can't resolve 'react-hook-form' and /login returns 500. Keep the override and the workspace pin identical and delete the nested lock entry. Never fix it with a symlink inside the container.