Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,13 @@ eslint-typegen.d.ts

# Workspace package build output
packages/runtime/dist/
packages/gateway/dist/
apps/action/dist/

# BEGIN oh-my-opencode-slim clonedeps
.slim/clonedeps/repos/
# END oh-my-opencode-slim clonedeps

# Deploy stack — secrets and local overrides
deploy/secrets/
deploy/compose.override.yaml
142 changes: 142 additions & 0 deletions deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# fro-bot Deploy Stack

Docker Compose v2 stack for the fro-bot gateway. Runs three services:

| Service | Role |
| ----------- | --------------------------------------------------------------------------------- |
| `gateway` | Discord gateway daemon — connects to Discord, handles slash commands and mentions |
| `workspace` | Workspace agent container (placeholder in v1; real agent wired in Unit 7) |
| `mitmproxy` | Egress proxy enforcing an allowlist of permitted outbound hosts |

## Prerequisites

- Docker 24+ with Compose v2 (`docker compose version`)
- Access to a Discord application (bot token + application ID)
- An S3-compatible object store (bucket, region, optional endpoint)

## One-Time Setup

### 1. Copy the override example

```bash
cp deploy/compose.override.example.yaml deploy/compose.override.yaml
# Edit compose.override.yaml for your environment (e.g. expose mitmproxy web UI in dev)
```

`compose.override.yaml` is gitignored — never commit it.

### 2. Create secrets

Create one file per secret under `deploy/secrets/`. Files must be readable only by the owner (`chmod 0600`).

```bash
mkdir -p deploy/secrets
echo -n 'YOUR_DISCORD_BOT_TOKEN' > deploy/secrets/discord-token
echo -n 'YOUR_DISCORD_APP_ID' > deploy/secrets/discord-application-id
echo -n 'your-s3-bucket-name' > deploy/secrets/s3-bucket
echo -n 'us-east-1' > deploy/secrets/s3-region
# Optional — omit for AWS S3; set for R2 or other S3-compatible stores:
echo -n 'https://your-endpoint.r2.dev' > deploy/secrets/s3-endpoint

chmod 0600 deploy/secrets/*
```

`deploy/secrets/` is gitignored — never commit secret files.

### 3. Bootstrap the mitmproxy CA

Run once to generate the mitmproxy CA and place it in the shared Docker volume:

```bash
bash deploy/init-certs.sh
```

This is idempotent — safe to run again; skips if the CA already exists.

## Starting the Stack

```bash
docker compose -f deploy/compose.yaml -f deploy/compose.override.yaml up -d
```

Or without an override file:

```bash
docker compose -f deploy/compose.yaml up -d
```

## Viewing Logs

```bash
# Follow gateway logs
docker compose -f deploy/compose.yaml logs -f gateway

# Follow all services
docker compose -f deploy/compose.yaml logs -f
```

## Validating the Stack

After `up -d`, run the smoke-test script:

```bash
bash deploy/validate-stack.sh
```

This checks:

- Compose YAML is valid
- Service status
- Recent log output
- Gateway exit code (fails if gateway crashed in the last cycle)

## Stopping the Stack

```bash
docker compose -f deploy/compose.yaml down
```

## mitmproxy CA Cert (Dev Only)

If you want your host browser or tools to trust the mitmproxy CA (useful for inspecting proxied traffic in dev), extract and install it:

```bash
# Extract the cert from the Docker volume
docker run --rm \
-v fro-bot_mitmproxy-certs:/certs \
alpine cat /certs/mitmproxy-ca-cert.pem \
| sudo tee /usr/local/share/ca-certificates/mitmproxy-fro-bot.crt

# Install (Linux)
sudo update-ca-certificates

# Install (macOS)
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain \
/usr/local/share/ca-certificates/mitmproxy-fro-bot.crt
```

**Do not install the mitmproxy CA on production hosts.** It is only needed on developer machines that want to inspect proxied traffic via the mitmproxy web UI.

## Egress Allowlist

The mitmproxy addon at `deploy/mitmproxy/allowlist.py` enforces a static allowlist of permitted outbound hosts. Changes require restarting the mitmproxy container. The allowlist covers:

- GitHub API + raw content
- npm registry
- Discord API + gateway
- LLM providers (Anthropic, OpenAI, Google)

Any host not on the list receives a 403 and the connection is dropped. Both HTTPS CONNECT tunnels and plain HTTP requests are enforced.

### Object-store bucket scoping

The allowlist does **not** include broad S3/R2 wildcards (`*.s3.amazonaws.com`, `*.r2.cloudflarestorage.com`). Instead, set the `OBJECT_STORE_HOSTS` environment variable on the `mitmproxy` service to the exact bucket host(s) your deployment uses:

```
OBJECT_STORE_HOSTS=my-bucket.s3.amazonaws.com,my-account.r2.cloudflarestorage.com
```

If `OBJECT_STORE_HOSTS` is unset or empty, all S3/R2 traffic is blocked (fail-closed default). This prevents workspace processes from exfiltrating data to attacker-controlled buckets in those clouds.

Set the variable in your `.env` file or `compose.override.yaml` (see `compose.override.example.yaml` for an example).
39 changes: 39 additions & 0 deletions deploy/compose.override.example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# compose.override.example.yaml
#
# Copy this file to compose.override.yaml and edit for your local dev environment.
# compose.override.yaml is gitignored — never commit it.
#
# Usage:
# docker compose -f compose.yaml -f compose.override.yaml up -d
#
# This file is intentionally minimal. Only override what differs from compose.yaml.

services:
mitmproxy:
# Expose the mitmproxy web UI on the host for dev inspection.
# Remove or comment out in production.
ports:
- '127.0.0.1:8081:8081'
command: >
mitmdump
-s /scripts/allowlist.py
--listen-host 0.0.0.0
--listen-port 8080
--web-host 0.0.0.0
--web-port 8081
--set confdir=/home/mitmproxy/.mitmproxy
--set ssl_insecure=false
# environment:
# OBJECT_STORE_HOSTS is intentionally omitted here so it falls through to
# compose.yaml's interpolation: OBJECT_STORE_HOSTS: ${OBJECT_STORE_HOSTS:-}
# Set it in your .env file or shell environment instead.
# Example: "my-bucket.s3.amazonaws.com,my-account.r2.cloudflarestorage.com"
# If unset or empty, all S3/R2 traffic is blocked (fail-closed default).

gateway:
# Override the mitmproxy-certs volume path for dev if you want a local dir
# instead of the named volume (useful for inspecting the CA cert directly).
# volumes:
# - ./dev-certs:/etc/ssl/certs:ro
environment:
LOG_LEVEL: debug
108 changes: 108 additions & 0 deletions deploy/compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
name: fro-bot

services:
gateway:
build:
context: ..
dockerfile: deploy/gateway.Dockerfile
depends_on:
# Wait until mitmproxy has actually written the CA cert into the shared
# volume, not just until its container has started. Without this the
# gateway can launch with NODE_EXTRA_CA_CERTS pointing at a missing
# file — Node silently ignores it and the egress allowlist is bypassed
# for any request the gateway makes during the boot window.
mitmproxy:
condition: service_healthy
environment:
# Secret file paths (read via readSecret helper)
DISCORD_TOKEN_FILE: /run/secrets/discord_token
DISCORD_APPLICATION_ID_FILE: /run/secrets/discord_application_id
S3_BUCKET_FILE: /run/secrets/s3_bucket
S3_REGION_FILE: /run/secrets/s3_region
S3_ENDPOINT_FILE: /run/secrets/s3_endpoint
# Egress proxy (regular proxy mode — NOT transparent)
HTTPS_PROXY: http://mitmproxy:8080
HTTP_PROXY: http://mitmproxy:8080
NO_PROXY: workspace,localhost,127.0.0.1
NODE_EXTRA_CA_CERTS: /etc/ssl/certs/mitmproxy-ca-cert.pem
volumes:
# Credentials — bind-mounted as accepted-risk (no Docker Swarm secrets in v1)
- ./secrets/discord-token:/run/secrets/discord_token:ro
- ./secrets/discord-application-id:/run/secrets/discord_application_id:ro
- ./secrets/s3-bucket:/run/secrets/s3_bucket:ro
- ./secrets/s3-region:/run/secrets/s3_region:ro
- ./secrets/s3-endpoint:/run/secrets/s3_endpoint:ro
# mitmproxy CA cert (written by mitmproxy on first start, read here for trust)
- mitmproxy-certs:/etc/ssl/certs:ro
networks:
- gateway-net
- sandbox-net
restart: unless-stopped
healthcheck:
test: [CMD, node, -e, process.exit(0)]
interval: 30s
timeout: 5s
retries: 3

workspace:
build:
context: ..
dockerfile: deploy/workspace.Dockerfile
depends_on:
mitmproxy:
condition: service_healthy
environment:
# Egress proxy — same as gateway
HTTPS_PROXY: http://mitmproxy:8080
HTTP_PROXY: http://mitmproxy:8080
NO_PROXY: localhost,127.0.0.1,workspace
networks:
- sandbox-net
restart: unless-stopped

mitmproxy:
image: mitmproxy/mitmproxy:11.0.2
command: >
mitmdump
-s /scripts/allowlist.py
--listen-host 0.0.0.0
--listen-port 8080
--set confdir=/home/mitmproxy/.mitmproxy
--set ssl_insecure=false
environment:
# Comma-separated list of exact object-store hosts to allow (e.g.
# "my-bucket.s3.amazonaws.com,my-account.r2.cloudflarestorage.com").
# The static allowlist does NOT include broad S3/R2 wildcards — set this
# to the exact bucket host(s) your deployment uses. If unset or empty,
# all S3/R2 traffic is blocked (fail-closed default).
OBJECT_STORE_HOSTS: ${OBJECT_STORE_HOSTS:-}
volumes:
- ./mitmproxy/allowlist.py:/scripts/allowlist.py:ro
# Named volume: mitmproxy writes CA here; gateway/workspace read it for trust
- mitmproxy-certs:/home/mitmproxy/.mitmproxy
networks:
- sandbox-net
restart: unless-stopped
healthcheck:
# Gate dependent services on the CA actually existing in the shared
# volume. mitmproxy generates ~/.mitmproxy/mitmproxy-ca-cert.pem on
# first start; it takes ~1-3 seconds on cold-volume boot. Until this
# check passes, gateway/workspace will not start.
test: [CMD-SHELL, test -f /home/mitmproxy/.mitmproxy/mitmproxy-ca-cert.pem]
interval: 5s
timeout: 3s
retries: 12
start_period: 30s

volumes:
# Shared CA cert volume: mitmproxy writes, gateway + workspace read
mitmproxy-certs:

networks:
# External-facing network for gateway ↔ Discord
gateway-net:
driver: bridge
# Internal sandbox network: workspace + mitmproxy; gateway bridges both
sandbox-net:
driver: bridge
internal: true
45 changes: 45 additions & 0 deletions deploy/gateway.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# syntax=docker/dockerfile:1

# ── Stage 1: build ────────────────────────────────────────────────────────────
FROM node:24-alpine AS build

WORKDIR /workspace

# Enable corepack for pnpm
RUN corepack enable

# Copy workspace root manifests first (layer-cache friendly)
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml tsconfig.base.json ./

# Copy only the packages we need for the gateway build
COPY packages/runtime/ packages/runtime/
COPY packages/gateway/ packages/gateway/

# Install dependencies for gateway and its workspace deps only
RUN pnpm install --frozen-lockfile --filter @fro-bot/gateway...

# Build runtime first (gateway depends on it)
RUN pnpm --filter @fro-bot/runtime build

# Build gateway
RUN pnpm --filter @fro-bot/gateway build

# ── Stage 2: runtime ──────────────────────────────────────────────────────────
FROM node:24-alpine AS runtime

WORKDIR /app

# Copy production node_modules from build stage
COPY --from=build /workspace/node_modules ./node_modules
COPY --from=build /workspace/packages/runtime/package.json ./packages/runtime/package.json
COPY --from=build /workspace/packages/runtime/dist/ ./packages/runtime/dist/
COPY --from=build /workspace/packages/gateway/package.json ./packages/gateway/package.json
COPY --from=build /workspace/packages/gateway/dist/ ./packages/gateway/dist/

WORKDIR /app/packages/gateway

# Placeholder healthcheck — real HTTP health endpoint arrives in Unit 7
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
CMD node -e 'process.exit(0)'

CMD ["node", "dist/main.mjs"]
42 changes: 42 additions & 0 deletions deploy/init-certs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# init-certs.sh — Bootstrap the mitmproxy CA into the named Docker volume.
#
# Run this once before starting the stack for the first time. Idempotent:
# skips generation if the CA cert already exists in the volume.
#
# Usage (from repo root):
# bash deploy/init-certs.sh
set -euo pipefail

VOLUME_NAME="fro-bot_mitmproxy-certs"
CERT_FILE="mitmproxy-ca-cert.pem"

echo "==> Checking for existing mitmproxy CA in volume '${VOLUME_NAME}'..."

# Check if the cert already exists in the volume
if docker run --rm \
-v "${VOLUME_NAME}:/certs" \
--entrypoint sh \
mitmproxy/mitmproxy:11.0.2 \
-c "test -f /certs/${CERT_FILE}"; then
echo "==> CA cert already present — skipping generation."
exit 0
fi

echo "==> Generating mitmproxy CA (this may take a moment)..."

# Run mitmdump with -n (no-server mode) so it initialises the confdir and exits.
# The CA key + cert are written to the volume at /home/mitmproxy/.mitmproxy.
docker run --rm \
-v "${VOLUME_NAME}:/home/mitmproxy/.mitmproxy" \
mitmproxy/mitmproxy:11.0.2 \
mitmdump --set confdir=/home/mitmproxy/.mitmproxy --quiet -n

echo "==> CA generated successfully."
echo " Volume: ${VOLUME_NAME}"
echo " Cert: ${CERT_FILE} (inside volume at /home/mitmproxy/.mitmproxy/)"
echo ""
echo " To add the CA to your host trust store (optional, dev only):"
echo " docker run --rm -v ${VOLUME_NAME}:/certs alpine cat /certs/${CERT_FILE} \\"
echo " | sudo tee /usr/local/share/ca-certificates/mitmproxy-fro-bot.crt"
echo " sudo update-ca-certificates"
2 changes: 2 additions & 0 deletions deploy/mitmproxy/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
__pycache__/
*.pyc
Loading
Loading