Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 67 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,32 @@ jobs:
- name: Run backend tests
run: bun run test

# ─────────────────────────────────────────────────────────────────────────────
# Compose sync — named volumes in docker-compose.yml must match the embedded
# packaging/electron/resources/docker-compose.yml. Divergence caused the
# v1.0.2 data-wipe bug; catch it early in CI, not just at release time.
# ─────────────────────────────────────────────────────────────────────────────
verify-compose-sync:
name: Verify Compose Volume Sync
runs-on: ubuntu-24.04
timeout-minutes: 5
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4

- name: Check named volumes match between compose files
run: |
ROOT_VOLS=$(awk '/^volumes:/{found=1; next} found && /^ [a-zA-Z]/{gsub(/:$/, "", $1); print $1}' docker-compose.yml | sort)
ELECTRON_VOLS=$(awk '/^volumes:/{found=1; next} found && /^ [a-zA-Z]/{gsub(/:$/, "", $1); print $1}' packaging/electron/resources/docker-compose.yml | sort)
echo "Root volumes : $ROOT_VOLS"
echo "Electron volumes: $ELECTRON_VOLS"
if [ "$ROOT_VOLS" != "$ELECTRON_VOLS" ]; then
echo "ERROR: Named volumes out of sync — add missing volumes to packaging/electron/resources/docker-compose.yml"
exit 1
fi
echo "Compose volume sync OK."

# ─────────────────────────────────────────────────────────────────────────────
# Quality Gate — all quality checks must pass before Docker builds run.
# Prevents wasting expensive Docker build time on broken commits.
Expand All @@ -273,6 +299,7 @@ jobs:
- build-frontend
- test-frontend
- test-backend
- verify-compose-sync
if: always()
steps:
- name: Check all gates passed
Expand Down Expand Up @@ -328,6 +355,7 @@ jobs:
needs: [build-image]
permissions:
contents: read
security-events: write # required to upload SARIF to GitHub Security
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4

Expand All @@ -340,14 +368,31 @@ jobs:
- name: Load Docker image
run: docker load < /tmp/vision-ci.tar

- name: Run Trivy vulnerability scan
- name: Run Trivy vulnerability scan (table — fail on HIGH/CRITICAL)
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
image-ref: vision:ci
format: table
severity: HIGH,CRITICAL
exit-code: '1'

- name: Run Trivy vulnerability scan (SARIF — upload to GitHub Security)
if: always()
uses: aquasecurity/trivy-action@ed142fd0673e97e23eac54620cfb913e5ce36c25 # v0.36.0
with:
image-ref: vision:ci
format: sarif
output: trivy-results.sarif
severity: HIGH,CRITICAL
exit-code: '0'

- name: Upload Trivy SARIF to GitHub Security
if: always()
uses: github/codeql-action/upload-sarif@9e907b5e64f6b83e7804b09294d44122997950d6 # v4
with:
sarif_file: trivy-results.sarif
category: trivy

# ─────────────────────────────────────────────────────────────────────────────
# Docker verify — spin up compose, hit /health
# ─────────────────────────────────────────────────────────────────────────────
Expand Down Expand Up @@ -475,3 +520,24 @@ jobs:
- name: Tear down
if: always()
run: docker compose down -v

# ─────────────────────────────────────────────────────────────────────────────
# CI Complete — aggregates the Docker-tier results so branch protection has a
# single required-status-check that covers the full pipeline.
# Set "CI Complete" as the required check in branch protection settings.
# ─────────────────────────────────────────────────────────────────────────────
ci-complete:
name: CI Complete
runs-on: ubuntu-24.04
timeout-minutes: 5
needs: [trivy-scan, docker-verify, test-live-api-contracts]
if: always()
steps:
- name: Check Docker-tier stages passed
run: |
results='${{ toJson(needs.*.result) }}'
if echo "$results" | grep -qE '"(failure|cancelled)"'; then
echo "CI failed in Docker tier. Job results: $results"
exit 1
fi
echo "All CI stages complete."
12 changes: 12 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,18 @@ jobs:
- name: Install dependencies
run: bun install --frozen-lockfile

- name: Check named volumes match between compose files
run: |
ROOT_VOLS=$(awk '/^volumes:/{found=1; next} found && /^ [a-zA-Z]/{gsub(/:$/, "", $1); print $1}' docker-compose.yml | sort)
ELECTRON_VOLS=$(awk '/^volumes:/{found=1; next} found && /^ [a-zA-Z]/{gsub(/:$/, "", $1); print $1}' packaging/electron/resources/docker-compose.yml | sort)
echo "Root volumes : $ROOT_VOLS"
echo "Electron volumes: $ELECTRON_VOLS"
if [ "$ROOT_VOLS" != "$ELECTRON_VOLS" ]; then
echo "ERROR: Named volumes out of sync — add missing volumes to packaging/electron/resources/docker-compose.yml before releasing"
exit 1
fi
echo "Compose volume sync OK."

- name: Check version tag matches package.json
run: |
TAG="${{ github.event.inputs.tag || github.ref_name }}"
Expand Down
119 changes: 119 additions & 0 deletions docs/adr/051-docker-compose-sync-named-volumes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
title: Docker Compose Named Volumes Sync Policy
type: adr
status: accepted
date: 2026-05-07
tags: [adr, docker-compose, named-volumes, ci-cd, electron, release, data-loss, v1.0.2-bug]
description: Enforces synchronization of named volumes between root and embedded Docker Compose files via CI gate to prevent data loss on updates
aliases: [compose-sync, volume-sync, named-volumes-policy]
related_code: [".github/workflows/ci.yml", ".github/workflows/release.yml", "docker-compose.yml", "packaging/electron/resources/docker-compose.yml"]
---

# ADR-051: Docker Compose Named Volumes Sync Policy

## Status
Accepted (May 2026)

## Date
2026-05-07

## Context

### The v1.0.2 Attachment Wipe Bug

Vision ships as an Electron desktop application. The packaged app includes an embedded Docker Compose file (`packaging/electron/resources/docker-compose.yml`) that is baked into the `.app` bundle at build time. This embedded file mirrors the production Docker Compose configuration (`docker-compose.yml` at repo root).

In v1.0.2, a named volume was added to the root `docker-compose.yml` to persist attachment files. However, **the embedded Electron compose file was not updated**. When users launched the packaged app:

1. The app's embedded compose file lacked the new volume definition
2. Docker Compose ran with only the volumes from the embedded file
3. The attachment volume mount failed silently or used a different location
4. On the next update, file locations changed or data was not persisted
5. **Result:** Users lost all attachments

This was a **critical data loss bug** that should have been caught before release.

### Synchronization Challenge

Both compose files serve different purposes:
- **Root `docker-compose.yml`:** Source of truth for production/development deployments
- **Embedded `packaging/electron/resources/docker-compose.yml`:** Packaged into the Electron `.app`; must be manually synced at release

The sync is manual and easy to forget because:
- The embedded file is in a subdirectory (`packaging/electron/resources/`)
- There is no automated check that they match
- Developers might add a volume to root and forget to propagate it
- The bug only surfaces in packaged builds, not during dev

## Decision

**Enforce strict synchronization of named volumes between both Docker Compose files via automated CI checks.**

### CI Implementation

1. **`verify-compose-sync` job in ci.yml** (May 2026)
- Runs on every push to main and PR
- Extracts named volumes from both compose files
- Compares them; fails if they diverge
- Added to `quality-gate` prerequisites (blocks Docker build)

2. **`verify` job in release.yml** (May 2026)
- Runs before any Docker push or packaging
- Same volume sync check as ci.yml
- Must pass before release can proceed

### Sync Check Logic

```bash
ROOT_VOLS=$(awk '/^volumes:/{found=1; next} found && /^ [a-zA-Z]/{gsub(/:$/, "", $1); print $1}' docker-compose.yml | sort)
ELECTRON_VOLS=$(awk '/^volumes:/{found=1; next} found && /^ [a-zA-Z]/{gsub(/:$/, "", $1); print $1}' packaging/electron/resources/docker-compose.yml | sort)

if [ "$ROOT_VOLS" != "$ELECTRON_VOLS" ]; then
echo "ERROR: Named volumes out of sync"
exit 1
fi
```

Parses the YAML `volumes:` section, extracts named volume keys, compares.

### Workflow Implications

**Adding a new named volume:**

1. Edit `docker-compose.yml` — add volume definition
2. **Must also edit** `packaging/electron/resources/docker-compose.yml` — add same volume
3. Push changes to PR
4. CI `verify-compose-sync` job runs
- If volumes match: passes, PR can merge
- If volumes don't match: fails, blocks merge until both files are in sync
5. Once merged to main, pre-release checks and release.yml verify job confirm sync before packaging

**Key constraint:** *Every new named volume must be added to both files, or the PR will be blocked.*

## Consequences

### Positive

- **Prevents data loss:** Automated guard rail catches sync drift before release
- **Release safety:** `verify` job in release.yml ensures no data-loss bugs ship
- **Developer awareness:** CI failure message educates: "add missing volumes to packaging/electron/resources/docker-compose.yml"
- **Cost:** Minimal — simple YAML parsing, runs in <5 seconds

### Neutral

- Adds one more CI gate in quality-gate (verify-compose-sync)
- Developers must remember to sync both files when adding volumes
- Mitigated by CI failure message
- Mitigated by future docs and PR templates

### Negative

- None identified; the policy is defensive and low-cost

## Related

- [[docs/guides/cicd-pipelines#3-verify-compose-sync--docker-compose-sync-check|CI/CD Pipelines: Verify Compose Sync]] — Detailed CI job documentation
- [[docs/guides/deployment#6-admin-endpoints-security|Deployment Guide: Admin Endpoints Security]] — Cross-references compose sync requirement
- [[docs/architecture/electron#cicd-integration-april-may-2026|Electron Architecture: CI/CD Integration]] — Release workflow context
- `.github/workflows/ci.yml` — `verify-compose-sync` job
- `.github/workflows/release.yml` — `verify` job compose check
8 changes: 6 additions & 2 deletions docs/adr/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ status: active
date: 2026-04-23
updated: 2026-05-07
last_modified: 2026-05-07
tags: [adr, index, architecture, decisions, phase-1, phase-4, phase-5, phase-6, phase-7, security, dependency-slim-down, container-hardening, docker, backup, encryption, aead, aes-256-gcm, codeql, dependabot, rate-limiting, tailwind-v4, css-architecture, dependencies, ai, streaming, useSyncExternalStore, bug-hunt, recovery-hardening, updated-at-constraints, concurrent-backup, ci-cd, secrets-scanning, supply-chain-security, gitleaks, deps-audit, trivy-scan]
description: Architecture Decision Records documenting significant technical choices and their rationale. May 2026: CI supply chain security tooling (ADR-050), Phase 6.1–7 bug hunt recovery hardening (ADR-049), AI Chat module-level stream store (ADR-048), Tailwind CSS v4 migration (ADR-047).
tags: [adr, index, architecture, decisions, phase-1, phase-4, phase-5, phase-6, phase-7, security, dependency-slim-down, container-hardening, docker, backup, encryption, aead, aes-256-gcm, codeql, dependabot, rate-limiting, tailwind-v4, css-architecture, dependencies, ai, streaming, useSyncExternalStore, bug-hunt, recovery-hardening, updated-at-constraints, concurrent-backup, ci-cd, secrets-scanning, supply-chain-security, gitleaks, deps-audit, trivy-scan, docker-compose-sync, named-volumes, data-loss]
description: Architecture Decision Records documenting significant technical choices and their rationale. May 2026: Docker Compose volume sync policy (ADR-051), CI supply chain security tooling (ADR-050), Phase 6.1–7 bug hunt recovery hardening (ADR-049), AI Chat module-level stream store (ADR-048), Tailwind CSS v4 migration (ADR-047).
aliases: [ADRs, decisions, architecture decisions]
---

Expand Down Expand Up @@ -45,6 +45,10 @@ See [[docs/adr/template\|the ADR template]] for the format to use when creating

## Recent Decisions

### 2026-05-07: Docker Compose Named Volumes Sync Policy

[[docs/adr/051-docker-compose-sync-named-volumes|ADR-051]] — Enforce strict synchronization of named volumes between root `docker-compose.yml` and embedded `packaging/electron/resources/docker-compose.yml` via CI gates. Automated `verify-compose-sync` job in ci.yml runs on every push/PR, extracting and comparing named volumes from both files; fails if divergence detected. Same check added to release.yml `verify` job before any packaging. Prevents v1.0.2 data-loss bug where missing attachment volume in embedded Electron compose caused attachments to vanish on updates. Consequences: new named volumes blocked from merging until both files are synced, release safety guaranteed, developer awareness via CI failure message.

### 2026-05-07: CI Supply Chain Security Tooling

[[docs/adr/050-ci-supply-chain-security-tooling|ADR-050]] — Four-layer supply chain hardening via CI automation: (1) **Secrets scan** — gitleaks in CI (`secrets-scan` job, full history on every push/PR) + pre-commit hook (local dev, staged changes only), blocks merge if credentials/tokens/keys found; (2) **Dependency audit** — `bun audit --audit-level=high` fails on HIGH/CRITICAL vulns, dependency overrides for basic-ftp/ip-address/postcss pinned to safe versions; (3) **Container scan** — Trivy scans Docker image for OS package CVEs, blocks merge if HIGH/CRITICAL found; (4) **Electron hardening** — `session.defaultSession.setPermissionRequestHandler` denies all renderer permission requests (camera, mic, geolocation, clipboard), strict CSP on error.html (no unsafe-inline). Consequences: secrets never reach CI, transitive dependency exploits blocked, container images patched, renderer cannot escalate to system resources even if XSSed. Friction: developer pre-commit hook setup, CI latency +2-3 min, gitleaks false-positive maintenance.
Expand Down
31 changes: 23 additions & 8 deletions docs/architecture/electron.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,20 +417,35 @@ On failure at any step: error toast shown, user can manually restore from `pre-u
- Manual "Check for Updates" button
- Displays latest available version if newer

#### CI/CD Integration (April 2026)
#### CI/CD Integration (April–May 2026)

**release.yml** (May 2026):
- `verify` job checks:
- Named volumes in `docker-compose.yml` match `packaging/electron/resources/docker-compose.yml`
- Version tag matches both package.json files
- JS dependencies audit passes
- Blocks all other jobs until checks pass

**release.yml:**
- `verify` job (version tag check, lint, tests) blocks all others
- `docker` job pushes image with version tag to GHCR
- `package-mac` job generates `.sha256` checksum alongside ZIP
- Both artifacts uploaded to GitHub release

**ci.yml:**
- Lint, typecheck, unit tests
- `docker-verify` job: build image, start compose, poll health on port 3002
- `security-scan` job: Trivy scan with SARIF upload to GitHub Security
**ci.yml** (May 2026):
- Early stage:
- `secrets-scan`, `deps-audit`, `pip-audit`, `lint`, `typecheck`, `build-frontend`, `test-frontend`, `test-backend`
- `verify-compose-sync` — compares named volumes between root and embedded compose files
- `quality-gate` — aggregates all early checks, gates expensive Docker build

- Docker stage (after quality-gate passes):
- `build-image` — builds Docker image once, reused by downstream jobs
- `trivy-scan` — scans image for OS/system CVEs
- `docker-verify` — container health check (build image, start compose, poll health on port 3002)
- `test-live-api-contracts` — validates MSW fixtures against real backend responses
- `ci-complete` — final aggregation gate (set as required status check in branch protection)

**Security focus:** Compose sync verification prevents the v1.0.2 data-loss bug where omitted volumes caused attachments to vanish on update.

Detailed workflow definitions in [[docs/features/application-updates|Application Updates Feature]].
Detailed workflow definitions in [[docs/features/application-updates|Application Updates Feature]] and [[docs/guides/cicd-pipelines|CI/CD Pipelines Guide]].

---

Expand Down
Loading
Loading