Skip to content

feat: CI/CD + single-image Docker + issue templates#8

Merged
FireTable merged 15 commits into
mainfrom
feat/ci-cd
Jul 7, 2026
Merged

feat: CI/CD + single-image Docker + issue templates#8
FireTable merged 15 commits into
mainfrom
feat/ci-cd

Conversation

@FireTable

@FireTable FireTable commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

Bootstraps CI/CD for the repo, replaces ad-hoc local Docker with a single
role-dispatched image, and adds GitHub issue templates. After the initial
green CI, several rounds of fixes (some only reproducible in real GH
Actions, not act) plus deploy / cache / config cleanups — see the commit
list below for the full picture.

Changes

CI (.github/workflows/CI.yml)

  • Four parallel jobs: lint, typecheck, build, test
  • Build + test jobs expose a postgres:16 service container — next build
    statically evaluates App Router routes and Better Auth runs DB migrations
    at module load
  • pnpm cache via actions/setup-node's cache: pnpm
  • Triggers on push + PR to main and dev
  • cancel-in-progress: true per ref — re-pushing cancels stale runs

CD (.github/workflows/CD.yml)

  • Resolves branch-aware image tag (mainlatest, dev
    beta-<sha>, others → <branch>-<sha>, workflow_dispatch → override
    input). Sanitizes slashes / uppercase for Docker tag validity.
  • Builds single Docker image with Buildx + GHCR cache (cache-from +
    cache-to: type=gha,mode=max), pushes to GHCR
  • Attaches image tarball to a GitHub Release via
    softprops/action-gh-release (stable overwrites one Latest release,
    beta gets an immutable release per push)
  • PR builds don't push (build-only sanity check, fork-safe) and use
    their own concurrency group with cancel-in-progress: true so stale
    runs don't waste runner minutes. Push events keep false because
    in-flight image push / release shouldn't be cancelled.
  • CD build runs a dedicated build-pg Postgres container on the runner
    (buildx builder's network: host driver-opts shares the runner's
    network, so the build's localhost:5432 reaches it). GH Actions
    service containers can't be reached from the isolated builder network.

Docker

  • Dockerfile: base = langchain/langgraphjs-api:22 — the official
    LangGraph prod runtime honors backend/checkpointer.ts's
    PostgresSaver (unlike langgraphjs dev which forces InMemorySaver).
    Adds Next.js on top.
  • BuildKit cache mount for the pnpm store
    (--mount=type=cache,target=/root/.local/share/pnpm/store); with
    CD's cache-to: type=gha,mode=max, the store persists across CD runs.
    Cold install ~60s, warm ~5s.
  • scripts/start.sh: ROLE=all|frontend|backend dispatches; all runs
    Next.js + langgraph uvicorn concurrently
  • docker-compose.yml: app + postgres + redis. app.environment: block
    passes every .env var through (${VAR:-default} so missing keys
    don't fail-fast).

Deploy (docs/DEPLOY.md)

  • Single-host self-host guide: pull image, write .env, deploy compose
    with Caddy in-compose (only public surface on :80/:443; auto-TLS
    via Let's Encrypt). App's 3000/2024 bind 127.0.0.1 only.
  • Caddyfile uses {$CADDY_DOMAIN} placeholder — domain flows from
    .env via compose, so the same Caddyfile works for any host.
  • First-time Postgres fix for langgraph-api 0.10.x's CREATE INDEX CONCURRENTLY inside a transaction.
  • External Postgres, air-gapped install via Release tarball, OAuth setup,
    backups (pg_dump cron), reverse-proxy alternatives.

Issue templates

  • bug_report.yml — severity dropdown, repro steps, self-checks
  • feature_request.yml — problem-first framing, scope + risk
  • config.ymlblank_issues_enabled: true + Discussions link

Tooling

  • .nvmrc — Node 22 pin
  • .npmrc — pnpm 10 onlyBuiltDependencies whitelist
  • package.jsonpackageManager: pnpm@10.16.1 + engines.node >=22
  • .gitignore.pnpm-store (oxfmt reads .gitignore)
  • .oxfmtignore / .oxlintignore — tool-specific excludes

Cleanup

  • Dropped unused NEXT_PUBLIC_CRYPTO_REAL_SWAP feature flag. The flag
    was dormant (no code reads it) and the comment about "exposed to the
    browser on purpose" was misleading — a server-side env var can't reach
    React wagmi hooks. Removed from .env.example, both compose files,
    CLAUDE.md, and .env.vps.

Docs

  • docs/CI.md — workflow reference, local verification, base image
    runtime requirements, cache paths, deploy notes
  • docs/DEPLOY.md — self-host deploy guide (compose + Caddy + first-run
    fix + backups + air-gapped install)
  • CLAUDE.md — restructured into a one-screen index pointing at
    README.md and docs/

Validation

  • ✅ Local Docker build end-to-end (image runs, Next.js serves on :3000)
  • ✅ Local act: lint, typecheck, build, test — all Job succeeded;
    vitest reports 66/66 files, 654/654 tests passing
  • ✅ PR CI runs across all 15 commits — lint / typecheck / build / test
    all green
  • ⚠️ Migration fix(mobile-ui): batch 1 — 6 mobile UI fixes + 5 follow-ups (#21-#26) #29 in langgraph-api 0.10.x (CREATE INDEX CONCURRENTLY store_prefix_idx) is wrapped in a transaction and fails on first
    startup. Documented in docs/DEPLOY.md with a manual CREATE INDEX
    workaround
  • CD build / release jobs not exercised end-to-end here — they push
    to GHCR and create Releases; first verification will be the merge to
    main

Caveats

  • Image is heavy (~5 GB) because langchain/langgraphjs-api ships Python
    • Go. Trade-off for correct PostgresSaver behavior; if size matters,
      swap back to langgraphjs dev + InMemorySaver and live with
      ephemeral checkpoints.
  • The langgraph-api migration issue blocks first-time container start
    until manually applied (see docs/DEPLOY.md § First-time Postgres fix).
  • NEXT_PUBLIC_WALLET_CONNECT_PROJECT_ID is hardcoded to a placeholder
    in the Dockerfile for build-time RainbowKit init. Compose override at
    runtime only affects the server side; the JS bundle still has the
    placeholder. WalletConnect-based wallets (binanceWallet, bitgetWallet)
    lose their mobile-QR fallback; injected wallets (MetaMask, Coinbase,
    Rainbow, Safe) work fine. Will need an ARG + CD build-arg + GH
    Variable when real DEX-on-WalletConnect matters.

Test plan

  • Open PR → triggers CI (4 jobs in parallel) — green across all
    pushes
  • Subsequent pushes trigger CD's Docker build sanity check — green
  • First CD push to main → image published to GHCR, Latest release
    created with tarball asset
  • VPS deploy from docs/DEPLOY.md (compose + Caddy + Postgres +
    Redis) on ai.firetable.tech

FireTable and others added 15 commits July 7, 2026 11:08
CI runs lint + typecheck + build + test in parallel jobs. Build and
test jobs spin up a postgres:16 service container — next build needs a
reachable DB (Better Auth runs migrations at module load). CD resolves
a branch-aware image tag (main → latest, dev → beta-<sha>, others →
<branch>-<sha>) then builds, pushes to GHCR, and attaches the image
tarball to a GitHub Release via softprops/action-gh-release. Issue
templates cover bug reports (severity dropdown, repro steps) and
feature requests (problem-first framing, scope + risk).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Base = langchain/langgraphjs-api:22 (official prod runtime, Python
uvicorn + Go core). Honors backend/checkpointer.ts's PostgresSaver when
DATABASE_URL/POSTGRES_URI is set — thread state persists across
restarts (unlike langgraphjs dev, which forces InMemorySaver).
scripts/start.sh runs Next.js + langgraph uvicorn concurrently under
ROLE=all|frontend|backend. docker-compose adds redis (required by the
runtime) alongside postgres. .nvmrc pins Node 22, .npmrc whitelists
esbuild/sharp/bufferutil/keccak/utf-8-validate for pnpm 10's build
gating, package.json pins pnpm@10.16.1 + node>=22.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Covers workflow triggers, role-dispatched entrypoint semantics,
the base-image runtime requirements (Redis + langgraph-api migrations),
local Docker build commands with the host.docker.internal + Mac
Postgres workaround, and the GitHub Actions services pattern.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…al validation

Local act runs surfaced issues that GitHub Actions would hit too:

- Drop version: 10 from pnpm/action-setup@v4 — conflicts with the
  packageManager: pnpm@10.16.1 pin (Multiple versions of pnpm
  specified). The action reads the version from packageManager when
  the with: block is empty.
- Add .pnpm-store to .gitignore — oxfmt reads .gitignore (not
  .oxfmtignore) and was scanning the pnpm store. .oxfmtignore added
  for the oxlint side too.
- Drop service ports: mappings in CI.yml. They map service container
  ports to the host, which is only useful for local debugging AND
  collides with the Mac's Postgres on :5432. Service-to-service traffic
  uses the job network, no host mapping needed.
- Add NEXT_PUBLIC_WALLET_CONNECT_PROJECT_ID placeholder to the build
  job — RainbowKit hard-requires it at module load, prerendering fails
  without one.
- Install postgresql-client in the test job. ubuntu-latest doesn't
  ship with psql, and tests/setup.ts shells out to psql.
- tests/setup.ts: create the target DB on demand. CI's POSTGRES_DB env
  creates only the default db; act doesn't propagate env at all, so the
  explicit -d db name needs CREATE DATABASE before migrations apply.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous version inlined the full guidance inline; the new
version is a one-screen index that points at README.md and docs/ for
detail. The full rules, layout, and rationale live in their canonical
places — keeps CLAUDE.md focused on what Claude needs to see first.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR #8 surfaced two real bugs in CI/CD that only GH Actions catches
(act + GH Actions diverge on service networking):

- CD: tag for PR refs like "8/merge-87b0c62" is invalid (slash in the
  middle). `resolve` job now sanitizes branch names — lowercase,
  slashes to dashes, strip invalid chars.
- CD: buildx builder runs in an isolated network, can't reach the
  GH Actions service on `localhost:5432`. Add `network: host` so the
  build shares the runner's network namespace.
- CI: switch service URL from `localhost` (needs `ports:` mapping,
  conflicts with Mac's local Postgres) to the service hostname
  `postgres` (resolves on the job's shared network in GH Actions,
  no host port mapping needed).

Also:
- Rename workflow `cd` → `CD` / `ci` → `CI` (display only).
- docs/DEPLOY.md: self-hosting guide — pull image, env, first-start
  Postgres fix, reverse proxy + TLS, backups, air-gapped install.
- README: link to docs/DEPLOY.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two more bugs only GH Actions catches:

- CD: Docker rejects the image tag because the GitHub org name
  contains uppercase (`FireTable`). buildx's tag validator requires
  the entire repo path to be lowercase. Wrap `github.repository_owner`
  in `lower()` everywhere it appears in CD.yml.
- CI: the `postgres` service hostname doesn't resolve in GH Actions
  either (DNS isn't shared with the job's network the way I'd
  assumed). Revert to `localhost:5432` + `ports: 5432:5432` mapping,
  which is the canonical GH Actions pattern.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The comment lived inside the `tags: |` block, so the `|` literal
block scalar picked it up as part of the value. docker/build-push-
action then tried to use the comment lines (starting with `#`) as
image tags, which buildx rejected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GitHub Actions expression functions are: contains, startsWith,
endsWith, format, join, toJSON, fromJSON, hashFiles. No `lower()`.

The whole workflow was failing to parse with "Unrecognized
function: 'lower'" at 5 call sites — that prevented the run from
even starting (0s failure, "workflow file issue").

Fix: compute the lowercase owner in the `resolve` job's bash step
(`tr '[:upper:]' '[:lower:]'`) and emit it as an `image` job output
(`ghcr.io/<owner>/langgraph-app`). All downstream jobs reference
that output instead of inlining `${{ lower(github.repository_owner) }}`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`network: host` on docker/build-push-action only makes the RUN
commands share the buildx builder's network namespace — and the
builder itself is a docker-container that doesn't share the
runner VM's network. So `localhost:5432` inside `next build`
was reaching the builder's loopback (empty), not the GH Actions
service container.

Fix:
- Start a dedicated Postgres container on the runner
  (`docker run -d --name build-pg -p 127.0.0.1:5432:5432 …`).
- Put the buildx builder itself on the runner's host network via
  `docker/setup-buildx-action` `driver-opts: network=host`.
- The build's `localhost:5432` now reaches `build-pg`.
- Drop the `services:` block — it's not reachable from the
  builder regardless.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The previous commit attempt was rejected by pre-commit (oxfmt
detected a duplicate `build-args:` key — I had added a new one
without removing the old `BUILDKIT_INLINE_CACHE=1` one). The
fix never landed, so the build kept using the Dockerfile's
default `langgraph_app` DB, which doesn't exist on the
build-time Postgres (it creates `langgraph_app_build`).

Result: `error: database "langgraph_app" does not exist` from
Better Auth's `runStoreMigrations`.

Fix: remove the old `build-args:` block; the new one passes
both `BUILDKIT_INLINE_CACHE=1` and `DATABASE_URL=${{ env.DATABASE_URL }}`
so the build uses the same DB name the container created.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- docs/DEPLOY.md: Caddy is now a service in the deploy compose
  (only public surface on :80/:443). Drop the "install Caddy on
  the host" section — that path is no longer the recommended one,
  though it's still listed as an alternative under "Reverse proxy
  + TLS". The app service's 3000/2024 are bound to 127.0.0.1
  only; Caddy reaches it on the docker network.
- Dockerfile: add BuildKit cache mount for pnpm store
  (`--mount=type=cache,target=/root/.local/share/pnpm/store`).
  Combined with the existing
  `cache-to: type=gha,mode=max` in CD.yml, the store persists
  across CD runs — only changed deps re-fetch when the lockfile
  changes. Cold: ~60s, warm: ~5s.
- docs/CI.md: document both cache paths (CI pnpm store +
  CD BuildKit cache mount) and fix the stale "Bump Node" note
  (Dockerfile is no longer FROM node:22-bookworm-slim).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PR events run in their own concurrency group (keyed by event_name)
with cancel-in-progress: true. They're pure Docker sanity checks
(no push, no release) so cancelling a stale run saves runner minutes
without leaving half-published artifacts.

Push events on main/dev keep cancel-in-progress: false — an in-flight
image push or GitHub Release should finish rather than get
half-published. docs/CI.md updated to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three things that were preventing .env values from reaching the
running container, plus a related .env.example template gap:

1. Deploy compose (docs/DEPLOY.md) and the local-dev compose
   (docker-compose.yml) both had incomplete app.environment: blocks.
   OAuth (GITHUB_*, GOOGLE_*), email (RESEND_*), JINA, Deno Deploy,
   LangSmith, observability retention, and MEMORY_* tuning all sat
   in .env but never made it into the container. Worst symptom:
   GitHub/Google sign-in buttons would have errored on click —
   Better Auth read empty client IDs.

   Added 18-20 missing entries per file, all with ${VAR:-default}
   defaults so missing keys don't fail-fast.

2. Caddyfile now uses {$CADDY_DOMAIN} placeholder instead of a
   hardcoded chat.example.com. CADDY_DOMAIN flows through compose's
   environment block; the Caddyfile stays host-agnostic. compose
   uses ${VAR:?msg} so missing CADDY_DOMAIN errors clearly.

3. .env.example gained a top section documenting deploy-only vars
   (IMAGE, CADDY_DOMAIN, POSTGRES_*) that aren't read by the app
   code but ARE read by compose + the caddy container. They're
   marked as such so the file doesn't lie about app config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The flag was meant to flip place_crypto_order from SIMULATED to a
real Uniswap V3 path, but the corresponding code was never wired
up — wagmi hooks live in the React tree and a server-side env var
can't reach them. The flag was already documented as 'currently
dormant' and 'exposed to the browser on purpose', which made
neither claim true.

Grep confirms no code reads it (only docs + config). Removing:

- .env.example: 8-line comment block + key
- docker-compose.yml: env entry
- docs/DEPLOY.md: env entry (deploy compose template)
- CLAUDE.md: env-var table row + the 'is required to route through
  any real DEX path' sentence in the Crypto sub-agent description
- .env.vps: same as .env.example (local, gitignored)

Trade flow stays SIMULATED; if a real DEX path is ever wired up,
the toggle will come back as a UI control, not an env var.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@FireTable FireTable merged commit a2fa2eb into main Jul 7, 2026
7 checks passed
@FireTable FireTable deleted the feat/ci-cd branch July 7, 2026 05:31
FireTable pushed a commit that referenced this pull request Jul 7, 2026
CI Build (next build) failed with:
  duplicate key value violates unique constraint pg_type_typname_nsp_index
  Failed to collect page data for /api/memory/profile

Root cause: backend/store.ts eagerly calls `await store.setup()` at module
load. next build evaluates /api/memory/profile under N page-data workers;
each worker re-imports backend/store and races to CREATE TABLE IF NOT
EXISTS store_migrations. langgraph's setup() doesn't use a Postgres
advisory lock, so concurrent first-callers collide on pg_type.

Fix:
- Cache the setup promise on globalThis so same-process workers
  (Turbopack worker_threads) coalesce to one CREATE TABLE.
- Swallow Postgres 23505 (unique_violation) so cross-process workers
  don't fail when another process wins the race.

Either path leaves the tables present. The previous PR #8 CI run
passed by luck (workers happened to start in serial); this fix makes
the build deterministic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant