Skip to content

Audit remediation: docs/config truth, CI gates, silent-failure wiring, trust-layer completion - #72

Open
QuintinBotes wants to merge 47 commits into
mainfrom
audit-remediation
Open

Audit remediation: docs/config truth, CI gates, silent-failure wiring, trust-layer completion#72
QuintinBotes wants to merge 47 commits into
mainfrom
audit-remediation

Conversation

@QuintinBotes

Copy link
Copy Markdown
Owner

Summary

47 commits closing every fixable gap from the 2026-07-20 whole-project audit. Executed subagent-driven: each task implemented by a fresh agent, adversarially reviewed against its brief, with mandatory fix loops; a final whole-branch review verified the cross-task seams and returned ready to merge.

Phase 1 — Docs & config truth

  • .env.example/docs now document the real FORGE_MODEL_* BYOK contract; worker logs a loud warning before falling back to the offline scripted client
  • Doc-drift sweep: FORGE_SECRET_KEY in quickstart, shipped-feature claims (marketplace publish, leaderboard, OIDC), 20-package maps, compose tables
  • CHANGELOG + RELEASE_READINESS regenerated at HEAD; docs/trust-layer.md written from code for all four trust-layer features; nginx reverse-proxy config + guide; FORGE_SPEC realigned with shipped reality

Phase 2 — CI gates what exists

  • Web vitest suite (now 820 tests) + tsc --noEmit run on every PR
  • mypy covers all 23 workspace modules (4 were silently skipped; 23 errors fixed properly)
  • pytest basename-collision guard (the known exit-2 gotcha); schema-drift gate via alembic compare_metadata (surfaced and resolved 8 pre-existing drift items); dead-config hygiene + release.yml draft input wired tag-push-safe

Phase 3 — Silent-failure wiring

  • Slack /forge status returns live run status; legacy in-memory /approval router retired
  • Spec review Reject/Request-changes persist end-to-end (engine transitions → API → web)
  • Supervised multi-agent mode actually dispatches forge_coordinator (was silently single-agent)
  • PM webhooks (Jira/Linear + 7 more) now sync to the board: adapter re-fetch → workspace-scoped upsert, three idempotency layers, real Celery retries for transient faults

Phase 4 — Trust-layer completion

Discovered along the way (fixed on this branch)

  • Helm chart shipped the dead MODEL_PROVIDER_KEY var → k8s deploys silently ran the scripted model; now wired to the real contract with an install-time blank-key warning
  • Realtime websockets fell through Caddy's catch-all to the web app → explicit /ws routes both proxies; mcp-gateway wasn't on the edge network at all
  • Web client baked ws://localhost:8000/http://localhost:8000 into deployed bundles → same-origin runtime fallbacks for both WS and HTTP
  • Compose never passed the model/provider env vars (or MULTI_AGENT_ENABLED) into containers → passthrough added in both compose files + Helm knob

Verification

ruff clean · mypy 648 files clean · 4831 tests collect cleanly · web 820/820 + production build · deploy/tests 106 passed · helm lint/template/unittest 41/41 · per-task targeted suites green throughout (evidence in each task's review). Postgres-gated lanes run in CI.

Known follow-ups (triaged non-blocking by the final review)

forge_db SpecStatus enum sync, suite-version lexicographic ordering, F36 viewer positive-read test, URL-helper consolidation, eslint argsIgnorePattern, FORGE_SPEC deployment-snippet sweep, PT-14 punch-list entry for the Slack UUID-boundary residual, Slack approval interactivity → F36 migration.

🤖 Generated with Claude Code

Forge Swarm and others added 30 commits July 20, 2026 22:29
…d-client fallback

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ration

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y under mypy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ted, reverted)

--import-mode=importlib breaks whole-repo collection (exit 2, 51 errors):
importlib mode does not put each test dir on sys.path, so bare sibling
imports fail (41x conftest, _provider_fakes, _sandbox_fakes, mcp_fakes,
cost_support, helm_chart_lib, _helpers, _mp_helpers). Guard test ships
with prepend-mode semantics matching CI. Full importlib migration needs
repackaging ~50 sibling-helper imports first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add packages/db/tests/test_schema_drift.py: upgrade a fresh Postgres to
head and assert alembic.autogenerate.compare_metadata() returns an empty
diff against Base.metadata, using the exact autogenerate settings env.py
configures (compare_type=True; compare_server_default unset; no
include_object). Mirrors test_migration.py's @pytest.mark.postgres
fixture/skip conventions (scratch throwaway DB, FORGE_TEST_DATABASE_URL).
A second sanity test proves the comparison is drift-sensitive by diffing
against an empty MetaData().

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…enum

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…parity

Item 1 (websocket routing through Caddy) — INVESTIGATED, real gap, fixed.
Outcome: the web client opens a BARE /ws (never /api/ws). The WS URL is built
purely from NEXT_PUBLIC_WS_URL (default ws://localhost:8000/ws) with no /api
prefix and no same-origin/window.location derivation
(apps/web/src/lib/realtime/use-board-realtime.ts:55-56,
use-spec-collab.ts:133-136); the API mounts them rootless
(apps/api/forge_api/main.py:138-140 -> routers/realtime.py:75,109). NEXT_PUBLIC_WS_URL
is set in no compose/env/Dockerfile/doc, so a same-origin operator value is
wss://<domain>/ws. The pre-fix Caddyfile had no /ws route, so bare /ws fell
through to the web:3000 catch-all -> realtime silently dead behind Caddy. Added
`handle /ws` + `handle /ws/spec/*` -> api:8000, mirroring deploy/nginx/forge.conf;
documented the canonical route + NEXT_PUBLIC_WS_URL contract in
docs/self-hosting/reverse-proxy.md.

Item 2 (mcp-gateway unreachable via proxies) — verified, fixed. mcp-gateway was
on [backend, mcp] while caddy is on [edge] only: no shared network, so /mcp/*
would 502. Added `edge` to mcp-gateway (now [edge, backend, mcp]; mcp/backend
preserved). docker-compose.dev.yml has no networks block (single default net),
so no gap there. `docker compose config -q` passes for both files; deploy/tests
green (103 passed, 2 pre-existing skips), incl. two new regression tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s unset or relative

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…B-backed /approvals

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Forge Swarm and others added 17 commits July 21, 2026 03:10
The deployment task module defined `forge.deployments.advance` and
`forge.deployments.request` but was absent from `celery_app`'s `include`
list, so nothing imported it at worker boot — the tasks were unregistered
dead code. Add `forge_worker.tasks.deployments` to `include`.

Decision rule (kept the sync path — no enqueue wired): `forge.deployments.request`
merely duplicates what `DeploymentService.request_deployment` already does
synchronously in-process (create + `DeploymentOrchestrator.advance()`). Evidence
of no surfaced retry/backoff benefit: no `.delay`/`.apply_async` call-site for
these tasks anywhere, no beat-schedule entry references them, and no code
references the task-name constants outside the task module. So we register +
test the tasks so they drive `DeploymentOrchestrator.advance()` correctly when
invoked, and leave the synchronous API path untouched.

TDD: RED — the module was not in `celery_app.conf.include`, so the tasks did
not resolve via the worker's boot path (`loader.import_default_modules()`).
GREEN after registration. Added behaviour tests that invoking
`request_deployment_task` / `advance_deployment` drives
`DeploymentOrchestrator.advance()` on the target deployment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o-end

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dinator

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…panel

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tatus

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ntainers

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant