From 500d6d6c6ecb0835e33c5ed6aa0ad07cf7018b5b Mon Sep 17 00:00:00 2001 From: Jahvon Dockery Date: Mon, 27 Jul 2026 14:52:07 -0400 Subject: [PATCH 1/4] docs: wire flow mcp as the default execution path The repo told Claude Code to prefer the flow MCP tools while making Bash the cheaper path: .mcp.json was untracked so a fresh clone had no flow server at all, CLAUDE.md led with shell commands and duplicated a staler copy of the flow-context skill, and the four task skills restricted allowed-tools to Bash(flow ...) with no mcp__flow__* entries at all. - commit .mcp.json so every clone gets the server - make CLAUDE.md defer to flow-context as the single source of truth and present executables by ref rather than as shell invocations - add mcp__flow__* to each skill's allowed-tools, and fix the frontmatter to the documented comma-separated form (was space-separated, and pr-ready used Bash(git *) instead of Bash(git:*)) - correct stale paths: handlers are in cmd/internal/, runner types are subpackages, schemas are types/executable/*_schema.yaml -> *.gen.go Also documents committing .mcp.json in the AI Tools guide, which is the step that turns one user's setup into a team's. Co-Authored-By: Claude Opus 5 (1M context) --- .claude/skills/new-command/SKILL.md | 16 +++--- .claude/skills/new-exec-type/SKILL.md | 18 ++++--- .claude/skills/pr-ready/SKILL.md | 6 ++- .claude/skills/validate/SKILL.md | 16 ++++-- .mcp.json | 9 ++++ CLAUDE.md | 76 +++++++++++++++------------ docs/guides/ai-tools.md | 20 +++++++ 7 files changed, 108 insertions(+), 53 deletions(-) create mode 100644 .mcp.json diff --git a/.claude/skills/new-command/SKILL.md b/.claude/skills/new-command/SKILL.md index a3737526..87b8ef00 100644 --- a/.claude/skills/new-command/SKILL.md +++ b/.claude/skills/new-command/SKILL.md @@ -3,19 +3,22 @@ name: new-command description: Scaffold a new Cobra CLI command following the project's patterns. disable-model-invocation: true argument-hint: " [noun] — what the command does" -allowed-tools: Bash(flow build:*) Bash(go build:*) Read +allowed-tools: mcp__flow__execute, mcp__flow__run_command, mcp__flow__list_executables, Bash(flow build:*), Bash(go build:*), Read --- Scaffold a new Cobra CLI command for: $ARGUMENTS Before writing any code, read a similar existing command to match the exact style: -- Simple verb commands: `cmd/exec.go` -- Noun/verb subcommands: any file under `cmd/workspace/` or `cmd/vault/` +- Simple verb commands: `cmd/internal/exec.go` +- Noun/verb subcommands: `cmd/internal/workspace.go` or `cmd/internal/vault.go` — each groups its + subcommands in a single file rather than a directory Then follow these patterns: -1. **File location**: `cmd/.go` for top-level, or `cmd//.go` for subcommands -2. **Command registration**: register in the parent command's `init()` or `cmd/root.go` +1. **File location**: `cmd/internal/.go`. Only `root.go` lives directly in `cmd/`; every + command handler is under `cmd/internal/`. Shared helpers go in `cmd/internal/helpers.go`, + flags in `cmd/internal/flags/`, output shaping in `cmd/internal/response/`. +2. **Command registration**: register on the parent command, or add to `rootCmd` in `cmd/root.go` 3. **Error handling**: - Runtime errors → `errhandler.HandleFatal(ctx, cmd, err)` - Flag/arg misuse → `errhandler.HandleUsage(ctx, cmd, "message", args...)` @@ -23,4 +26,5 @@ Then follow these patterns: 4. **Context**: resolve workspace context via `pkg/context` before delegating to `internal/services` 5. **Output**: respect `--output` flag (text/json/yaml) for structured responses -After scaffolding, verify it builds: `flow build binary ./bin/flow` +After scaffolding, verify it builds — prefer `mcp__flow__execute` with ref `build binary` and +argument `./bin/flow` over a raw shell call. diff --git a/.claude/skills/new-exec-type/SKILL.md b/.claude/skills/new-exec-type/SKILL.md index 86673052..444f7c21 100644 --- a/.claude/skills/new-exec-type/SKILL.md +++ b/.claude/skills/new-exec-type/SKILL.md @@ -3,26 +3,30 @@ name: new-exec-type description: Add a new executable type to the flow runner (a new kind of automation block users can define in .flow files). disable-model-invocation: true argument-hint: " — description of what this executable type does" -allowed-tools: Bash(flow generate:*) Bash(flow validate:*) Bash(flow build:*) Bash(go test:*) Read +allowed-tools: mcp__flow__execute, mcp__flow__run_command, mcp__flow__list_executables, Bash(flow generate:*), Bash(flow validate:*), Bash(flow build:*), Bash(go test:*), Read --- Add a new executable type to the flow runner for: $ARGUMENTS An "executable type" is a new automation block users can declare in `.flow` files (like `exec`, `serial`, `parallel`, `render`). Follow these steps in order: -1. **Schema first** — add the new type's fields to `types/executable/schema.yaml`. - Read the existing schema to match the structure. Run `flow generate` to regenerate `types/executable/generated.go`. +1. **Schema first** — add the new type's fields to `types/executable/executable_schema.yaml`. + Read the existing schema to match the structure. Regenerate with the `generate` executable + (`mcp__flow__execute`, ref `generate`) — it rewrites `types/executable/executable.gen.go`. + Never edit the `.gen.go` file directly. -2. **Runner handler** — create `internal/runner/.go` implementing the runner interface. - Read `internal/runner/exec.go` or `internal/runner/serial.go` as reference for the exact interface and pattern. +2. **Runner handler** — each type is its own package: create `internal/runner//.go`. + Read `internal/runner/exec/exec.go` or `internal/runner/serial/serial.go` as reference for the + exact interface and pattern. 3. **Register the type** — wire the new handler into `internal/runner/runner.go` (the dispatch table). 4. **Parser support** — update `internal/fileparser/` if needed to recognize and validate the new type during YAML parsing. -5. **Tests** — add unit tests in `internal/runner/_test.go` using Ginkgo. +5. **Tests** — add unit tests in `internal/runner//_test.go` using Ginkgo. Use `Describe`/`It`/`Entry` — never `FDescribe`/`FIt`. Cover happy path and error cases. -6. **Validate** — run `flow validate` to confirm generate, lint, and tests all pass. +6. **Validate** — run the `validate` executable (`mcp__flow__execute`, ref `validate`) to confirm + generate, lint, and tests all pass. Do not skip the schema step — editing generated files directly will cause CI to fail. diff --git a/.claude/skills/pr-ready/SKILL.md b/.claude/skills/pr-ready/SKILL.md index 24c0e842..f1c7553d 100644 --- a/.claude/skills/pr-ready/SKILL.md +++ b/.claude/skills/pr-ready/SKILL.md @@ -2,7 +2,7 @@ name: pr-ready description: Run a pre-PR readiness check and report READY or NOT READY. disable-model-invocation: true -allowed-tools: Bash(git *) Bash(flow validate:*) Bash(flow generate:*) Bash(go test:*) Read +allowed-tools: mcp__flow__execute, mcp__flow__run_command, mcp__flow__list_executables, Bash(git:*), Bash(flow validate:*), Bash(flow generate:*), Bash(go test:*), Read --- Check whether the current branch is ready to open a PR. Work through each item and report PASS or FAIL: @@ -10,7 +10,9 @@ Check whether the current branch is ready to open a PR. Work through each item a 1. **No focus markers** — `grep -rn "FDescribe\|FIt\|FEntry\|FContext\|FWhen" --include="*.go" .` Any match is a FAIL — these silently exclude all other tests in the suite. -2. **Validation passes** — run `flow validate`. All steps must pass. +2. **Validation passes** — run the `validate` executable via `mcp__flow__execute` (ref: `validate`). + All steps must pass. Use `mcp__flow__run_command` for the `git`/`grep` checks below so they land + in flow's history alongside it. 3. **No debug artifacts** — grep for `fmt.Println`, `spew.Dump` in `cmd/`, `internal/`, `pkg/`. Flag anything that looks like leftover debug output (not legitimate logging). diff --git a/.claude/skills/validate/SKILL.md b/.claude/skills/validate/SKILL.md index 0f2247c8..b1719c1d 100644 --- a/.claude/skills/validate/SKILL.md +++ b/.claude/skills/validate/SKILL.md @@ -1,16 +1,22 @@ --- name: validate description: Run flow validate and fix any failures. Invoke after completing a feature or bug fix to confirm the codebase is clean before committing. -allowed-tools: Bash(flow validate:*) Bash(flow generate:*) Bash(flow lint:*) Bash(flow test:*) Bash(go test:*) Read +allowed-tools: mcp__flow__execute, mcp__flow__run_command, mcp__flow__list_executables, mcp__flow__get_execution_logs, Bash(flow validate:*), Bash(flow generate:*), Bash(flow lint:*), Bash(flow test:*), Bash(go test:*), Read --- -Run `flow validate` — it runs these steps in order: `generate` → `lint` → `test` → `validate generated` (checks for uncommitted generated diffs in CI). +Run the `validate` executable via `mcp__flow__execute` (ref: `validate`) rather than a raw shell +call, so the run inherits workspace env/secrets and is captured in flow's history. + +It runs these steps in order: `generate` → `lint` → `test` → `validate generated` (checks for uncommitted generated diffs in CI). For each failure, diagnose and fix before moving on: -- **generate fails**: Schema syntax error in `types/*/schema.yaml` — read and fix the schema +- **generate fails**: Schema syntax error in the source schema — `types/executable/*_schema.yaml`, or `types/{config,workspace,common}/schema.yaml`. Read and fix the schema, never the `.gen.go` output. - **lint fails**: Read the golangci-lint output, fix each violation, re-run - **test fails**: Read the Ginkgo output, identify the failing spec, fix the root cause — do not skip or comment out tests -- **validate generated fails**: Generated files are out of sync — run `flow generate` and stage the regenerated files; this is always the fix +- **validate generated fails**: Generated files are out of sync — re-run the `generate` executable and stage the regenerated files; this is always the fix + +Use `mcp__flow__get_execution_logs` with `mine: true` to re-read output from a run instead of +re-running it. -Do not report done until `flow validate` exits 0 with all steps passing. +Do not report done until `validate` exits 0 with all steps passing. diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 00000000..19fec29f --- /dev/null +++ b/.mcp.json @@ -0,0 +1,9 @@ +{ + "mcpServers": { + "flow": { + "type": "stdio", + "command": "flow", + "args": ["mcp"] + } + } +} diff --git a/CLAUDE.md b/CLAUDE.md index 0bae3c5f..27567b33 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,20 +24,21 @@ Read these before touching any code: ``` flow/ -├── cmd/ # Cobra CLI entry points and command handlers +├── cmd/ # Cobra CLI. Only root.go lives here; handlers are in cmd/internal/ ├── pkg/ # Shared, importable packages │ ├── cache/ # Workspace and executable cache management │ ├── cli/ # Shared CLI helpers and flag definitions │ ├── context/ # Global app context (workspace, config, vault) │ ├── errors/ # Typed errors with machine-readable codes │ ├── filesystem/ # Path helpers and workspace file I/O +│ ├── imports/ # Cross-workspace flow file imports │ ├── logger/ # Structured logging │ └── store/ # Persistence layer interfaces and implementations ├── internal/ # App logic NOT exported outside the binary │ ├── fileparser/ # Flow file YAML parsing and validation │ ├── io/ # Terminal UI and output rendering (tuikit) │ ├── mcp/ # MCP server implementation (tools, resources) -│ ├── runner/ # Executable execution engine +│ ├── runner/ # Execution engine; one subpackage per executable type │ ├── services/ # Business logic orchestration layer │ ├── templates/ # Workflow template expansion │ ├── updater/ # Auto-update logic @@ -86,41 +87,44 @@ types/*/schema.yaml → go-jsonschema → types/*/generated.go (DO NOT EDIT) ## Development Workflow -The project uses flow itself for dev automation: - -```bash -flow build binary ./bin/flow # Build the CLI binary -flow test # Run all tests (unit + e2e) in parallel -flow lint # Run golangci-lint -flow generate # Run all code generation -flow validate # Full check: generate → lint → test → diff validation -flow install tools # Install/update Go tools -flow mcp # Start the MCP server -flow browse # TUI explorer for discovering executables -``` +The project uses flow itself for dev automation, and the flow MCP server is wired up in +`.mcp.json`. **Run work through the `mcp__flow__*` tools, not raw Bash** — see the +`flow-context` skill (`.claude/skills/flow-context/SKILL.md`), which is the authoritative +guide and loads automatically every session. -### Using flow's MCP Tools in Sessions +The short version: `mcp__flow__execute` for a named executable, `mcp__flow__run_command` for a +one-off shell command, `mcp__flow__run_executable` for an inline multi-step spec. Discover names +with `mcp__flow__list_executables` — don't assume them. -During a Claude Code session, prefer MCP tools over raw shell when possible — they respect workspace config and handle environment setup: +The main executables, by ref: -``` -mcp__flow__list_executables # Browse all available executables -mcp__flow__execute # Run an executable (e.g., ref: "test unit", ref: "lint") -mcp__flow__get_executable # Inspect a specific executable's definition -mcp__flow__get_info # Get current workspace context -mcp__flow__get_workspace # Get workspace details -``` +| Ref | What it does | +|-----|--------------| +| `build binary` | Build the CLI binary (pass `./bin/flow` as the output arg) | +| `test` | Run all tests (unit + e2e) in parallel | +| `test unit` / `test e2e` | Run one suite | +| `lint` | Run golangci-lint | +| `generate` | Run all code generation | +| `validate` | Full check: generate → lint → test → diff validation | +| `install tools` | Install/update Go tools | + +`flow browse` (TUI explorer) and `flow mcp` (start the server) are interactive and belong in a +real terminal, not a tool call. --- ## Testing -- **Unit tests** (`-tags=unit`): Fast, no binary needed. `go test -race -tags=unit ./...` -- **E2E tests** (`-tags=e2e`): Require the `flow` binary on PATH. Build first: `flow build binary ./bin/flow`, then `go test -race -tags=e2e ./tests/...` +Prefer the `test`, `test unit`, and `test e2e` executables via `mcp__flow__execute` — they set +build tags, env, and the binary path for you. The underlying commands are documented here only so +you can recognize what a failure is telling you. + +- **Unit tests** (`-tags=unit`): Fast, no binary needed. Underlying: `go test -race -tags=unit ./...` +- **E2E tests** (`-tags=e2e`): Require the `flow` binary on PATH. The `test e2e` executable builds it first; underlying: `go test -race -tags=e2e ./tests/...` - **Focusing tests**: Use `FDescribe`/`FIt`/`FEntry` temporarily to filter — **always remove before committing** - **Golden file updates**: Set `UPDATE_GOLDEN_FILES=true` when output changes are intentional -Run both together: `flow test` (parallel, handles tags and env automatically) +Run both suites together with the bare `test` ref (parallel, handles tags and env automatically). --- @@ -130,10 +134,11 @@ The project generates code from YAML schemas. **Always edit schemas, never gener | Source | Generated output | |--------|-----------------| -| `types/*/schema.yaml` | `types/**/*.go` | +| `types/executable/{executable,flowfile,template}_schema.yaml` | `types/executable/*.gen.go` | +| `types/{config,workspace,common}/schema.yaml` | that package's `*.gen.go` | | Go definitions | `docs/cli/*.md`, `docs/types/*.md` | -After any schema change: `flow generate` — CI runs `validate generated` which fails on uncommitted diffs. +After any schema change, run the `generate` executable — CI runs `validate generated`, which fails on uncommitted diffs. --- @@ -153,9 +158,10 @@ Available codes: `INVALID_INPUT`, `NOT_FOUND`, `EXECUTION_FAILED`, `TIMEOUT`, `C ## Common Pitfalls -- **Editing `types/*.go` directly** → CI fails on `validate generated`. Edit `types/*/schema.yaml` instead. -- **`go test ./...` without build tags** → most tests silently skip. Always use `-tags=unit` or `-tags=e2e`. -- **Running e2e tests without a built binary** → tests panic. Run `flow build binary ./bin/flow` first. +- **Editing `types/**/*.gen.go` directly** → CI fails on `validate generated`. Edit the source `*schema.yaml` instead. +- **`go test ./...` without build tags** → most tests silently skip. Use the `test` executables, or pass `-tags=unit` / `-tags=e2e`. +- **Running e2e tests without a built binary** → tests panic. Use the `test e2e` ref, which builds first. +- **Reaching for Bash when a flow executable exists** → the run loses workspace env/secrets and leaves no history entry. Check `mcp__flow__list_executables` first. - **Leaving `FDescribe`/`FIt` in committed code** → all other tests in that suite are silently excluded. - **Adding a Cobra command with bare `log.Fatal`** → breaks structured error output. Use `errhandler`. @@ -163,7 +169,7 @@ Available codes: `INVALID_INPUT`, `NOT_FOUND`, `EXECUTION_FAILED`, `TIMEOUT`, `C ## PR & Code Quality -Before marking a PR ready, run `flow validate` — it runs generate, lint, test, and checks for uncommitted generated diffs in one shot. +Before marking a PR ready, run the `validate` executable — it runs generate, lint, test, and checks for uncommitted generated diffs in one shot. - Commit messages: imperative mood, lowercase, ≤72 chars (`fix: ...`, `feat: ...`, `refactor: ...`) - No WIP code in PRs: remove all `FDescribe`/`FIt` focus markers, debug prints, and open TODOs @@ -175,7 +181,11 @@ Before marking a PR ready, run `flow validate` — it runs generate, lint, test, - **`flow.yaml`**: Workspace configuration for the flow repo itself - **`go.mod`**: Go dependencies and version (Go 1.25+) - **`.execs/`**: flow dev workflow definitions (build, test, lint, release, etc.) -- **`.claude/settings.local.json`**: Claude Code permission allowlist for this project +- **`.mcp.json`**: registers the flow MCP server (`flow mcp`) — committed so every clone gets it +- **`.claude/settings.json`**: Claude Code permission allowlist for this project (committed). + `.claude/settings.local.json` is gitignored and holds per-user overrides only. +- **`.claude/skills/`**: project skills. `flow-context` loads every session; `validate`, + `pr-ready`, `new-command`, and `new-exec-type` are user-invoked via `/`. ## Development Setup diff --git a/docs/guides/ai-tools.md b/docs/guides/ai-tools.md index 5cf6c500..bb8070de 100644 --- a/docs/guides/ai-tools.md +++ b/docs/guides/ai-tools.md @@ -27,6 +27,26 @@ Add this to your MCP client configuration (Claude Code, Cursor, Cline, or any MC The server runs over stdio. That's the entire setup. +**Commit it to your repo.** Claude Code and Cursor both read a `.mcp.json` at the repository root, +so checking that file in means every teammate — and every fresh clone — gets flow's tools without +per-person setup. Put the snippet above in `.mcp.json` and commit it: + +```json title=".mcp.json" +{ + "mcpServers": { + "flow": { + "type": "stdio", + "command": "flow", + "args": ["mcp"] + } + } +} +``` + +Each user still approves the server on first use, so committing it grants no access on its own — +it just removes the setup step. Pair it with the skill in [Wiring It Up](#wiring-it-up-for-your-project) +below: the `.mcp.json` supplies the tools, the skill tells the assistant to reach for them. + ### What's available **Tools** From 4950b7b3b49bfa596aed7038f376bfb807963806 Mon Sep 17 00:00:00 2001 From: Jahvon Dockery Date: Mon, 27 Jul 2026 15:08:41 -0400 Subject: [PATCH 2/4] chore(claude): enforce repo rules via permissions, document sibling deps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Critical Rule #1 ("never edit generated files") and the no-push convention were instructions the model could simply not follow. Move them into permission rules, which deny takes precedence over and no local setting can override. - deny writes to types/**/*.gen.go and the generated docs trees - deny secret-leaking reads: flow secret get/list, env dumps, .env files - ask on push, force-reset, gh pr/issue/release, publish, docker push - document tuikit and vault as the two first-party modules that carry most behavior, and the version-skew trap: no replace directives, so a sibling working copy is often at a different version than the build - state the scope boundary explicitly — flow provides AI tools via MCP and does not consume an LLM in the CLI Machine-specific paths (additionalDirectories for the module cache and sibling checkouts) stay in the gitignored settings.local.json so the committed file remains portable. Co-Authored-By: Claude Opus 5 (1M context) --- .claude/settings.json | 35 +++++++++++++++++++++++++++++++--- CLAUDE.md | 44 ++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 73 insertions(+), 6 deletions(-) diff --git a/.claude/settings.json b/.claude/settings.json index 520e716b..08f7954a 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -15,8 +15,12 @@ "Bash(flow browse:*)", "Bash(flow mcp:*)", "Bash(flow exec:*)", - "Bash(node:*)", - "Bash(npm view:*)", + "mcp__flow__run_command", + "mcp__flow__run_executable", + "mcp__flow__get_execution_logs", + "mcp__flow__sync_executables", + "mcp__flow__write_flowfile", + "mcp__flow__get_workspace_config", "mcp__flow__list_executables", "mcp__flow__get_info", "mcp__flow__get_executable", @@ -24,6 +28,31 @@ "mcp__flow__list_workspaces", "mcp__flow__execute" ], - "deny": [] + "ask": [ + "Bash(git push:*)", + "Bash(git reset --hard:*)", + "Bash(git clean:*)", + "Bash(gh pr create:*)", + "Bash(gh issue create:*)", + "Bash(gh release:*)", + "Bash(flow publish:*)", + "Bash(docker push:*)" + ], + "deny": [ + "Edit(types/**/*.gen.go)", + "Write(types/**/*.gen.go)", + "Edit(docs/cli/**)", + "Write(docs/cli/**)", + "Edit(docs/types/**)", + "Write(docs/types/**)", + "Edit(docs/public/schemas/**)", + "Write(docs/public/schemas/**)", + "Bash(flow secret get:*)", + "Bash(flow secret list:*)", + "Bash(env)", + "Bash(printenv:*)", + "Read(./.env)", + "Read(./.env.*)" + ] } } diff --git a/CLAUDE.md b/CLAUDE.md index 27567b33..170259d7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -66,13 +66,21 @@ cmd/ (Cobra command) → pkg/context (workspace + config resolution) **Type generation pipeline:** ``` -types/*/schema.yaml → go-jsonschema → types/*/generated.go (DO NOT EDIT) +types/*/*schema.yaml → go-jsonschema → types/*/*.gen.go (DO NOT EDIT) → internal/fileparser (YAML parsing and validation) ``` **MCP server:** `internal/mcp` exposes the same execution pipeline to AI tools over the Model Context Protocol. The `flow mcp` command starts the server. Claude Code, Cursor, and other MCP clients can call `mcp__flow__*` tools to run executables directly. +**Scope boundary — flow is an AI *tool provider*, not an AI *consumer*.** +The core's job is to expose deterministic, well-described capabilities: the MCP server, the +published JSON schemas, and `llms.txt`. Do not add LLM calls, natural-language command parsing, +or AI-driven generation *into* the CLI — that would put vendor API keys, per-call cost, and +non-deterministic output in the critical path of a task runner. Anything that wants to apply a +model to flow does so from the outside, by consuming the MCP surface. Treat proposals to add +"AI features" to the CLI itself as out of scope for this repo. + --- ## Key Technologies @@ -85,6 +93,32 @@ types/*/schema.yaml → go-jsonschema → types/*/generated.go (DO NOT EDIT) --- +## Sibling Repositories + +Two first-party Go modules carry a large share of this repo's behavior, and the seam between +them is where most breakage happens: + +| Module | Owns | +|--------|------| +| `github.com/flowexec/tuikit` | All TUI rendering — `flow browse`, the logs view, interactive prompts. Bugs that look like rendering or input glitches usually live here, not in `internal/io`. | +| `github.com/flowexec/vault` | Secret storage providers (AES-256, age, keyring, env passthrough). `internal/vault` is a thin type-alias wrapper over it. | + +Related repos in the same org, not imported here: `action` (GitHub Action), `examples` +(workspaces users can `flow workspace add`), `homebrew-tap` (distribution). + +**Read the pinned version, not a local checkout.** There are no `replace` directives — builds +use the versions pinned in `go.mod`, which resolve to the module cache +(`$(go env GOMODCACHE)/github.com/flowexec/@`). A sibling working copy is often on +a feature branch at a *different* version than what compiles, so answering an integration +question from it produces confidently wrong results. Use the module cache for "what does the +code I build against actually do", and a local checkout only when deliberately co-developing an +upstream change. `go doc` and `go list -m` are the quickest way to check what's actually pinned. + +Upgrading either dependency tends to be a breaking-change adaptation rather than a version bump; +check that module's release notes before assuming an API is unchanged. + +--- + ## Development Workflow The project uses flow itself for dev automation, and the flow MCP server is wired up in @@ -182,8 +216,12 @@ Before marking a PR ready, run the `validate` executable — it runs generate, l - **`go.mod`**: Go dependencies and version (Go 1.25+) - **`.execs/`**: flow dev workflow definitions (build, test, lint, release, etc.) - **`.mcp.json`**: registers the flow MCP server (`flow mcp`) — committed so every clone gets it -- **`.claude/settings.json`**: Claude Code permission allowlist for this project (committed). - `.claude/settings.local.json` is gitignored and holds per-user overrides only. +- **`.claude/settings.json`**: Claude Code permissions for this project (committed). `deny` blocks + edits to generated output and reads of secret material; `ask` gates pushes, releases, and other + outward-facing actions. Keep it free of machine-specific absolute paths — it ships to everyone. +- **`.claude/settings.local.json`**: gitignored, per-user. The right home for absolute paths, such + as `permissions.additionalDirectories` pointing at the module cache and any sibling checkouts + (see [Sibling Repositories](#sibling-repositories)). - **`.claude/skills/`**: project skills. `flow-context` loads every session; `validate`, `pr-ready`, `new-command`, and `new-exec-type` are user-invoked via `/`. From f20cdb5812d8ff1362de990ab001fe5054fa6299 Mon Sep 17 00:00:00 2001 From: Jahvon Dockery Date: Mon, 27 Jul 2026 15:13:45 -0400 Subject: [PATCH 3/4] docs: trim CLAUDE.md and fix the generated-files rule CLAUDE.md is the only file loaded in full every session, so it was the one place worth optimizing. Skill bodies load on invocation, not at startup, so their length is nearly free. Critical Rule #1 claimed all of types/**/*.go was generated. Only the three *.gen.go files are; eight hand-written files sit alongside them, so the rule told the model not to edit code it should edit. - fix rule #1 to name *.gen.go, and point at the real schema sources - merge Common Pitfalls into Critical Rules; they overlapped ~60% - fold the Code Generation section into rule #1, now that permission rules enforce it rather than prose - drop Key Technologies (version is in setup; the rest is evident from imports) and collapse the package tree annotations - correct the claim that the flow-context skill auto-loads each session Net 13% smaller than before the sibling-repo and scope sections were added, 42% below peak. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 246 ++++++++++++++++++------------------------------------ 1 file changed, 81 insertions(+), 165 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 170259d7..a6a4754e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,22 +1,26 @@ # flow repo — Claude Code Context -## Project Overview - -**flow** is a workflow automation hub for organizing automation across multiple projects (workspaces) with built-in secrets, templates, and cross-workspace composition. Users define workflows in YAML flow files, discover them visually, and run them anywhere. - -This repo contains the flow CLI (Go). flow itself is used for all dev automation — build, test, generate, lint, release — via `.execs/*.flow` files. +**flow** is a workflow automation hub: automation organized across projects (workspaces), with +built-in secrets, templates, and cross-workspace composition. Users define workflows in YAML flow +files and run them anywhere. This repo is the flow CLI (Go), and flow runs its own dev automation +via `.execs/*.flow`. --- ## Critical Rules -Read these before touching any code: - -1. **NEVER edit generated files** — `types/**/*.go`, `docs/cli/*.md`, `docs/types/*.md` are all auto-generated. Edit the schema source, not the output. -2. **Run `flow generate` after any schema change** in `types/*/schema.yaml` — CI will reject uncommitted generated diffs. -3. **Remove test focus markers before committing** — `FDescribe`, `FIt`, `FEntry` are temporary debugging tools, never ship them. -4. **Never use `logger.Log().FatalErr()` in `cmd/`** — use `errhandler.HandleFatal(ctx, cmd, err)` instead. -5. **`pkg/` is the stable API surface; `internal/` is unexported** — packages in `pkg/` may be imported outside the binary; `internal/` may not. +1. **Only `*.gen.go` is generated — the rest of `types/` is hand-written.** Never edit + `types/**/*.gen.go`, `docs/cli/`, `docs/types/`, or `docs/public/schemas/`; edit the source + schema and run the `generate` executable. Sources are `types/executable/*_schema.yaml` and + `types/{config,workspace,common}/schema.yaml`. CI's `validate generated` fails on uncommitted + diffs. (Permission rules block these writes, so a failed edit here is expected, not a bug.) +2. **Remove `FDescribe`/`FIt`/`FEntry` before committing** — they silently exclude every other + spec in the suite. +3. **In `cmd/`, never `log.Fatal`, `os.Exit`, or `logger.Log().FatalErr()`** — use + `errhandler.HandleFatal(ctx, cmd, err)`, or `HandleUsage` for flag/arg misuse. +4. **`pkg/` is importable API surface; `internal/` is not.** +5. **Run work through flow's MCP tools, not raw Bash** — see Development Workflow. +6. **`go test ./...` without build tags silently skips most tests.** Use the `test` refs. --- @@ -24,210 +28,122 @@ Read these before touching any code: ``` flow/ -├── cmd/ # Cobra CLI. Only root.go lives here; handlers are in cmd/internal/ -├── pkg/ # Shared, importable packages -│ ├── cache/ # Workspace and executable cache management -│ ├── cli/ # Shared CLI helpers and flag definitions -│ ├── context/ # Global app context (workspace, config, vault) -│ ├── errors/ # Typed errors with machine-readable codes -│ ├── filesystem/ # Path helpers and workspace file I/O -│ ├── imports/ # Cross-workspace flow file imports -│ ├── logger/ # Structured logging -│ └── store/ # Persistence layer interfaces and implementations -├── internal/ # App logic NOT exported outside the binary -│ ├── fileparser/ # Flow file YAML parsing and validation -│ ├── io/ # Terminal UI and output rendering (tuikit) -│ ├── mcp/ # MCP server implementation (tools, resources) +├── cmd/ # Cobra CLI. Only root.go here; handlers in cmd/internal/ +├── pkg/ # Importable: cache, cli, context, errors, filesystem, +│ # imports, logger, store +├── internal/ # Not importable outside the binary +│ ├── io/ # Terminal UI and output rendering (wraps tuikit) +│ ├── mcp/ # MCP server (tools, resources) │ ├── runner/ # Execution engine; one subpackage per executable type -│ ├── services/ # Business logic orchestration layer +│ ├── services/ # Business logic orchestration │ ├── templates/ # Workflow template expansion -│ ├── updater/ # Auto-update logic -│ ├── utils/ # Internal utilities -│ ├── validation/ # Schema and config validation -│ ├── vault/ # Secret management -│ └── version/ # Build version info -├── types/ # Generated Go types from YAML schemas — DO NOT EDIT -├── tests/ # E2E test suite (Ginkgo, -tags=e2e) -├── docs/ # Documentation source (flowexec.io) — CLI/type docs are generated -├── tools/ # Code generation and build tooling -└── .execs/ # flow dev automation executables (build, test, lint, release) +│ ├── vault/ # Thin wrapper over the vault module +│ └── ... # fileparser, updater, utils, validation, version +├── types/ # Schemas + generated types (*.gen.go) + hand-written helpers +├── tests/ # E2E suite (Ginkgo, -tags=e2e) +├── docs/ # flowexec.io source; docs/cli and docs/types are generated +└── .execs/ # flow's own dev automation ``` --- ## Architecture -**CLI execution path:** -``` -cmd/ (Cobra command) → pkg/context (workspace + config resolution) - → internal/services (business logic) → internal/runner (execution engine) - → type-specific handler in internal/runner/ -``` - -**Type generation pipeline:** ``` -types/*/*schema.yaml → go-jsonschema → types/*/*.gen.go (DO NOT EDIT) - → internal/fileparser (YAML parsing and validation) +cmd/internal (Cobra) → pkg/context (workspace + config) → internal/services + → internal/runner → type-specific subpackage ``` -**MCP server:** -`internal/mcp` exposes the same execution pipeline to AI tools over the Model Context Protocol. The `flow mcp` command starts the server. Claude Code, Cursor, and other MCP clients can call `mcp__flow__*` tools to run executables directly. +`internal/mcp` exposes that same pipeline over the Model Context Protocol; `flow mcp` starts the +server. -**Scope boundary — flow is an AI *tool provider*, not an AI *consumer*.** -The core's job is to expose deterministic, well-described capabilities: the MCP server, the -published JSON schemas, and `llms.txt`. Do not add LLM calls, natural-language command parsing, -or AI-driven generation *into* the CLI — that would put vendor API keys, per-call cost, and -non-deterministic output in the critical path of a task runner. Anything that wants to apply a -model to flow does so from the outside, by consuming the MCP surface. Treat proposals to add -"AI features" to the CLI itself as out of scope for this repo. - ---- - -## Key Technologies - -### Go CLI -- **Language**: Go 1.25+ (`go.mod:3`) -- **CLI Framework**: Cobra (`github.com/spf13/cobra`) -- **TUI**: Custom tuikit (`github.com/flowexec/tuikit`) built on Bubble Tea -- **Testing**: Ginkgo v2 BDD framework (`github.com/onsi/ginkgo/v2`) +**Scope boundary — flow is an AI *tool provider*, not an AI *consumer*.** The core exposes +deterministic capabilities (MCP server, published JSON schemas, `llms.txt`). Do not add LLM calls, +natural-language command parsing, or AI generation *into* the CLI — that puts vendor keys, +per-call cost, and non-determinism in a task runner's critical path. Anything applying a model to +flow does so from outside, via the MCP surface. Treat "add AI features to the CLI" as out of scope. --- ## Sibling Repositories -Two first-party Go modules carry a large share of this repo's behavior, and the seam between -them is where most breakage happens: +Two first-party modules carry much of this repo's behavior, and their seam is where most breakage +happens: -| Module | Owns | -|--------|------| -| `github.com/flowexec/tuikit` | All TUI rendering — `flow browse`, the logs view, interactive prompts. Bugs that look like rendering or input glitches usually live here, not in `internal/io`. | -| `github.com/flowexec/vault` | Secret storage providers (AES-256, age, keyring, env passthrough). `internal/vault` is a thin type-alias wrapper over it. | +- **`flowexec/tuikit`** — all TUI rendering (`flow browse`, logs view, prompts). Apparent + rendering or input bugs usually live here, not in `internal/io`. +- **`flowexec/vault`** — secret storage providers (AES-256, age, keyring, env). `internal/vault` + is a thin type-alias wrapper. -Related repos in the same org, not imported here: `action` (GitHub Action), `examples` -(workspaces users can `flow workspace add`), `homebrew-tap` (distribution). +**Read the pinned version, not a local checkout.** There are no `replace` directives, so builds +use the `go.mod` versions from `$(go env GOMODCACHE)/github.com/flowexec/@`. A +sibling working copy is often on a feature branch at a *different* version than what compiles, so +answering an integration question from it yields confidently wrong results. Use a checkout only +when deliberately co-developing upstream. Upgrades here are usually breaking-change adaptations, +not version bumps — check release notes before assuming an API is unchanged. -**Read the pinned version, not a local checkout.** There are no `replace` directives — builds -use the versions pinned in `go.mod`, which resolve to the module cache -(`$(go env GOMODCACHE)/github.com/flowexec/@`). A sibling working copy is often on -a feature branch at a *different* version than what compiles, so answering an integration -question from it produces confidently wrong results. Use the module cache for "what does the -code I build against actually do", and a local checkout only when deliberately co-developing an -upstream change. `go doc` and `go list -m` are the quickest way to check what's actually pinned. - -Upgrading either dependency tends to be a breaking-change adaptation rather than a version bump; -check that module's release notes before assuming an API is unchanged. +Same org, not imported: `action` (GitHub Action), `examples`, `homebrew-tap`. --- ## Development Workflow -The project uses flow itself for dev automation, and the flow MCP server is wired up in -`.mcp.json`. **Run work through the `mcp__flow__*` tools, not raw Bash** — see the -`flow-context` skill (`.claude/skills/flow-context/SKILL.md`), which is the authoritative -guide and loads automatically every session. - -The short version: `mcp__flow__execute` for a named executable, `mcp__flow__run_command` for a -one-off shell command, `mcp__flow__run_executable` for an inline multi-step spec. Discover names -with `mcp__flow__list_executables` — don't assume them. +flow's MCP server is wired up in `.mcp.json`. **Prefer `mcp__flow__*` over raw Bash** so runs get +workspace env/secrets and land in flow's history. The `flow-context` skill has the full guidance. -The main executables, by ref: +`mcp__flow__execute` runs a named executable, `run_command` a one-off shell command, +`run_executable` an inline multi-step spec. Discover names with `mcp__flow__list_executables` — +don't assume them. | Ref | What it does | |-----|--------------| -| `build binary` | Build the CLI binary (pass `./bin/flow` as the output arg) | -| `test` | Run all tests (unit + e2e) in parallel | -| `test unit` / `test e2e` | Run one suite | -| `lint` | Run golangci-lint | -| `generate` | Run all code generation | -| `validate` | Full check: generate → lint → test → diff validation | +| `build binary` | Build the CLI (pass `./bin/flow` as the output arg) | +| `test` | All tests (unit + e2e), parallel | +| `test unit` / `test e2e` | One suite | +| `lint` | golangci-lint | +| `generate` | All code generation | +| `validate` | generate → lint → test → generated-diff check | | `install tools` | Install/update Go tools | -`flow browse` (TUI explorer) and `flow mcp` (start the server) are interactive and belong in a -real terminal, not a tool call. - ---- - -## Testing +`flow browse` and `flow mcp` are interactive — they belong in a real terminal, not a tool call. -Prefer the `test`, `test unit`, and `test e2e` executables via `mcp__flow__execute` — they set -build tags, env, and the binary path for you. The underlying commands are documented here only so -you can recognize what a failure is telling you. - -- **Unit tests** (`-tags=unit`): Fast, no binary needed. Underlying: `go test -race -tags=unit ./...` -- **E2E tests** (`-tags=e2e`): Require the `flow` binary on PATH. The `test e2e` executable builds it first; underlying: `go test -race -tags=e2e ./tests/...` -- **Focusing tests**: Use `FDescribe`/`FIt`/`FEntry` temporarily to filter — **always remove before committing** -- **Golden file updates**: Set `UPDATE_GOLDEN_FILES=true` when output changes are intentional - -Run both suites together with the bare `test` ref (parallel, handles tags and env automatically). - ---- - -## Code Generation - -The project generates code from YAML schemas. **Always edit schemas, never generated output.** - -| Source | Generated output | -|--------|-----------------| -| `types/executable/{executable,flowfile,template}_schema.yaml` | `types/executable/*.gen.go` | -| `types/{config,workspace,common}/schema.yaml` | that package's `*.gen.go` | -| Go definitions | `docs/cli/*.md`, `docs/types/*.md` | - -After any schema change, run the `generate` executable — CI runs `validate generated`, which fails on uncommitted diffs. +**Testing notes:** e2e needs the binary on PATH (the `test e2e` ref builds it first). Set +`UPDATE_GOLDEN_FILES=true` when output changes are intentional. --- ## Error Handling -The CLI surfaces a structured JSON/YAML error envelope (`{"error":{"code","message","details"}}`) on stderr when the user passes `--output json` or `--output yaml`, and plain-text otherwise. Both paths go through `cmd/internal/errors.HandleFatal`. +The CLI emits a structured error envelope (`{"error":{"code","message","details"}}`) on stderr for +`--output json|yaml`, plain text otherwise. Both paths go through `cmd/internal/errors.HandleFatal`. -**In `cmd/` handlers:** -- Use `errhandler.HandleFatal(ctx, cmd, err)` — not `logger.Log().FatalErr(err)` -- Use `errhandler.HandleUsage(ctx, cmd, "...", args...)` for flag/arg misuse → callers see `INVALID_INPUT` + exit 2 - -**Typed errors in `pkg/errors/errors.go`** implement `Code() string`. Extend that set rather than returning bare `fmt.Errorf` when a stable machine-readable code matters. - -Available codes: `INVALID_INPUT`, `NOT_FOUND`, `EXECUTION_FAILED`, `TIMEOUT`, `CANCELLED`, `VALIDATION_FAILED`, `INTERNAL_ERROR`, `PERMISSION_DENIED` - ---- - -## Common Pitfalls - -- **Editing `types/**/*.gen.go` directly** → CI fails on `validate generated`. Edit the source `*schema.yaml` instead. -- **`go test ./...` without build tags** → most tests silently skip. Use the `test` executables, or pass `-tags=unit` / `-tags=e2e`. -- **Running e2e tests without a built binary** → tests panic. Use the `test e2e` ref, which builds first. -- **Reaching for Bash when a flow executable exists** → the run loses workspace env/secrets and leaves no history entry. Check `mcp__flow__list_executables` first. -- **Leaving `FDescribe`/`FIt` in committed code** → all other tests in that suite are silently excluded. -- **Adding a Cobra command with bare `log.Fatal`** → breaks structured error output. Use `errhandler`. +Typed errors in `pkg/errors/errors.go` implement `Code() string`. Extend that set rather than +returning bare `fmt.Errorf` when a stable machine-readable code matters. Codes: `INVALID_INPUT`, +`NOT_FOUND`, `EXECUTION_FAILED`, `TIMEOUT`, `CANCELLED`, `VALIDATION_FAILED`, `INTERNAL_ERROR`, +`PERMISSION_DENIED`. --- ## PR & Code Quality -Before marking a PR ready, run the `validate` executable — it runs generate, lint, test, and checks for uncommitted generated diffs in one shot. - -- Commit messages: imperative mood, lowercase, ≤72 chars (`fix: ...`, `feat: ...`, `refactor: ...`) -- No WIP code in PRs: remove all `FDescribe`/`FIt` focus markers, debug prints, and open TODOs +Run the `validate` executable before marking a PR ready. Commit messages: imperative, lowercase, +≤72 chars (`fix:`, `feat:`, `refactor:`). No focus markers, debug prints, or open TODOs. --- ## Configuration Files -- **`flow.yaml`**: Workspace configuration for the flow repo itself -- **`go.mod`**: Go dependencies and version (Go 1.25+) -- **`.execs/`**: flow dev workflow definitions (build, test, lint, release, etc.) -- **`.mcp.json`**: registers the flow MCP server (`flow mcp`) — committed so every clone gets it -- **`.claude/settings.json`**: Claude Code permissions for this project (committed). `deny` blocks - edits to generated output and reads of secret material; `ask` gates pushes, releases, and other - outward-facing actions. Keep it free of machine-specific absolute paths — it ships to everyone. -- **`.claude/settings.local.json`**: gitignored, per-user. The right home for absolute paths, such - as `permissions.additionalDirectories` pointing at the module cache and any sibling checkouts - (see [Sibling Repositories](#sibling-repositories)). -- **`.claude/skills/`**: project skills. `flow-context` loads every session; `validate`, - `pr-ready`, `new-command`, and `new-exec-type` are user-invoked via `/`. +- **`flow.yaml`** — this repo's workspace config +- **`.execs/`** — flow's dev automation definitions +- **`.mcp.json`** — registers the flow MCP server; committed so every clone gets it +- **`.claude/settings.json`** — committed permissions. `deny` blocks generated-file writes and + secret reads; `ask` gates pushes and releases. Keep absolute paths out — it ships to everyone. +- **`.claude/settings.local.json`** — gitignored, per-user. Where absolute paths belong, e.g. + `permissions.additionalDirectories` for the module cache and sibling checkouts. ## Development Setup -1. Prerequisites: Go 1.25+, flow CLI installed +1. Go 1.25+, flow CLI installed 2. `flow workspace add flow . --set` 3. `flow install tools` 4. `flow validate` From d6d695fb559c73cf892283c0ab18d1d96afdb56f Mon Sep 17 00:00:00 2001 From: Jahvon Dockery Date: Mon, 27 Jul 2026 15:17:05 -0400 Subject: [PATCH 4/4] docs: require a branch per change, never commit on main Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index a6a4754e..5783599b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,6 +21,9 @@ via `.execs/*.flow`. 4. **`pkg/` is importable API surface; `internal/` is not.** 5. **Run work through flow's MCP tools, not raw Bash** — see Development Workflow. 6. **`go test ./...` without build tags silently skips most tests.** Use the `test` refs. +7. **Never commit directly on `main`.** Before the first commit of any change, create a branch + (`git switch -c /`) so the work lands as a PR. If you're already on `main` with + commits, move them onto a branch and reset `main` back to `origin/main`. ---