diff --git a/CHANGELOG.md b/CHANGELOG.md index d92f3d0..068bce4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added +- `mix loop` drives an autonomous feature/debug AI loop against a GitHub issue: it classifies the issue as a feature or a bug, repeatedly invokes `claude` to implement or fix it, verifies each attempt with `mix quality`, feeds failures back into the next attempt until the suite is green (or the iteration budget is spent), then commits, pushes, opens a PR closing the issue, and comments the outcome back on the issue - Add sev upgrade as an alias for sev update - Add research doc cataloging ways past the macOS interrupted-shutdown dialog - Add research doc on Do Not Disturb control diff --git a/README.md b/README.md index 87c515c..6476420 100644 --- a/README.md +++ b/README.md @@ -250,6 +250,26 @@ Small, well-understood changes go straight to code. For anything larger: 1. **Plan** — `docs/plans/.md` 1. **Execute** — one phase at a time +#### Autonomous issue loop + +`mix loop` runs a self-correcting AI loop against a GitHub issue. It picks +an open issue, classifies it as a feature or a bug, and repeatedly invokes +`claude` to implement (or fix) it — verifying each attempt with +`mix quality` and feeding any failure back into the next attempt until the +suite passes or the iteration budget is spent. On success it commits, +pushes, opens a PR that closes the issue, and comments the outcome back on +the issue (it opens a PR for review rather than auto-merging). + +```bash +mix loop # work the next open issue +mix loop --issue 42 # work a specific issue +mix loop --label bug # only consider issues with this label +mix loop --max 8 # cap the loop at 8 attempts (default 5) +mix loop --full # run the full mix quality each iteration +``` + +Requires the `gh` and `claude` CLIs on your PATH. + ## Roadmap - Homebrew distribution - Linux support diff --git a/docs/specs/2026-06-08-feature-debug-loop-design.md b/docs/specs/2026-06-08-feature-debug-loop-design.md new file mode 100644 index 0000000..47d5ad5 --- /dev/null +++ b/docs/specs/2026-06-08-feature-debug-loop-design.md @@ -0,0 +1,139 @@ +# Mix Loop Task Design + +## Context + +`mix todo` and `mix bump` already prove out a pattern: a Mix task gathers +context and hands a coding agent a complete, structured prompt. But both +still rely on a human (or a separate agent process) to actually do the +work and to keep re-running until the change is good. + +`mix loop` closes that gap for issue-driven work. It is an **autonomous, +self-driving loop**: it picks a GitHub issue, classifies it as a feature +or a bug, drives `claude` headless to implement (or fix) it, verifies the +result with `mix quality`, feeds any failure back into the next attempt, +and repeats until the suite is green or a max-iteration budget is spent. +On success it commits, pushes, opens a PR that closes the issue, and +comments the outcome back on the issue. + +The goal: `mix loop` (optionally `--issue N`) takes an issue from backlog +to green-and-pushed with no human in the inner loop, while leaving the PR +for human review rather than auto-merging. + +## Design + +### `mix loop [--issue N] [--label L] [--max M] [--full]` + +1. Check `gh` and `claude` are installed → stderr error if missing, exit 1. +2. Read `README.md` for project context (fed to the agent like `mix todo`). +3. Select the issue: + - `--issue N` → `gh issue view N --json number,title,body,labels`. + - otherwise → `gh issue list --state open --json ... [--label L]`, + take the first open issue. + - no open issues → stderr message, exit 1. +4. Classify the issue from its labels: any of `bug`, `fix`, `defect`, + `regression`, `debug` → `:bug`; otherwise `:feature`. This selects the + prompt framing (a bug is reproduced with a failing test first; a + feature is built test-first). +5. Create/checkout a working branch `feature/-` or + `fix/-` off `main`. +6. Run the iteration loop, up to `--max` (default 5): + - Iteration 1 → `build_initial_prompt` (issue + README + instructions). + - Later iterations → `build_retry_prompt` (issue + the tail of the + previous `mix quality` output + "fix without reverting progress"). + - Invoke `claude` headless with the prompt + (`claude -p --dangerously-skip-permissions`) so it can edit + files autonomously. + - Run `mix quality` (`--quick` by default, full with `--full`). + - `quality_passed?` (exit 0) → done. Else, if iterations remain → + retry with the failure tail; if budget spent → exhausted. +7. Outcome: + - **done** → `git add -A` + commit (`feat:`/`fix:` subject, `Closes #N` + body), push the branch, `gh pr create` (falling back to the existing + PR), comment the summary + PR URL on the issue, print the summary. + - **exhausted** → leave the branch in place for manual review, comment + the failure summary on the issue, print it, exit 1. + +Status messages ("Iteration 2/5: running mix quality...") go to stderr so +stdout stays a clean summary. + +### Guard rails + +- **Tool checks:** both `gh` and `claude` are probed up front with a clear + message and a non-zero exit if missing. +- **Bounded:** the loop can never run forever — `--max` caps attempts and + `should_continue?/3` is the single decision point. +- **Non-destructive on failure:** an exhausted loop never force-anything; + it leaves the branch and surfaces the failing output. +- **No auto-merge:** unlike `mix todo`, `mix loop` opens a PR for human + review rather than merging it. +- **Clean exit:** errors use `exit({:shutdown, 1})` — non-zero status, no + stacktrace. + +## Files + +| File | Action | Purpose | +|---|---|---| +| `lib/mix/tasks/loop.ex` | Create | The Mix task — single module | +| `test/mix/tasks/loop_test.exs` | Create | Tests for all pure functions | +| `CHANGELOG.md` | Edit | `### Added` entry | +| `README.md` | Edit | Document the command under Development | + +## Module Design + +Single module: `Mix.Tasks.Loop`. + +`issue` type: `%{number: integer(), title: String.t(), body: String.t(), +labels: [String.t()]}`. `mode` type: `:feature | :bug`. + +### Pure functions (unit-testable, `async: true`) + +| Function | Signature | Purpose | +|---|---|---| +| `parse_issues/1` | `String.t() → {:ok, [issue]} \| {:error, :invalid_json}` | Decode a `gh issue list --json` array | +| `parse_issue/1` | `String.t() → {:ok, issue} \| {:error, :invalid_json}` | Decode a `gh issue view --json` object | +| `first_open_issue/1` | `[issue] → {:ok, issue} \| {:error, :no_issues}` | First issue in the list | +| `classify_issue/1` | `issue → :feature \| :bug` | Label-based feature/bug split | +| `slugify/1` | `String.t() → String.t()` | Branch-safe slug | +| `build_branch_name/2` | `issue, mode → String.t()` | `feature/-` / `fix/-` | +| `pr_title/2` | `issue, mode → String.t()` | `feat:`/`fix:` subject line | +| `commit_message/2` | `issue, mode → String.t()` | Subject + `Closes #N` body | +| `build_initial_prompt/3` | `issue, mode, readme → String.t()` | First-attempt agent prompt | +| `build_retry_prompt/5` | `issue, mode, failure, iteration, max → String.t()` | Retry prompt with failure tail | +| `summarize_failure/2` | `String.t(), pos_integer() → String.t()` | Keep the last N lines of output | +| `quality_passed?/1` | `integer() → boolean()` | Exit code 0 | +| `should_continue?/3` | `boolean(), pos_integer(), pos_integer() → :done \| :retry \| :exhausted` | Loop decision | +| `claude_args/1` | `String.t() → [String.t()]` | Headless `claude` argv | +| `build_summary/4` | `issue, mode, :done \| :exhausted, integer() → String.t()` | Final report / issue comment | +| `extract_pr_url/1` | `String.t() → {:ok, String.t()} \| {:error, :no_pr_url}` | Pull the PR URL from `gh pr create` output | + +### Side-effecting helpers + +`cmd/2` wraps `System.cmd/3`; `check_gh_installed/0` and +`check_claude_installed/0` probe the tools; `fetch_issue/1` runs the `gh` +queries; `run_claude/1` and `run_quality/1` shell out per iteration; +`checkout_branch/1`, `git_commit/2`, `git_push/1`, `create_pr/2`, and +`comment_issue/2` finalize. Per project convention, git/gh/claude shell +commands are not unit-tested — the pure functions hold all the logic. + +### Entry point + +```elixir +def run(argv) do + {opts, _, _} = OptionParser.parse(argv, strict: [...]) + loop(File.cwd!(), opts) +end +``` + +`loop/2` is a `with` chain delegating to `handle_error/1`, mirroring +`mix todo`. + +## JSON decoding + +Issues are decoded with OTP's built-in `:json` module (Severance targets +OTP 29+), wrapped so malformed input returns `{:error, :invalid_json}` +instead of raising. JSON `null` bodies are coerced to `""`. + +## Test strategy + +All pure functions tested with `async: true`. Side-effecting git/gh/claude +wrappers are exercised manually, consistent with `mix todo`. diff --git a/lib/mix/tasks/loop.ex b/lib/mix/tasks/loop.ex new file mode 100644 index 0000000..d5b81fe --- /dev/null +++ b/lib/mix/tasks/loop.ex @@ -0,0 +1,619 @@ +defmodule Mix.Tasks.Loop do + @shortdoc "Autonomously build or debug a GitHub issue with an AI loop" + + @moduledoc """ + Drives a self-correcting AI loop against a GitHub issue. + + `mix loop` picks an open issue, classifies it as a feature or a bug, + and repeatedly invokes `claude` headless to implement (or fix) it, + verifying each attempt with `mix quality`. Failing output is fed back + into the next attempt until the suite passes or the iteration budget is + spent. On success it commits, pushes, opens a PR that closes the issue, + and comments the outcome back on the issue. + + ## Usage + + mix loop # work the next open issue + mix loop --issue 42 # work a specific issue + mix loop --label bug # only consider issues with this label + mix loop --max 8 # cap the loop at 8 attempts (default 5) + mix loop --full # run the full `mix quality` each iteration + + Unlike `mix todo`, the loop opens a PR for human review rather than + merging it. + """ + + use Mix.Task + + @type issue :: %{ + number: integer(), + title: String.t(), + body: String.t(), + labels: [String.t()] + } + + @type mode :: :feature | :bug + @type outcome :: :done | :exhausted + + @default_max 5 + @bug_labels ~w(bug fix defect regression debug) + @claude_flags ["--dangerously-skip-permissions"] + @issue_fields "number,title,body,labels" + + @doc """ + Decodes a `gh issue list --json` array into a list of issue maps. + + Returns `{:error, :invalid_json}` for malformed input or when the top + level is not a JSON array. + """ + @spec parse_issues(String.t()) :: {:ok, [issue()]} | {:error, :invalid_json} + def parse_issues(json) do + case decode(json) do + {:ok, list} when is_list(list) -> {:ok, Enum.map(list, &to_issue/1)} + {:ok, _other} -> {:error, :invalid_json} + :error -> {:error, :invalid_json} + end + end + + @doc """ + Decodes a `gh issue view --json` object into a single issue map. + + Returns `{:error, :invalid_json}` for malformed input or when the top + level is not a JSON object. + """ + @spec parse_issue(String.t()) :: {:ok, issue()} | {:error, :invalid_json} + def parse_issue(json) do + case decode(json) do + {:ok, %{} = obj} -> {:ok, to_issue(obj)} + {:ok, _other} -> {:error, :invalid_json} + :error -> {:error, :invalid_json} + end + end + + @doc """ + Returns the first issue in the list, or `{:error, :no_issues}` when empty. + """ + @spec first_open_issue([issue()]) :: {:ok, issue()} | {:error, :no_issues} + def first_open_issue([]), do: {:error, :no_issues} + def first_open_issue([issue | _]), do: {:ok, issue} + + @doc """ + Classifies an issue as `:bug` or `:feature` from its labels. + + Any label in `#{inspect(@bug_labels)}` (case-insensitive) marks the + issue as a bug; everything else is treated as a feature. + + ## Examples + + iex> Mix.Tasks.Loop.classify_issue(%{number: 1, title: "x", body: "", labels: ["bug"]}) + :bug + + iex> Mix.Tasks.Loop.classify_issue(%{number: 1, title: "x", body: "", labels: []}) + :feature + """ + @spec classify_issue(issue()) :: mode() + def classify_issue(%{labels: labels}) do + normalized = Enum.map(labels, &String.downcase/1) + if Enum.any?(normalized, &(&1 in @bug_labels)), do: :bug, else: :feature + end + + @doc """ + Turns free text into a branch-safe slug, truncated to 50 characters. + + ## Examples + + iex> Mix.Tasks.Loop.slugify("Add Dark Mode!") + "add-dark-mode" + """ + @spec slugify(String.t()) :: String.t() + def slugify(text) do + text + |> String.downcase() + |> String.replace(~r/[^a-z0-9]+/, "-") + |> String.trim("-") + |> String.slice(0, 50) + end + + @doc """ + Builds the working branch name for an issue, e.g. `feature/42-add-auth`. + + Bugs use the `fix/` prefix; features use `feature/`. When the title has + no usable slug, only the issue number is used. + """ + @spec build_branch_name(issue(), mode()) :: String.t() + def build_branch_name(%{number: number, title: title}, mode) do + prefix = branch_prefix(mode) + + case slugify(title) do + "" -> "#{prefix}/#{number}" + slug -> "#{prefix}/#{number}-#{slug}" + end + end + + @doc """ + Builds the PR / commit subject line, e.g. `feat: Add auth (#42)`. + """ + @spec pr_title(issue(), mode()) :: String.t() + def pr_title(%{number: number, title: title}, mode) do + "#{title_prefix(mode)}: #{title} (##{number})" + end + + @doc """ + Builds the commit message: the subject line plus a `Closes #N` trailer. + """ + @spec commit_message(issue(), mode()) :: String.t() + def commit_message(%{number: number} = issue, mode) do + "#{pr_title(issue, mode)}\n\nCloses ##{number}" + end + + @doc """ + Builds the first-attempt agent prompt: the issue, the README context, and + TDD instructions framed for a feature or a bug. + """ + @spec build_initial_prompt(issue(), mode(), String.t()) :: String.t() + def build_initial_prompt(issue, mode, readme) do + """ + You are working autonomously on the Severance project, driven by `mix loop`. + + #{task_line(issue, mode)} + + # Issue ##{issue.number}: #{issue.title} + + #{issue_body(issue)} + + ## Project Context + + #{readme} + + ## Instructions + + 1. Read AGENTS.md and the codebase to understand the conventions and patterns. + 2. Follow TDD. #{tdd_note(mode)} + 3. Keep your changes focused on this issue. Do not edit CHANGELOG.md. + 4. Your work is verified by `mix quality` — it must pass with zero failures. + 5. Do not commit, push, or open a pull request. `mix loop` does that once + `mix quality` is green. + """ + end + + @doc """ + Builds a retry prompt that feeds the previous `mix quality` failure back + to the agent, noting the iteration number and budget. + """ + @spec build_retry_prompt(issue(), mode(), String.t(), pos_integer(), pos_integer()) :: + String.t() + def build_retry_prompt(issue, _mode, failure_output, iteration, max) do + """ + You are on iteration #{iteration} of #{max}, still working on GitHub issue + ##{issue.number} inside `mix loop`. + + The previous attempt did not pass `mix quality`. Here is the tail of the + output: + + ``` + #{failure_output} + ``` + + Fix the failures above without reverting the progress you have already made. + Keep following TDD and the conventions in AGENTS.md. `mix quality` must pass. + """ + end + + @doc """ + Keeps the last `max_lines` lines of command output. + + Quality failures land at the end of the output, so the tail carries the + most useful signal for the retry prompt while keeping it compact. + """ + @spec summarize_failure(String.t(), pos_integer()) :: String.t() + def summarize_failure(output, max_lines \\ 40) do + lines = String.split(output, "\n") + + lines + |> Enum.drop(max(length(lines) - max_lines, 0)) + |> Enum.join("\n") + end + + @doc """ + Returns `true` when `mix quality` exited successfully. + + ## Examples + + iex> Mix.Tasks.Loop.quality_passed?(0) + true + + iex> Mix.Tasks.Loop.quality_passed?(1) + false + """ + @spec quality_passed?(integer()) :: boolean() + def quality_passed?(exit_code), do: exit_code == 0 + + @doc """ + Decides what the loop does after an attempt. + + Returns `:done` once quality passes, `:retry` while attempts remain, and + `:exhausted` when the budget is spent on a still-failing run. + + ## Examples + + iex> Mix.Tasks.Loop.should_continue?(true, 1, 5) + :done + + iex> Mix.Tasks.Loop.should_continue?(false, 5, 5) + :exhausted + """ + @spec should_continue?(boolean(), pos_integer(), pos_integer()) :: :done | :retry | :exhausted + def should_continue?(true, _iteration, _max), do: :done + def should_continue?(false, iteration, max) when iteration >= max, do: :exhausted + def should_continue?(false, _iteration, _max), do: :retry + + @doc """ + Builds the argv for a headless `claude` invocation. + + Uses print mode (`-p`) so the call returns when the agent is done, and + skips interactive permission prompts so it can edit files unattended. + """ + @spec claude_args(String.t()) :: [String.t()] + def claude_args(prompt), do: ["-p", prompt | @claude_flags] + + @doc """ + Builds the human-readable summary printed to stdout and posted on the + issue, for both the `:done` and `:exhausted` outcomes. + """ + @spec build_summary(issue(), mode(), outcome(), non_neg_integer()) :: String.t() + def build_summary(issue, mode, :done, iterations) do + """ + Loop complete for issue ##{issue.number} (#{mode_label(mode)}): #{issue.title} + Passed `mix quality` after #{iterations} iteration(s). + """ + end + + def build_summary(issue, mode, :exhausted, iterations) do + """ + Loop could not complete issue ##{issue.number} (#{mode_label(mode)}): #{issue.title} + `mix quality` is still failing after #{iterations} iteration(s). The branch + has been left in place for manual review. + """ + end + + @doc """ + Extracts the PR URL from `gh pr create` output, which may prepend + warnings before the URL on its final line. + """ + @spec extract_pr_url(String.t()) :: {:ok, String.t()} | {:error, :no_pr_url} + def extract_pr_url(output) do + output + |> String.split("\n") + |> Enum.reverse() + |> Enum.find(&String.starts_with?(&1, "https://")) + |> case do + nil -> {:error, :no_pr_url} + url -> {:ok, String.trim(url)} + end + end + + @impl Mix.Task + def run(argv) do + {opts, _rest, _invalid} = + OptionParser.parse(argv, + strict: [issue: :integer, label: :string, max: :integer, full: :boolean] + ) + + loop(File.cwd!(), opts) + end + + @doc false + def loop(root \\ File.cwd!(), opts \\ []) do + max = opts[:max] || @default_max + full? = opts[:full] || false + + with :ok <- check_gh_installed(), + :ok <- check_claude_installed(), + {:ok, readme} <- read_readme(root), + {:ok, issue} <- fetch_issue(opts), + mode = classify_issue(issue), + branch = build_branch_name(issue, mode), + :ok <- checkout_branch(branch) do + case run_loop(issue, mode, readme, max, full?) do + {:ok, iterations} -> finalize(issue, mode, branch, iterations) + {:exhausted, iterations, failure} -> report_exhausted(issue, mode, iterations, failure) + {:error, _} = error -> handle_error(error) + end + else + error -> handle_error(error) + end + end + + # --- Loop --- + + defp run_loop(issue, mode, readme, max, full?) do + iterate(issue, mode, readme, max, full?, 1, nil) + end + + defp iterate(issue, mode, readme, max, full?, iteration, previous_failure) do + prompt = prompt_for(issue, mode, readme, max, iteration, previous_failure) + stderr("Iteration #{iteration}/#{max}: invoking claude...") + + with {:ok, _output} <- run_claude(prompt) do + stderr("Iteration #{iteration}/#{max}: running mix quality...") + {quality_output, exit_code} = run_quality(full?) + + case should_continue?(quality_passed?(exit_code), iteration, max) do + :done -> + {:ok, iteration} + + :exhausted -> + {:exhausted, iteration, summarize_failure(quality_output)} + + :retry -> + failure = summarize_failure(quality_output) + iterate(issue, mode, readme, max, full?, iteration + 1, failure) + end + end + end + + defp prompt_for(issue, mode, readme, _max, 1, _previous) do + build_initial_prompt(issue, mode, readme) + end + + defp prompt_for(issue, mode, _readme, max, iteration, previous) do + build_retry_prompt(issue, mode, previous, iteration, max) + end + + # --- Finalize --- + + defp finalize(issue, mode, branch, iterations) do + summary = build_summary(issue, mode, :done, iterations) + + with :ok <- git_commit(issue, mode), + :ok <- git_push(branch), + {:ok, pr_url} <- ensure_pr(issue, mode), + :ok <- comment_issue(issue, "#{summary}\nPR: #{pr_url}") do + IO.write("#{summary}PR: #{pr_url}\n") + else + error -> handle_error(error) + end + end + + defp report_exhausted(issue, mode, iterations, failure) do + summary = build_summary(issue, mode, :exhausted, iterations) + comment_issue(issue, "#{summary}\n```\n#{failure}\n```") + stderr(summary) + exit({:shutdown, 1}) + end + + # --- Side-effect helpers --- + + defp cmd(executable, args) do + case System.cmd(executable, args, stderr_to_stdout: true) do + {output, 0} -> {:ok, String.trim(output)} + {output, code} -> {:error, {output, code}} + end + end + + defp check_gh_installed do + if System.find_executable("gh"), do: :ok, else: {:error, :no_gh} + end + + defp check_claude_installed do + if System.find_executable("claude"), do: :ok, else: {:error, :no_claude} + end + + defp read_readme(root) do + case File.read(Path.join(root, "README.md")) do + {:ok, content} -> {:ok, content} + {:error, _} -> {:error, :no_readme} + end + end + + defp fetch_issue(opts) do + case opts[:issue] do + nil -> fetch_next_issue(opts[:label]) + number -> fetch_issue_by_number(number) + end + end + + defp fetch_next_issue(label) do + args = ["issue", "list", "--state", "open", "--json", @issue_fields, "--limit", "30"] + + with {:ok, output} <- cmd("gh", args ++ label_args(label)), + {:ok, issues} <- parse_issues(output) do + first_open_issue(issues) + end + end + + defp fetch_issue_by_number(number) do + args = ["issue", "view", Integer.to_string(number), "--json", @issue_fields] + + with {:ok, output} <- cmd("gh", args) do + parse_issue(output) + end + end + + defp label_args(nil), do: [] + defp label_args(label), do: ["--label", label] + + defp checkout_branch(branch) do + stderr("Preparing branch #{branch}...") + + with {:ok, _} <- cmd("git", ["checkout", "-B", branch]) do + :ok + end + end + + defp run_claude(prompt) do + case System.cmd("claude", claude_args(prompt), stderr_to_stdout: true) do + {output, 0} -> {:ok, output} + {output, code} -> {:error, {output, code}} + end + end + + defp run_quality(full?) do + System.cmd("mix", quality_args(full?), stderr_to_stdout: true) + end + + defp quality_args(true), do: ["quality"] + defp quality_args(false), do: ["quality", "--quick"] + + defp git_commit(issue, mode) do + stderr("Committing changes...") + + with {:ok, _} <- cmd("git", ["add", "-A"]) do + commit_or_detect_empty(commit_message(issue, mode)) + end + end + + defp commit_or_detect_empty(message) do + case cmd("git", ["commit", "-m", message]) do + {:ok, _} -> + :ok + + {:error, {output, _code}} = error -> + if nothing_to_commit?(output), do: {:error, :no_changes}, else: error + end + end + + defp nothing_to_commit?(output), do: String.contains?(output, "nothing to commit") + + defp git_push(branch) do + stderr("Pushing branch...") + + with {:ok, _} <- cmd("git", ["push", "-u", "origin", branch]) do + :ok + end + end + + defp ensure_pr(issue, mode) do + stderr("Opening pull request...") + body = "Closes ##{issue.number}\n\nOpened automatically by `mix loop`." + + case cmd("gh", ["pr", "create", "--title", pr_title(issue, mode), "--body", body]) do + {:ok, output} -> extract_pr_url(output) + {:error, _} -> find_existing_pr() + end + end + + defp find_existing_pr do + case cmd("gh", ["pr", "view", "--json", "url", "-q", ".url"]) do + {:ok, url} -> {:ok, url} + _error -> {:error, :no_pr} + end + end + + defp comment_issue(issue, body) do + case cmd("gh", ["issue", "comment", Integer.to_string(issue.number), "--body", body]) do + {:ok, _} -> :ok + error -> error + end + end + + # --- Decoding --- + + defp decode(json) do + {:ok, :json.decode(json)} + rescue + _ -> :error + end + + defp to_issue(obj) when is_map(obj) do + %{ + number: Map.get(obj, "number"), + title: string_or_empty(Map.get(obj, "title")), + body: string_or_empty(Map.get(obj, "body")), + labels: obj |> Map.get("labels", []) |> to_labels() + } + end + + defp to_labels(labels) when is_list(labels), do: Enum.flat_map(labels, &label_name/1) + defp to_labels(_labels), do: [] + + defp label_name(%{"name" => name}) when is_binary(name), do: [name] + defp label_name(name) when is_binary(name), do: [name] + defp label_name(_other), do: [] + + defp string_or_empty(value) when is_binary(value), do: value + defp string_or_empty(_value), do: "" + + # --- Prompt fragments --- + + defp task_line(issue, :bug) do + "Your task is to reproduce and fix the bug described in GitHub issue ##{issue.number}." + end + + defp task_line(issue, :feature) do + "Your task is to implement the feature described in GitHub issue ##{issue.number}." + end + + defp tdd_note(:bug) do + "Start by writing a test that reproduces the bug, then fix it until the test passes." + end + + defp tdd_note(:feature) do + "Write a failing test for the new behavior, then implement it until the test passes." + end + + defp issue_body(%{body: body}) do + case String.trim(body) do + "" -> "(No description provided.)" + trimmed -> trimmed + end + end + + defp branch_prefix(:bug), do: "fix" + defp branch_prefix(:feature), do: "feature" + + defp title_prefix(:bug), do: "fix" + defp title_prefix(:feature), do: "feat" + + defp mode_label(:bug), do: "bug" + defp mode_label(:feature), do: "feature" + + # --- Errors --- + + defp handle_error({:error, :no_gh}) do + stderr("gh CLI not found. Install it: https://cli.github.com/") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :no_claude}) do + stderr("claude CLI not found. Install it: https://docs.claude.com/claude-code") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :no_readme}) do + stderr("README.md not found") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :no_issues}) do + stderr("No open issues found. Nothing to loop on!") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :invalid_json}) do + stderr("Could not parse the issue data returned by gh.") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :no_changes}) do + stderr("The agent produced no changes — nothing to commit. Leaving the branch as-is.") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :no_pr}) do + stderr("Pushed the branch but could not open or find a PR. Open one manually.") + exit({:shutdown, 1}) + end + + defp handle_error({:error, :no_pr_url}) do + stderr("Opened a PR but could not read its URL from the gh output.") + exit({:shutdown, 1}) + end + + defp handle_error({:error, {output, code}}) do + stderr("Command failed (exit #{code}):\n#{output}") + exit({:shutdown, 1}) + end + + defp stderr(msg), do: IO.puts(:stderr, msg) +end diff --git a/test/mix/tasks/loop_test.exs b/test/mix/tasks/loop_test.exs new file mode 100644 index 0000000..ea60be6 --- /dev/null +++ b/test/mix/tasks/loop_test.exs @@ -0,0 +1,302 @@ +defmodule Mix.Tasks.LoopTest do + use ExUnit.Case, async: true + + alias Mix.Tasks.Loop + + defp issue(overrides \\ %{}) do + Map.merge( + %{number: 42, title: "Add dark mode", body: "It would be nice to have dark mode.", labels: []}, + overrides + ) + end + + describe "parse_issues/1" do + test "decodes a single issue with label names" do + json = ~s([{"number":42,"title":"Add dark mode","body":"Please","labels":[{"name":"feature"}]}]) + + assert {:ok, [item]} = Loop.parse_issues(json) + assert item.number == 42 + assert item.title == "Add dark mode" + assert item.body == "Please" + assert item.labels == ["feature"] + end + + test "decodes multiple issues preserving order" do + json = + ~s([{"number":1,"title":"One","body":"a","labels":[]},) <> + ~s({"number":2,"title":"Two","body":"b","labels":[]}]) + + assert {:ok, [first, second]} = Loop.parse_issues(json) + assert first.number == 1 + assert second.number == 2 + end + + test "coerces a null body to an empty string" do + json = ~s([{"number":7,"title":"Null body","body":null,"labels":[]}]) + + assert {:ok, [item]} = Loop.parse_issues(json) + assert item.body == "" + end + + test "defaults missing labels to an empty list" do + json = ~s([{"number":7,"title":"No labels","body":"x"}]) + + assert {:ok, [item]} = Loop.parse_issues(json) + assert item.labels == [] + end + + test "returns error for malformed json" do + assert {:error, :invalid_json} = Loop.parse_issues("not json") + end + + test "returns error when the top level is not an array" do + assert {:error, :invalid_json} = Loop.parse_issues(~s({"number":1})) + end + end + + describe "parse_issue/1" do + test "decodes a single object" do + json = ~s({"number":99,"title":"Crash","body":"boom","labels":[{"name":"bug"}]}) + + assert {:ok, item} = Loop.parse_issue(json) + assert item.number == 99 + assert item.labels == ["bug"] + end + + test "returns error when the top level is an array" do + assert {:error, :invalid_json} = Loop.parse_issue(~s([{"number":1}])) + end + + test "returns error for malformed json" do + assert {:error, :invalid_json} = Loop.parse_issue("{") + end + end + + describe "first_open_issue/1" do + test "returns the first issue" do + assert {:ok, %{number: 1}} = Loop.first_open_issue([issue(%{number: 1}), issue(%{number: 2})]) + end + + test "returns error for an empty list" do + assert {:error, :no_issues} = Loop.first_open_issue([]) + end + end + + describe "classify_issue/1" do + test "classifies a bug label as :bug" do + assert Loop.classify_issue(issue(%{labels: ["bug"]})) == :bug + end + + test "treats fix, defect, regression, and debug as bugs" do + for label <- ["fix", "defect", "regression", "debug"] do + assert Loop.classify_issue(issue(%{labels: [label]})) == :bug + end + end + + test "is case-insensitive" do + assert Loop.classify_issue(issue(%{labels: ["Bug"]})) == :bug + end + + test "classifies anything else as :feature" do + assert Loop.classify_issue(issue(%{labels: ["enhancement"]})) == :feature + end + + test "classifies an unlabeled issue as :feature" do + assert Loop.classify_issue(issue(%{labels: []})) == :feature + end + end + + describe "slugify/1" do + test "replaces spaces with hyphens and downcases" do + assert Loop.slugify("Add Dark Mode") == "add-dark-mode" + end + + test "strips special characters" do + assert Loop.slugify("Fix crash on start!") == "fix-crash-on-start" + end + + test "trims leading and trailing hyphens" do + assert Loop.slugify(" spaced ") == "spaced" + end + + test "truncates long titles" do + slug = Loop.slugify(String.duplicate("a", 100)) + assert String.length(slug) <= 50 + end + end + + describe "build_branch_name/2" do + test "uses the feature prefix for features" do + assert Loop.build_branch_name(issue(), :feature) == "feature/42-add-dark-mode" + end + + test "uses the fix prefix for bugs" do + assert Loop.build_branch_name(issue(%{number: 7, title: "Crash"}), :bug) == "fix/7-crash" + end + + test "falls back to just the number when the title has no slug" do + assert Loop.build_branch_name(issue(%{number: 5, title: "!!!"}), :feature) == "feature/5" + end + end + + describe "pr_title/2" do + test "prefixes features with feat:" do + assert Loop.pr_title(issue(), :feature) == "feat: Add dark mode (#42)" + end + + test "prefixes bugs with fix:" do + assert Loop.pr_title(issue(%{number: 7, title: "Crash"}), :bug) == "fix: Crash (#7)" + end + end + + describe "commit_message/2" do + test "includes the subject and a Closes trailer" do + message = Loop.commit_message(issue(), :feature) + assert message =~ "feat: Add dark mode (#42)" + assert message =~ "Closes #42" + end + end + + describe "build_initial_prompt/3" do + test "includes the issue number, title, and body" do + prompt = Loop.build_initial_prompt(issue(), :feature, "# Readme") + assert prompt =~ "#42" + assert prompt =~ "Add dark mode" + assert prompt =~ "It would be nice to have dark mode." + end + + test "includes the README context" do + prompt = Loop.build_initial_prompt(issue(), :feature, "Build commands here.") + assert prompt =~ "Build commands here." + end + + test "instructs TDD and references AGENTS.md" do + prompt = Loop.build_initial_prompt(issue(), :feature, "# Readme") + assert prompt =~ "TDD" + assert prompt =~ "AGENTS.md" + end + + test "names mix quality as the verification gate" do + prompt = Loop.build_initial_prompt(issue(), :feature, "# Readme") + assert prompt =~ "mix quality" + end + + test "tells the agent not to commit or open a PR" do + prompt = Loop.build_initial_prompt(issue(), :feature, "# Readme") + assert prompt =~ "Do not commit" + end + + test "frames a bug as reproduce-then-fix" do + prompt = Loop.build_initial_prompt(issue(%{labels: ["bug"]}), :bug, "# Readme") + assert prompt =~ "reproduce" + end + + test "frames a feature as implement" do + prompt = Loop.build_initial_prompt(issue(), :feature, "# Readme") + assert prompt =~ "implement" + end + end + + describe "build_retry_prompt/5" do + test "includes the iteration number and budget" do + prompt = Loop.build_retry_prompt(issue(), :feature, "boom", 2, 5) + assert prompt =~ "2" + assert prompt =~ "5" + end + + test "includes the failure output" do + prompt = Loop.build_retry_prompt(issue(), :feature, "** (RuntimeError) boom", 2, 5) + assert prompt =~ "** (RuntimeError) boom" + end + + test "tells the agent to fix without reverting progress" do + prompt = Loop.build_retry_prompt(issue(), :feature, "boom", 2, 5) + assert prompt =~ "without reverting" + end + + test "names mix quality as the gate" do + prompt = Loop.build_retry_prompt(issue(), :feature, "boom", 2, 5) + assert prompt =~ "mix quality" + end + end + + describe "summarize_failure/2" do + test "keeps the last n lines" do + output = Enum.map_join(1..100, "\n", &"line #{&1}") + summary = Loop.summarize_failure(output, 10) + + assert summary =~ "line 100" + assert summary =~ "line 91" + refute summary =~ "line 90" + end + + test "returns short output unchanged" do + assert Loop.summarize_failure("only line", 10) == "only line" + end + end + + describe "quality_passed?/1" do + test "true for exit code 0" do + assert Loop.quality_passed?(0) + end + + test "false for a non-zero exit code" do + refute Loop.quality_passed?(1) + end + end + + describe "should_continue?/3" do + test "done when quality passed" do + assert Loop.should_continue?(true, 1, 5) == :done + end + + test "retry when failing with budget remaining" do + assert Loop.should_continue?(false, 2, 5) == :retry + end + + test "exhausted when failing on the last iteration" do + assert Loop.should_continue?(false, 5, 5) == :exhausted + end + end + + describe "claude_args/1" do + test "runs headless with the prompt and skips permission prompts" do + args = Loop.claude_args("do the thing") + assert Enum.at(args, 0) == "-p" + assert Enum.at(args, 1) == "do the thing" + assert "--dangerously-skip-permissions" in args + end + end + + describe "build_summary/4" do + test "done summary names the issue and iteration count" do + summary = Loop.build_summary(issue(), :feature, :done, 3) + assert summary =~ "#42" + assert summary =~ "Add dark mode" + assert summary =~ "3" + assert summary =~ "mix quality" + end + + test "exhausted summary signals the loop did not complete" do + summary = Loop.build_summary(issue(), :bug, :exhausted, 5) + assert summary =~ "#42" + assert summary =~ "still failing" + end + end + + describe "extract_pr_url/1" do + test "returns the URL when output is just a URL" do + assert {:ok, "https://github.com/org/repo/pull/1"} = + Loop.extract_pr_url("https://github.com/org/repo/pull/1") + end + + test "extracts the URL from output with warnings" do + output = "Warning: 3 uncommitted changes\nhttps://github.com/org/repo/pull/1" + assert {:ok, "https://github.com/org/repo/pull/1"} = Loop.extract_pr_url(output) + end + + test "returns error when no URL is present" do + assert {:error, :no_pr_url} = Loop.extract_pr_url("nothing here") + end + end +end