CI: test the failure modes, not just the happy paths#39
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The old smoke block asserted almost nothing about invalid input, and `init` and `root` were untested entirely. None of the blockers in the pre-1.0 review would have been caught by it, and ShellCheck passes clean on code containing both the argument-parsing hang and the `_seed_agents` precedence bug — linting is not coverage. Extracts the inline block to `tests/smoke.sh` so contributors run exactly what CI runs. This does not violate the single-file constraint: that rule governs `git-trees`, not the test harness. The suite builds its own fixtures under `mktemp -d` from `file://` remotes — no network, no `jq` — and keeps running after a failure so one run reports every problem. New coverage, each assertion commented with the bug it pins: - `init` end to end, plus rollback when the post-clone fetch fails and the retry that rollback makes possible - `--host` / `--dir` with a missing value, bounded so the infinite-loop regression fails fast instead of hanging the runner until the job timeout (`timeout` is not installed on macOS, hence the background-PID dance) - `root`, bare-container adoption, and all four `_seed_agents` cases including the broken symlink that `-e` misses and `-L` catches - `add` rejecting `.`, `..`, and `'has space'` as bad branch names rather than directory collisions, and before `Preparing worktree` is printed - `list --json` for a worktree path containing a backslash and a quote - `add --no-push`, `TREES_NO_PUSH`, and that the default still pushes `add brandnew` now asserts the upstream is exactly `origin/brandnew`. The old `!= origin/main` form also passed on an empty or otherwise wrong upstream — the same bug wearing a different hat. `AGENTS.md` § Testing is rewritten: it claimed automated coverage that did not exist, and now describes what the suite actually checks. Closes #27. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`in_dir` is only ever passed as an argument to the assert_* helpers, so ShellCheck reports its body as unreachable. The code differs by version — 0.11 (local, and Homebrew on the macOS runner) says SC2329, 0.9 (apt on ubuntu-latest) says SC2317 — so the existing directive covered only half the matrix and the ubuntu job failed at lint before it ever ran the suite. List both codes.
|
Rebased onto the current
That is now a test that Also added the
Verified locally on macOS before pushing: One note for whoever merges: the comment justifying |
Merge order: 5 of 6 — merge this after #35, #36 and #37.
Closes #27.
Why
The old smoke block was genuinely useful — it pinned both documented regressions
and ran on Linux and macOS — but it asserted almost nothing about invalid input,
and
initandrootwere untested entirely. None of the blockers in thepre-1.0 review would have been caught by it.
And ShellCheck is not the safety net it looks like: it passes clean on code
containing both the argument-parsing hang and the
_seed_agentsprecedencebug. Linting is not coverage.
What
Extracts the inline block to
tests/smoke.sh, run by CI and by contributorswith the same command. This does not violate the single-file constraint — that
rule governs
git-trees, not the test harness, andAGENTS.mdnow says so.The suite builds its own fixtures under
mktemp -dfromfile://remotes: nonetwork, no
jq, nothing written outside the temp dir. It usesset -uo pipefailwithout-e, matching the tool's own style, so one run reports everyfailure instead of stopping at the first.
--host/--dirvalue exits nonzeroinitat all — works offline viafile://root,root --agents,_seed_agentsoverwrite refusallist --jsonwith a backslash in a pathjson.loadtrack/push makesaddexit nonzeroadd brandnewasserted loosely== origin/brandnew--older-than abc,clean --gonedefault branch,clean --applycleanPlus
add ./../'has space', and--no-push/TREES_NO_PUSHfrom #37.Two assertions deserve their comments:
so a plain assertion would hang the runner until the job timeout rather than
fail.
timeoutisn't installed on macOS, hence the background-PID +kill -0helper.
add brandnewasserts the exact upstream. The old!= origin/mainalsopassed on an empty or otherwise wrong upstream — the same bug in a different
hat.
AGENTS.md§ Testing claimed automated coverage that did not exist. Rewritten todescribe what the suite actually checks.
Verified
Against unfixed
main— 17 failures locally, and the same 17 on both CIrunners. They are exactly the right ones, and nothing else fails, so the suite
is not flaky against the current tool.
Portability — the first CI run caught a real bug in this PR, since fixed:
in_diris only ever passed as an argument, and ShellCheck reports its body asunreachable under a code that differs by version (SC2329 in 0.11 via Homebrew on
macOS, SC2317 in 0.9 via apt on ubuntu). The directive covered only half the
matrix, so the ubuntu job failed at lint before running anything. Both codes are
listed now, and ubuntu executes the full suite.
Against a real merge of
remove-clean+script-bugs+add-no-push+docs-and-install—smoke tests passed, all sections green including--no-push.That merge also answers the conflict question: #35, #36 and #37 merge into each
other cleanly. The only conflict in the whole set is #38 vs #35 in
README.md(the quickstart).
shellcheck -s bash git-trees install.sh tests/smoke.shclean;env -Cavoidedthroughout since macOS
envhas no such flag.Conflict resolution for
ci.yml#36 and #37 append assertions to the inline block that this PR deletes. Their
coverage is not lost — it is already reproduced inside
tests/smoke.sh(the
addname-validation, missing-value, JSON-escaping and--no-pushsections). Resolve by keeping
tests/smoke.shand the two-lineci.yml, anddropping the inline block entirely.
🤖 Generated with Claude Code