Consolidate E2E test suite into cli repo

[khaong]

Summary

• Moves the consolidated E2E test suite from entire-cli-e2e-tests into e2e/ as the single E2E suite
• Removes the old shadow-hook based suite at cmd/entire/cli/e2e_test/
• Updates mise tasks to point at new e2e/tests/... path with optional filter args

What's in e2e/

• 3 agents: Claude Code, Gemini CLI, OpenCode — all run via ForEachAgent
• E2E_AGENT env var for CI matrix targeting (when set, only matching agent registers)
• //go:build e2e tag on all test files (invisible to normal go build/go test)
• 37 tests covering: single/multi session, auto-commit, attribution, rewind, stash workflows, split commits, subagent flows, session lifecycle, edge cases, checkpoint metadata validation
• Artifact capture (git log, tree, checkpoint metadata, entire logs, console transcript) for failure debugging
• exploratory/ directory for on-demand test scenarios not run by CI

Key design decisions

• Always uses os.MkdirTemp instead of t.TempDir() — Go's nested subtest dirs confuse some agents' path resolution
• OpenCode gets opencode.json config written to allow external_directory permission in non-interactive mode
• filepath.EvalSymlinks resolves macOS /var → /private/var so agent CLIs see consistent paths

Test plan

• mise run test:e2e:claude TestSingleSessionManualCommit — PASS
• mise run test:e2e:opencode TestSingleSessionManualCommit — PASS
• go build ./... — clean (e2e tests invisible without build tag)
• go test -tags=e2e -list '.*' ./e2e/tests/... — all 37 tests listed
• Full suite run against Claude
• Full suite run against Gemini
• Full suite run agains Opencode
• CI workflow update

🤖 Generated with Claude Code

[cursor]

PR Summary

Medium Risk
Test/CI-heavy change that deletes and replaces the E2E harness, adds a new agent to the CI matrix, and changes artifact generation/upload; main product code impact is low but CI stability and E2E signal quality could regress if the new runner/bootstraps are flaky.

Overview
Consolidates end-to-end testing into a new top-level e2e/ suite with agent abstractions (Claude Code, Gemini CLI, OpenCode), tmux-backed interactive sessions, artifact capture, and a small testreport tool to render gotest JSON events into a readable report.

Removes the legacy shadow-hook based E2E suite under cmd/entire/cli/e2e_test/, and replaces/augments coverage by adding new e2e/tests/* cases plus an integration test for resume in a relocated repo.

Updates CI workflows to install needed deps (tmux), build and run the CLI, run an agent bootstrap step before tests, expand the matrix to include Gemini, and always upload e2e/artifacts/. Also updates docs/config (e2e/README.md, CLAUDE.md, .gitignore, golangci exclusions) to reflect the new suite and ignore generated artifacts/binaries.

^{Written by Cursor Bugbot for commit 94cc998. Configure here.}

[Copilot]

Pull request overview

This PR consolidates E2E test infrastructure by moving the comprehensive test suite from a separate repository into e2e/ within the CLI repo, replacing the older shadow-hook based suite at cmd/entire/cli/e2e_test/.

Changes:

• Adds new consolidated E2E test suite in e2e/ with 37 tests covering single/multi-session workflows, auto-commit, attribution, rewind, stash, split commits, and edge cases
• Removes old shadow-hook based E2E suite from cmd/entire/cli/e2e_test/
• Updates mise tasks to point at e2e/tests/... with optional filter arguments for running specific tests

Reviewed changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
mise.toml	Updates E2E test tasks to target new e2e/tests/... path with filter arguments
e2e/testutil/repo.go	Core test utilities for repo setup, agent execution, and git operations
e2e/testutil/metadata.go	Checkpoint and session metadata structures for validation
e2e/testutil/assertions.go	Test assertion helpers for checkpoint validation and verification
e2e/testutil/artifacts.go	Artifact capture system for debugging failed tests
e2e/tests/*.go	37 comprehensive E2E tests across multiple workflow scenarios
e2e/agents/*.go	Agent abstraction layer supporting Claude Code, Gemini CLI, and OpenCode
e2e/entire/entire.go	CLI wrapper functions for entire commands in tests
cmd/entire/cli/e2e_test/*.go	Removed old E2E test suite files (11 files deleted)
.gitignore	Adds e2e/artifacts/ to ignored files

[khaong]

bugbot run

[Soph]

I tried this locally, and I couldn't get it to work at all :(

Claude thinks:

⏺ That's the bug. StartSession has the comment:
  // Locally, we skip CLAUDE_CONFIG_DIR so the Keychain-based auth works.

  But RunPrompt always sets CLAUDE_CONFIG_DIR, even locally. On macOS, Claude Code uses Keychain for auth, which is tied to the default ~/.claude path. When you override CLAUDE_CONFIG_DIR to a temp dir, Claude Code can't find auth credentials in the Keychain for that path → "Not logged in".

  RunPrompt needs the same CI-only guard that StartSession has. But this is the branch author's code, not ours. Do you want me to fix it, or flag it to the branch author?

I'm not sure if this is a me thing or why it works for you, so stopping trying to fix it :(

Also:

1. files_touched per-checkpoint validation is gone from most tests
2. Shadow branch cleanup checks are gone
3. Subagent task checkpoint fields not validated (to be fair, those were logs only so fine to skip them)

And last but not least: it's just testing the binary in path, I feel mise run test:e2e makes it look like if it would test what's checked out and that's what we should do, right?

[khaong]

I tried this locally, and I couldn't get it to work at all :(

Claude thinks:

⏺ That's the bug. StartSession has the comment:
  // Locally, we skip CLAUDE_CONFIG_DIR so the Keychain-based auth works.

  But RunPrompt always sets CLAUDE_CONFIG_DIR, even locally. On macOS, Claude Code uses Keychain for auth, which is tied to the default ~/.claude path. When you override CLAUDE_CONFIG_DIR to a temp dir, Claude Code can't find auth credentials in the Keychain for that path → "Not logged in".

  RunPrompt needs the same CI-only guard that StartSession has. But this is the branch author's code, not ours. Do you want me to fix it, or flag it to the branch author?

I'm not sure if this is a me thing or why it works for you, so stopping trying to fix it :(

Also:

1. files_touched per-checkpoint validation is gone from most tests
2. Shadow branch cleanup checks are gone
3. Subagent task checkpoint fields not validated (to be fair, those were logs only so fine to skip them)

And last but not least: it's just testing the binary in path, I feel mise run test:e2e makes it look like if it would test what's checked out and that's what we should do, right?

@Soph - Yup, it was a real problem 🤦‍♂️

It turns out that I've got my claude set up just a little bit different to the 'standard OAuth' flow.

Thanks to @dvydra @toothbrush we've got to the bottom of it and fixed some other 'user scope' transgressions the tests were doing.

I think longer term we want to move these tests to docker in any case, just to be 100% certain about blast radius.

[khaong]

@Soph and as for the other points:

1. files_touched per-checkpoint validation

TestCheckpointMetadataDeepValidation validates files_touched end-to-end, and the assertion helpers (AssertCheckpointFilesTouched, AssertCheckpointFilesTouchedContains) are available for any test that needs them. Detailed per-field checkpoint validation is also covered extensively in integration tests (17 test files in cmd/entire/cli/integration_test/).

2. Shadow branch cleanup checks

TestShadowBranchCleanedAfterAgentCommit covers the e2e path. Detailed shadow branch lifecycle is covered in integration tests.

Edit: I've added post-shadow checks to all the scenarios now

3. Testing the binary in PATH

Intentional design - the suite is meant to run against any installed binary for version comparison and regression detection. TestMain prints the version and stamps entire-version.txt into the artifact dir. Also added the version to the mise task output so it's explicit upfront what binary is being tested.

Switched it to run against the code in the accompanying branch by default, overridable by E2E_ENTIRE_BIN

[khaong]

bugbot run

[Copilot]

Pull request overview

Copilot reviewed 48 out of 49 changed files in this pull request and generated 2 comments.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

[khaong]

so enabling the shadow branch leftover checks have surfaced cases where we're not cleaning up properly - these are showing as failing tests:
TestModifiedFileAlwaysGetsCheckpoint
TestContentOverlapRevertNewFile

we are choosing to leave these in as bug markers

[Soph]

  TestContentOverlapRevertNewFile

  The AssertNoShadowBranches assertion is incorrect for this test and should be removed.

  This test deliberately tests a scenario where no checkpoint should be created (content mismatch on a new file). When no checkpoint is condensed, the shadow branch correctly remains
  - it preserves the agent's discarded work for potential later access via entire rewind.

  TestModifiedFileAlwaysGetsCheckpoint

  This one should work - a checkpoint should be created because modified files always count as overlap. If the shadow branch isn't being cleaned up, there may be a bug in the
  condensation logic or FilesTouched population.```

So I think `TestContentOverlapRevertNewFile` is actually fine keeping the shadow branch, only `TestModifiedFileAlwaysGetsCheckpoint` is indicating an issue.

[Soph]

I had to run brew install gotestsum

[khaong]

I had to run brew install gotestsum

mise install should handle this I think?

[khaong]

TestContentOverlapRevertNewFile

The AssertNoShadowBranches assertion is incorrect for this test and should be removed.

This test deliberately tests a scenario where no checkpoint should be created (content mismatch on a new file). When no checkpoint is condensed, the shadow branch correctly remains

• it preserves the agent's discarded work for potential later access via entire rewind.

I think the mismatch here is how we're using the one-shot RunPrompt (which is 'ending' the session), I'll do a follow up where we hold the session open to better reflect the scenario. 👍