What's Changed
- fix: correct factory ceo examples to use positional prompt by @akashgit in #96
- fix: use descriptive experiment branch names by @shivchander in #97
- ci: add Codecov coverage reporting by @akashgit in #98
- fix: add CODECOV_TOKEN to coverage upload by @akashgit in #99
- fix: drop OBSIDIAN_VAULT_PATH fallback to prevent personal vault writes by @akashgit in #100
- ci: run CI on push to main for Codecov baseline by @akashgit in #101
- feat: experiment-level checkpoint/resume for CEO agent by @shivchander in #102
- fix: strip calendar-time estimates from agent pipeline by @akashgit in #104
- feat: unified backlog for improve-mode convergence by @akashgit in #105
- docs: update docs and README for unified backlog by @akashgit in #106
- feat: redefine --focus as targeted mode by @akashgit in #107
- docs: update docs and diagrams for targeted mode (--focus) by @akashgit in #108
- docs: fix --prompt usage across README and docs by @akashgit in #111
- feat: add Distiller agent and --interactive ideation mode by @akashgit in #110
- docs: restructure README and docs around Build / Improve / Focus workflows by @akashgit in #112
- feat: add session summary report at end of factory cycles by @akashgit in #114
- Fix factory_effectiveness scoring — wire finalize scores + auto-discovery by @akashgit in #118
- Fix research_grounding cold-start — citation backfill + extraction by @akashgit in #120
- fix: CEO agent must not auto-merge PRs by @akashgit in #121
- fix: interactive mode treats idea string as file path by @akashgit in #125
- docs: add meta mode cadence guidance by @Maxusmusti in #116
- feat: add Claude Code plugin/marketplace with /factory:implement by @xukai92 in #117
- feat: add Bob Shell runner support as alternative CLI backend by @colehurwitz in #113
- release: v0.2.0 by @akashgit in #130
- Add operational hypothesis support to Strategist and CEO prompts by @akashgit in #145
- fix: CEO must propagate and enforce operational execution steps by @akashgit in #148
- fix: add backlog item verification gates to CEO prompt by @akashgit in #149
- fix: move backlog verification before finalize to prevent stale metadata by @akashgit in #151
- feat: add research mode models, config parsing, and CLI wiring by @akashgit in #153
- feat: add research run infrastructure — runner, parser, store by @akashgit in #155
- feat: add research mode context to Strategist prompt by @akashgit in #157
- feat: CEO prompt — research mode cycle protocol by @akashgit in #159
- feat: add Researcher step (R1.5) to research mode cycle by @akashgit in #160
- feat: ground truth leakage guards for research mode by @akashgit in #161
- feat: research ideation mode for --mode research by @akashgit in #162
- feat: raise per-cycle limit to 8, remove daily limit by @colehurwitz in #136
- feat: add --no-github flag to disable GitHub operations by @lukeinglis in #163
- Fix Live Events panel expanding without bounds by @lukeinglis in #169
- docs: lifecycle narrative, research mode, updated diagrams by @akashgit in #172
- docs: revamp README with workflow walkthroughs and SVG diagrams by @akashgit in #174
- feat: overhaul ACE data pipeline — performance reports, registry, vault removal by @akashgit in #176
- docs: update for vault→archive migration by @akashgit in #177
- docs: add 'Built with the Factory' showcase section by @akashgit in #178
- feat: add real-time factory visualizer to web dashboard by @abhi1092 in #180
- Fix research_grounding eval to use .factory/archive/ as knowledge source by @abhi1092 in #182
- fix: make heartbeat loop interruptible by Ctrl+C/SIGTERM during sleep by @RohanAwhad in #193
- feat: expose factory agents as plugin subagents by @xukai92 in #139
- feat: finalize gate + hard constraints by @RobotSail in #195
- fix: force-add agents/ in plugin workflow to bypass .gitignore by @xukai92 in #197
- feat: add pipeline skills for dynamic multi-agent orchestration by @xukai92 in #198
- feat: sprint-based resume with scrum master agent by @shivchander in #137
- feat: user-to-CEO message channel by @RobotSail in #194
- feat: close plugin-vs-CLI gaps (#199) by @xukai92 in #201
- feat: expand --focus to accept issue refs (closes #202) by @akashgit in #213
- feat: use --append-system-prompt for headless, strengthen CEO identity by @akashgit in #221
- Add --force flag to factory finalize to bypass precheck gate by @akashgit in #232
- feat: add --dir parameter to factory ceo for explicit project directory naming by @johncohn in #228
- feat: smart project directory naming from descriptions by @akashgit in #233
- docs: add SDG Hub to Built with Factory showcase by @shivchander in #214
- docs: add GSD comparison section to README by @gx-ai-architect in #203
- docs: update all docs for scrum master agent and sprint-based resume by @shivchander in #204
- feat: consolidate factory configuration into config.toml by @akashgit in #241
- Fold Scrum Master into CEO prompt (exp #59) by @akashgit in #243
- feat: review-until-clean pipeline loop and draft PR flow (#240) by @akashgit in #255
- feat: support --mode interactive on existing projects by @akashgit in #254
- fix: unify on bare 'factory' CLI + fix path resolution (#239) by @akashgit in #252
- Branch isolation via git worktrees (Issue #240 Part 1) by @akashgit in #257
- Enforce mandatory review pipeline — Sacred Rule 9 + prompt hardening by @akashgit in #267
- feat: add factory backfill-archive CLI command by @akashgit in #259
- feat: research mode dashboard view (Issue #264) by @akashgit in #268
- Fix worktree creation: detect default branch instead of hardcoding 'main' by @akashgit in #271
- Fix interactive mode on existing projects — banner, CEO mode, Phase 0e research by @akashgit in #275
- Fix committed .factory symlink and add defensive mkdir helper by @akashgit in #278
- fix: interactive mode crash on file input and empty-repo worktree creation by @mairin in #280
- fix: persist spec for raw-idea interactive mode by @akashgit in #281
- fix: provide git identity in _ensure_repo for CI by @akashgit in #282
- fix: correct HypothesisBudget fields and harden read_config by @akashgit in #290
- docs: document interactive mode file input in README by @akashgit in #284
- README overhaul: restructure around 3 workflows, cut to ~200 lines by @akashgit in #293
- feat: add Codex runner for OpenAI Codex CLI support by @lukeinglis in #291
- Unify Phase 0 and Phase 0e — full ideation flow for existing projects by @akashgit in #298
- Add eval_spec: testing instructions for Evaluator context injection by @akashgit in #303
- feat: redesign welcome wizard as conversational agent by @akashgit in #305
- Welcome Wizard — interactive CLI entry point by @akashgit in #301
- Add OpenSkies Airline Corpus to Built with the Factory by @lukeinglis in #295
- Fix wizard ENAMETOOLONG crash and add explicit --mode flags by @akashgit in #311
- feat: promote eval_spec to scored dimensions + within-tier weight config by @akashgit in #315
- CEO: add progress tracking task list to every cycle by @akashgit in #322
- CEO identity drift defense — re-anchor at agent transitions by @akashgit in #318
- Fix inner/outer loop semantics and add validated interactive extraction by @akashgit in #328
- Implement inner/outer loop Python infrastructure by @akashgit in #336
- Fix inner/outer loop semantics and add validated interactive extraction (issue #326) by @akashgit in #338
- Implement inner/outer loop Python infrastructure (issue #333) by @akashgit in #339
- Fix review pipeline counter starvation: separate 2d-review and 2h-final counters by @akashgit in #331
- Enforce spec depth in Distiller — grounding protocol + quantified constraints by @akashgit in #341
- Wizard: redirect long input to file before classification by @akashgit in #334
- Add session naming to factory-spawned Claude Code sessions by @akashgit in #337
- feat: LLM-driven user profiling system by @akashgit in #343
- Add Refinement Stage — Refiner agent + --refine CLI flag + CEO Mode: Refine by @akashgit in #319
- Add explicit inner/outer loop controls to research mode (issue #313) by @akashgit in #323
- Fix inner/outer loop field name mismatch in Distiller/CEO prompts by @akashgit in #353
- docs: document inner/outer loop config for research mode by @akashgit in #355
- fix: derive smart session names from user intent by @akashgit in #348
- docs: document post-cycle refinement loop and Refiner agent by @akashgit in #357
- Clean PR mode: strip non-essential artifacts from PRs by @akashgit in #363
- docs: document Clean PR Mode by @akashgit in #371
- fix: defer project directory creation until just before worktree setup by @akashgit in #361
- docs: rebrand to re:factory and add logo by @colehurwitz in #377
- chore: update README.md to remove double re:factory title by @colehurwitz in #389
- feat: Codex CLI plugin support (manifest, agent generation, skills) by @lukeinglis in #395
- fix: stop treating the factory's own 'implementation' label as an unbuilt-repo signal (#378, #382) by @colehurwitz in #383
- Bob runner: strip ANSI escapes on live-terminal write to stop flicker by @colehurwitz in #381
- feat: token usage auditing and profiling across factory workflow by @akashgit in #412
- fix: increase timeout durations for eval and agent invocations by @akashgit in #415
- fix: wizard _quick_classify bypasses LLM for long inputs by @akashgit in #410
- fix: use sys.executable instead of bare 'python' in hygiene eval by @akashgit in #419
- feat: tmux persist mode for CLI runners by @xukai92 in #285
- fix: write agent prompts to temp files to avoid ARG_MAX limit by @Maxusmusti in #422
- ci: rename plugin branch to plugins and document plugin install by @xukai92 in #397
- feat: add --mode review --pr N for on-demand CEO PR review by @xukai92 in #421
- ci: debug ceo-review workflow for CI testing by @xukai92 in #434
- chore: test ceo-review workflow by @xukai92 in #435
- ci: auto-approve PR when CEO review verdict is KEEP by @xukai92 in #438
- fix: address PR #443 review — interactive mode + Codex auth by @akashgit in #468
- fix: ceo-review falls back to PR comment when formal review fails by @akashgit in #469
- feat: runner abstraction v2 — universal multi-runner factory by @akashgit in #443
- fix: auto-approve checks formal reviews, not stale comments by @akashgit in #477
- fix: skip test_available_runners_detected in CI by @akashgit in #481
- ci: allow Cole and Shiv to trigger @ceo-review by @akashgit in #482
- ci: allow osilkin98 and gx-ai-architect to trigger @ceo-review by @akashgit in #495
- fix: detect OpenCode npm vs Go binary incompatibility at runtime by @akashgit in #502
- fix: Bob RunnerMeta.check_auth() supports file-based auth by @akashgit in #503
- fix: replace tmux wait-for with Stop-hook sentinel for completion signaling by @shivchander in #500
- fix: address 6 confirmed review issues on PR #514 by @colehurwitz in #520
- feat: parallelize researcher spawning with --review-tag (fixes #524) by @akashgit in #528
- feat: replace wall-clock timeout with inactivity-based timeout (issue #525) by @akashgit in #531
- feat: merge Distiller into Strategist — dual-mode agent by @akashgit in #530
- fix: align Strategist ideation output with phased build-plan format by @akashgit in #534
- Fix 6 CI test failures: review_tag mocks, prompt assertion updates by @colehurwitz in #541
- fix: eliminate eval duplication for 4x speedup (#536) by @colehurwitz in #538
- Implement 6 stubbed eval dimensions + 3 combined methods for Go/Node/Rust by @colehurwitz in #548
- fix: eval language evaluator gaps — silent failures and scoring blindness by @colehurwitz in #551
- refactor: interactive_run() reuses build_command() across all runners by @gx-ai-architect in #552
- fix: make test_timeout configurable to handle long test suites by @colehurwitz in #559
- feat: enrich factory lifecycle events for external consumers by @colehurwitz in #557
- refactor: rename interactive mode to design by @xukai92 in #563
- docs: add abstract meta-harness specification by @xukai92 in #423
- Add Langfuse tracing for re-factory by @gx-ai-architect in #577
- Session persistence: capture and view CEO + specialist agent conversations by @gx-ai-architect in #569
- feat: add benchmark CI pipeline for pre-release validation by @RobotSail in #590
- feat: add PR comments and release notes to benchmark workflow by @RobotSail in #592
- fix: benchmark workflow skips all jobs on PR trigger by @RobotSail in #594
- fix: wire up pr_number input for workflow_dispatch by @RobotSail in #598
- fix: add missing pr_number input to workflow_dispatch by @RobotSail in #599
- fix: disable benchmark CI from running on every PR by @RobotSail in #601
- feat: add ProgramBench support to benchmark CI pipeline by @RobotSail in #600
- feat: --bg flag for background agent dispatch (rework of PR #424) by @xukai92 in #571
- Fix 4 mypy errors causing CI lint failure on main by @xukai92 in #605
- feat: run factory CEO as TerminalBench solver + fix GCP auth for CI by @RobotSail in #603
- feat: run factory CEO as solver for SWE-bench and FeatureBench by @RobotSail in #609
- fix: install factory CLI in benchmark GHA workflow by @RobotSail in #611
- feat: run factory CEO as ProgramBench solver by @RobotSail in #610
- fix: use git fsck instead of reflog for worktree recovery by @RobotSail in #612
- Unify Design Mode and Build Mode into a single Plan Loop by @akashgit in #615
- fix: make GCP credentials readable by agent user in ProgramBench container by @RobotSail in #617
- fix: replace :ro bind mount with docker cp for GCP credentials in ProgramBench by @RobotSail in #620
- feat: add --solver flag to benchmark CI for factory vs claude-code comparison by @RobotSail in #624
- fix: pass --extra-instruction-path for claude-code solver in TerminalBench by @RobotSail in #629
- feat: add cost and token tracking to benchmark results by @RobotSail in #635
- fix: read correct cost field name from Claude Code JSON output by @RobotSail in #638
- feat: add benchmark result history tracking and baseline comparison by @RobotSail in #639
- fix: YAML syntax error in benchmark workflow (inline Python breaking parser) by @RobotSail in #640
- Remove dead smoke_test_command parameter from run_precheck by @akashgit in #633
- feat: add benchmark dashboard and remove broken history commit by @RobotSail in #647
- Archivist async redesign: structured output, haiku model, reduced invocations by @akashgit in #636
- feat: benchmark data branch storage and dashboard charts by @RobotSail in #648
- fix: benchmark-data branch creation in report job by @RobotSail in #650
- fix: debug and fix benchmark data append to benchmark-data branch by @RobotSail in #652
- fix: move artifact download after checkout so results/ survives by @RobotSail in #654
- feat: dashboard UI overhaul and nightly scheduled benchmark runs by @RobotSail in #655
- fix: dashboard data pipeline — solver field, cost extraction, chart labels by @RobotSail in #658
- revamp: benchmark dashboard with main-branch filtering, PR comparison, full-width charts by @RobotSail in #661
- revamp: benchmark dashboard layout — averages first, grouped history by @RobotSail in #663
- fix: replace average duration/cost bar charts with line charts over time by @RobotSail in #664
- revamp: complete rewrite of benchmark dashboard (v4 spec) by @RobotSail in #667
- feat: add methodology section to benchmark PR comments by @RobotSail in #668
- feat: add non-blocking benchmark status check to PRs by @RobotSail in #669
- fix: benchmark-eval check only passes when factory jobs ran by @RobotSail in #671
- fix: enhance PR comment with dimension-level comparison vs main baseline by @RobotSail in #672
- fix: benchmark-eval status check requires all 4 factory benchmarks by @RobotSail in #673
- feat: increase default benchmark timeout to 7200s by @RobotSail in #674
- fix: move worktrees to .factory-worktrees/ to eliminate ELOOP by @akashgit in #676
- feat: SPEC.md-aware Plan Loop — Strategist reads SPEC.md, plan grounded in spec diff by @xukai92 in #622
- fix: benchmark CI — terminalbench resolved:false, $0 costs, nightly schedule by @RobotSail in #679
- fix: remove stale factory eval patterns from CEO prompt, add QA delegation tests by @akashgit in #683
- fix: enable claude-code solver in nightly benchmark schedule by @RobotSail in #678
- fix: ProgramBench resolved threshold requires 100% test pass rate by @RobotSail in #687
- fix: create project directory before git worktree prune by @RobotSail in #688
- Integrate Sentrux architectural quality check into CI by @xukai92 in #660
- ci: release benchmark notes + on-merge PR testing by @RobotSail in #695
- fix: design mode shows 'Mode: design' instead of 'Mode: build' in CEO task string by @xukai92 in #692
- perf: add pagination to benchmark dashboard run history by @RobotSail in #719
- fix: streaming CEO tracing + trace name corruption by @gx-ai-architect in #722
- Standardize review mode to reuse Improve pipeline QA steps by @xukai92 in #701
- fix: switch Claude runner to stream-json output to prevent watchdog kills by @gx-ai-architect in #725
- fix: FeatureBench harness — improve --focus with pre-seeded factory state by @RobotSail in #734
- feat: add architecture hygiene dimension via Sentrux integration by @xukai92 in #739
- feat: CI eval baseline pipeline + factory baseline CLI command by @RobotSail in #740
- fix: increase asyncio StreamReader limit to 1MB by @RobotSail in #742
- Feat/workflow graph engine by @xukai92 in #744
New Contributors
- @Maxusmusti made their first contribution in #116
- @xukai92 made their first contribution in #117
- @colehurwitz made their first contribution in #113
- @lukeinglis made their first contribution in #163
- @abhi1092 made their first contribution in #180
- @RohanAwhad made their first contribution in #193
- @johncohn made their first contribution in #228
- @gx-ai-architect made their first contribution in #203
- @mairin made their first contribution in #280
Full Changelog: https://github.com/akashgit/remote-factory/commits/v0.2.0
Benchmark Results
| Benchmark | Solver | Instance | Result | Score | Cost | Duration |
|---|---|---|---|---|---|---|
| swebench | factory | sympy__sympy-20590 |
✅ | 1 | $3.65 | 539s |
| terminalbench | factory | fix-git |
❌ | 0 | $0.00 | 1005s |
| featurebench | factory | pypa__packaging.013f3b03.test_metadata.e00b5801.lv1 |
❌ | 0 | $5.43 | 1074s |
| programbench | factory | abishekvashok__cmatrix.5c082c6 |
❌ | 0 | $9.93 | 1852s |