Skip to content

Releases: Smana/ai-native-ref

v3.0.0: Agent Factory v3

Choose a tag to compare

@Smana Smana released this 31 Oct 15:01
3f2341e

Agent Factory v3: tasks are shaped before code is written, code is checked before it is pushed, and one maintainer approval merges an agent's pull request. The v3 features were proven live on GCP (GKE); the merge gate's human path on AWS (EKS).

What v3 is

The Agent Factory turns a labelled issue into a reviewed pull request, built by agents in a gVisor sandbox under their own identity. It is a bundle of open-source tools: Maku (the factory controller, the rooms, the harness and bridge, the Slack app and the maku CLI), agentgateway, OpenHands, gVisor, octo-sts, RunLore, Kueue and others.

v3 aims for more accepted pull requests per dollar on the models each kind of work may use: public work on open-weight models, internal work on Claude. It works on the task, not the model: it shapes the task before any code is written, checks the code before the push, judges the result against the task's own acceptance checks, and reduces the human's part of a merge to one approval. The eval bench that would measure that aim is deferred.

Proven live

Every feature was proven offline first (unit, golden and container suites, rule tests, drift gates), then tested live on two clouds: GCP (GKE) and AWS (EKS).

GCP (GKE), Maku v0.24 to v0.30.3: the v3 features

Feature Evidence
Scoper: a bounded task, or one question to the author Layer 1
Completion judge, including impossible (no false met) Layer 2
Sandbox checks: a seeded defect fixed before the push, 0 fix runs Layer 3
Test-first tester: a contract committed first; not_testable runs without one runbook 21
Splitter: 3 parts in 8 min, with a merge order runbook 17
Slack intake: draft, edit, ✅, the thread follows the room runbook 22
The factory from Claude Code and OpenCode runbook 19
RunLore agent to agent: alert → investigation → factory/publish → fix PR → merge → soak → resolved runbook 20
Capability profiles: an internal run naming a public-only alias is refused at admission runbook 14
Kimi K3.1 Pro as reviewer and Mistral Medium as implementer, cost within 1% of the gateway providers
Spend per identity matches the gateway per run runbook 04

AWS (EKS), Maku v0.32.0 to v0.32.3: the merge gate's human path. Full record: AWS results.

Proof Result
Approve-to-merge (N6) PASS on v0.32.3. The owner approved #271 at 32d1731b; the merger App merged it at that head 14 s later. Task Done / merged, mergedHow: approved, approvedBy: Smana, no armedAt (outside the caps), labels factory/merged:docs + factory/approved-merge. On v0.32.0 the merge was right but its record was lost (maku #302, fixed)
Approval, then a force-push (A7) Safe: the stale approval was not counted; no merge
A hand-off names one action and writes a replayable decision record PASS
Regression breaker and recovery (M1, N8) A regression paused the class; three clean merges lifted it (recoverAfter: 3)
Follow-up detection (M2) 1/1 follow-up labelled, 1/1 unrelated PR left alone
Shadow while paused (AM4) and under a cap (N7) shadow_would_arm recorded with paused=true, and on cloud-native-ref#2700 with capped=true
dead-links drill (L2) Agent PR cloud-native-ref#2697 auto-merged and verified by the oracle. factory/drill + factory/revert → one revert PR, #2698, which the merger merged unattended (maku#158). Tally merges=1 drill=1 bad=0. The next decision, on #2700, read paused=false: the drill was not counted. The class is back in shadow
Request changes, then approve (A5) PARTIAL: the revision run started; the return on approval was not read

Not run: N1 policy replay, N3 release age, #182 the AwaitingCI bound, A4 the next-day return, G1 the host docs-links lift.

Features by area

Tasks before code

  • Scoper: a labelled issue becomes a bounded task with named acceptance checks, or one question to its author (factory/needs-author). An inline Acceptance: paragraph counts. The harness nudges it at scoper.nudgeAt and stops it at maxToolCalls (budget_calls).
  • Splitter: an oversized issue is proposed as parts, each its own sub-issue and PR, in an explained merge order; a maintainer accepts with /factory split.

Code before review

  • Sandbox checks: the implementer runs the repository's checks before it pushes. CI stays the judge of record.
  • Test-first tester (session switch): a failing test contract is committed first.
  • Search for agents: a read-only, fresh-context search tool answers the implementer's questions.
  • Completion judge: met, not_yet or impossible against the scoper's checks; continuations are time-boxed; a not_yet waiting only on pending checks waits within StuckAfter.
  • Reviewer: runs after the PR exists, with the task's brief.

Merging: the human path (Maku v0.32.0; see agents and the merge gate)

  • merge.approveToMerge: a maintainer's approval of the current head merges an agent PR outside the caps (factory/approved-merge). Since v0.32.2 an approval binds to the head's push time.
  • A handed-off task returns to the gate on an approval, on the next UTC day, or when a pause lifts.
  • factory/drill, set before factory/revert, makes a revert a drill the breaker does not count. merge.breaker.recoverAfter resumes a paused class after N clean, mature merges.
  • Shadow decisions no longer stop at class_paused or auto_merge_cap. Every decision writes an agent-factory-decision record; shadow-merge-tally.sh --replay re-reads it.
  • Metrics maku_factory_class_paused and maku_factory_handoff_to_merge_seconds; a "Human path" dashboard row; the AgentFactoryClassPausedLong alert; runbook 12.
  • Classes: docs-links is live; dead-links passed one live drill and is back in shadow; deps-patch is in shadow; deps-minor observes.

Harness roles (Maku v0.32.0, harness on AWS since crossplane-configuration v0.9.31)

  • The judge waits for pending checks and quotes the failing log; scripts has a judge.
  • Run cost by role (run_cost_usd{role}) and its panel; a timed-out model call is retried once; steering notes.

Intake

  • Slack (maku-slack): mention the factory, edit the draft in the thread; a maintainer's ✅ files the issue and starts the task.
  • Coding agents: Claude Code and OpenCode file, start, follow and check out work through maku, its MCP server and the factory-handoff skill (coding clients).
  • RunLore (session switch): an alert becomes an investigate task; factory/publish turns its proposal into a fix PR.

Models and spend

  • Capability profiles: a model allow-list per role and data class, compiled into the gateway's rules; every run is pinned to one model.
  • Moonshot (Kimi K3.1 Pro) and Mistral (Medium, Small), public data only, selected per class with loops.<class>.models.
  • Spend per identity with a cost cross-check alert; cache-read alerts; the daily task cap refuses at intake (429 daily_task_cap).

Rooms

  • The approvals page, a status line per room, forking a room into an attended run.
  • The room list refreshes on its own (10 s default, pausable) and the title counts rooms waiting on you. The run strip labels each fact and has a Next box with commands to copy.

Names and docs

  • The component formerly named agent-platform is now Maku: images, charts, CLI, CRD group maku.ogenki.io, metrics maku_factory_* / maku_rooms_*. The product is the Agent Factory.
  • The docs are reorganised into use / how it works / tune, w...
Read more

v2.0.0: agent factory v2

Choose a tag to compare

@Smana Smana released this 18 Oct 19:14
9edf75e

The agent factory's v2.0, running on gcp-0. It includes the GKE work first planned as v2.1. Status and evidence: Status and roadmap.

Landed

Feature Evidence
The factory on gcp-0: the whole factory runs on GKE alone, from release pins runbook 14
Live auto-merge of Markdown link fixes, at most one a day. The merger App merges at the head the gate decided (R02, R52), and a factory/revert opens the merger's revert and demotes the class runbook 12
Gateway budgets enforced: per-run, fleet and Anthropic buckets refuse with 429 agent_budget_exhausted, proven live; so is the shadow-overrun alert runbook 04, Part D
Suspend: a run waiting on a human gives up its pod and resumes with the same workspace and conversation runbook 01
A second repository (ai-native-ref), fenced by .github/agent-factory.yaml results
Warm sandboxes, off by default: runs start in 16–17 s instead of minutes design
agentgateway with model tiers; internal work on Claude with prompt caching ADR-0053
Own repository and site: signed OCI artifact, pinned by cloud-native-ref ADR-0057
Docs: model routing and cost, harness configuration and tuning routing, harness

Fixed by the live drill (agent-platform)

  • #154: handoffs and verdicts carry the full 40-hex head SHA.
  • #157: R52 reads the base at the merge base, and its code-fence scan leaves a fence.
  • #158: the merger merges a green revert at its head, since native auto-merge stays BLOCKED under agent-merge.

After v2.0

  • Per-person caps stay in shadow until cache reads count at their discounted weight (agent-platform #155).
  • The warm pool (binder W1) and the Substrate runtime, on the roadmap.
  • The refusal alert (ai-native-ref #81): agentgateway does not count a budget refusal.
  • Open issues: agent-platform #156, ai-native-ref #75 and #76.

Pins: agent-platform v0.15.6.