feat(agentic): add live agent follow and status - #1318
Conversation
|
@openhands-agent model=openrouter/qwen/qwen3.7-max provider=openrouter effort=xhigh output=pr-comment iterations=100 use harness Evaluate PR #1318 as the independent IMPL-EVAL for issue #1115 on the Verify both the issue body and its owner addendum, especially:
Run the smallest proving validation and return PASS or FAIL_FIX with concrete findings. SKILL
OUTPUT CONTRACT (mandatory — verdict first)
|
OpenHands Agent — CompletedOPENHANDS_VERDICT: PASS Model: IMPL-EVAL Summary: PR #1318Issue#1115: agentic: no way to see what a running agent is doing — add follow + live state for Codex and agy ChangesPR #1318 adds live agent observability across Codex and agy runtimes:
ValidationTests: 18/18 pass (7 rollout-live, 3 follow, 3 agy-live, 5 status)
Type-check: Clean across all 5 modified files Lint: No new ignores added (grep confirmed) Lock hygiene: No Documentation: Acceptance Criteria
Remaining RisksNone. The implementation is clean, well-tested, and meets all acceptance criteria. The watcher-before-read pattern in OPENHANDS_VERDICT: PASS Run: https://github.com/rickylabs/netscript/actions/runs/30990244817 |
|
[PHASE: IMPL-EVAL] [VERDICT: PASS] Evaluation SummaryPR #1318 implements live agent follow and status for Codex and agy runtimes, addressing issue #1115. All six acceptance criteria verified: 1. Rollout resolution shared ✓ 2. Evidence-derived status ✓ 3. Codex identity complete ✓ 4. Agy agents alongside Codex ✓ 5. Documentation discoverable ✓ 6. Quality gates ✓ OPENHANDS_VERDICT: PASS |
|
@openhands-agent model=openrouter/qwen/qwen3.7-max provider=openrouter effort=xhigh output=pr-comment iterations=75 use harness Evaluate PR #1318 as the independent IMPL-EVAL for issue #1115 on the Verify both the issue body and its owner addendum, especially:
Run the smallest proving validation and return PASS or FAIL_FIX with concrete findings. SKILL
OUTPUT CONTRACT (mandatory — verdict first)
|
OpenHands Agent — CompletedOPENHANDS_VERDICT: PASS Model: IMPL-EVAL SummaryPhase: IMPL-EVAL Verdict: PASSAll acceptance criteria satisfied. Implementation is correct, complete, and safe. ChangesPR #1318 adds New Files (8)
Modified Files (11)
Total: +1,156 lines, -89 lines (19 files) Validation1. Shared Rollout Resolution ✅Both Evidence: // codex-follow.ts:10
import { resolveCodexRollout } from './codex-rollout-live.ts';
// codex-watch.ts:47
import { resolveCodexRollout } from './codex-rollout-live.ts';2. Terminal Exit Handling ✅Follow tool exits cleanly on Test output: 3. Live Streaming ✅Streams reasoning, messages, commands, command exits, and file writes in real-time. Filters out noisy events (session_meta, turn_context). Pretty and JSON formats supported. Verified events:
4. Agentic Runtime Support ✅
Test output: 5. Documentation ✅
6. Tests Deterministic ✅18 tests pass using fake clocks and fixtures. No flaky timing dependencies. Test summary: Key FindingsRace-Condition MitigationThe follow tool arms Prose Cannot Fabricate Terminal StateThe test "quoted quota and refusal prose cannot fabricate a terminal state" proves that the implementation correctly distinguishes between:
This validates the core safety property: the follow tool won't exit prematurely based on agent output text that happens to mention completion or failure keywords. State Machine Correctness
The state is derived from trusted rollout events and recency, not process liveness or quoted prompt prose. ComplianceLockfile Hygiene ✅
Lint Compliance ✅
Compatibility Contract ✅
Harness Artifacts ✅All required artifacts present:
Remaining RisksNone identified. The implementation is complete and correct. ConclusionPR #1318 successfully implements live agent follow for Codex and agentic runtimes. The shared rollout resolution, terminal exit handling, live streaming, and mixed-fleet status are all correctly implemented and thoroughly tested. The critical safety property (prose cannot fabricate terminal state) is validated by explicit test coverage. Recommendation: Merge to canary/0.0.5-canary.13. OPENHANDS_VERDICT: PASS Run: https://github.com/rickylabs/netscript/actions/runs/30991295191 |
|
[PHASE: IMPL-EVAL] [VERDICT: PASS] IMPL-EVAL Evaluation: PR #1318 → Issue #1115Evaluator: Verdict: PASSAll 6 acceptance criteria satisfied. Implementation is correct, complete, and safe. 1.
|
| Check | Status |
|---|---|
| Shared rollout resolution | ✅ |
| Terminal exit without race | ✅ |
| State from events, not process | ✅ |
| Prose cannot fake terminal state | ✅ |
| Agentic alongside Codex | ✅ |
| Worktree-resolved agents | ✅ |
| Non-zero exit surfaced | ✅ |
| Documentation discoverable | ✅ |
| Tests deterministic | ✅ |
| No new lint ignores | ✅ |
| Compat contract holds | ✅ |
| deno.lock absent | ✅ |
Conclusion
This PR fully satisfies the issue's acceptance criteria and the owner addendum's expanded requirements for agentic runtime support. The implementation is architecturally sound (shared resolution, event-driven state machine, race-free watcher arming), thoroughly tested (18 tests covering all state transitions), and correctly documented in both the tooling README and the harness workflow index.
Recommendation: Merge to canary/0.0.5-canary.13.
OPENHANDS_VERDICT: PASS
Closes #1115
Summary
agentic:codex-follow, a readable JSONL/pretty live stream resolved by thread id through thesame shared resolver as
codex-watchagentic:codex-statusreport evidence-derived working, idle, stalled, dead, and refusedstates with model, effort, worktree, current reasoning, and commit/file artifacts
dispatch issue, current step, non-zero command exits, and artifact evidence
Harness
.llm/runs/feat-agent-follow-live-1115--canary13/canary/0.0.5-canary.13COMPOSEDper milestone-run D6deno.lockmodification excludedValidation
--worktree /home/codex/repos/ns-quickstartresolved one conversation with issue,current step, and commit