(MOT-4269) fix(harness): stabilize reactive E2E execution

[ytallo]

Summary

• prevent quickstart-only changes from selecting the harness integration and real-model E2E suites
• replace fixed wall-clock scenario deadlines with no-progress watchdogs and preserve partial metrics/transcripts on failure
• let reaction-spawned turns inherit the registering session's policy, filesystem scope, model limits, and shared budget
• notify the registering session when reactive execution fails
• validate database watches and state fallback according to the existing reactive automation prompt
• give the large reactive scenario enough shared token budget to complete its writers, reactors, and finalizer

Root cause

The failing workflow was selected by a quickstart-only change because all source changes under harness/ were treated as integration and E2E inputs. Once selected, the reactive scenario exposed three additional reliability gaps: reaction-spawned turns lost durable owner context, the runner used a fixed wall-clock deadline while returning empty metrics for incomplete session trees, and the one-million-token shared budget could be exhausted by discovery before the three writers started. Those behaviors could stop a progressing run and leave little diagnostic evidence.

The reactive automation prompt itself is unchanged. Its existing exact requirements for 15 events and rows, trigger provenance, the named finalizer, and cleanup remain enforced.

Regression run: https://github.com/iii-hq/workers/actions/runs/30410549175

Validation

• cargo test --locked --manifest-path harness/Cargo.toml -p harness
• cargo test --locked --manifest-path harness/Cargo.toml -p harness-e2e
• cargo clippy --locked --manifest-path harness/Cargo.toml -p harness -p harness-e2e -p harness-integration --all-targets -- -D warnings
• make -C harness integration-validate (11 fixtures)
• pinned-engine INT-005 reaction policy inheritance scenario (pass)
• local real-stack reactive_automation with zai/glm-5.2 (100/100, 1/1 pass)
• direct Python assertions for quickstart, E2E, and harness path routing
• git diff --check

Fixes MOT-4269
Refs MOT-4253

Summary by CodeRabbit

• New Features

  • Partial session metrics now preserve observable usage data while clearly indicating incomplete results.
  • Reactive child failures can notify the session that registered the reaction.
  • Reactive child sessions now retain ownership context for more reliable inheritance and failure handling.

• Bug Fixes

  • End-to-end monitoring now detects stalled executions based on observable progress, including metrics updates.
  • Stalled or failed executions can capture available partial metrics and transcripts before reporting errors.

• Tests

  • Expanded coverage for partial metrics, reactive ownership, fallback documentation, and quickstart changes.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
workers	Ready	Preview, Comment	Jul 29, 2026 2:36am
workers-tech-spec	Ready	Preview, Comment	Jul 29, 2026 2:36am

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bd911774-6804-4fe1-98f2-704ccdec77d8

📥 Commits

Reviewing files that changed from the base of the PR and between a7b2c39 and f3dfa62.

📒 Files selected for processing (17)

• .github/scripts/discover_changed_workers.py
• .github/scripts/tests/test_discover_changed_workers.py
• harness/src/functions/metrics.rs
• harness/src/functions/react.rs
• harness/src/functions/send.rs
• harness/src/subagent.rs
• harness/src/turn_loop.rs
• harness/src/types/turn.rs
• harness/tests/e2e/src/context.rs
• harness/tests/e2e/src/report.rs
• harness/tests/e2e/src/scenarios/direct_answer.rs
• harness/tests/e2e/src/scenarios/mod.rs
• harness/tests/e2e/src/scenarios/persistent_state.rs
• harness/tests/e2e/src/scenarios/reactive_automation/evaluate.rs
• harness/tests/e2e/src/scenarios/reactive_automation/mod.rs
• harness/tests/e2e/src/scenarios/security_review.rs
• harness/tests/e2e/src/suite.rs

---

📝 Walkthrough

Walkthrough

The PR updates suite-change detection, partial metrics aggregation, reactive child ownership and failure notifications, and E2E execution watchdogs. Scenario evaluation also gains revised fallback-documentation rules and associated tests.

Changes

Suite change detection

Layer / File(s)	Summary
Exclude quickstart-only changes .github/scripts/discover_changed_workers.py, .github/scripts/tests/test_discover_changed_workers.py	Quickstart edits no longer trigger integration or E2E suite selection, with coverage added for the behavior.

Partial session metrics

Layer / File(s)	Summary
Produce live partial snapshots harness/src/functions/metrics.rs	Incomplete session trees continue aggregating observed usage, report complete=false, omit traces, and preserve partial totals.

Reactive child ownership

Layer / File(s)	Summary
Persist reactive owner metadata harness/src/types/turn.rs, harness/src/subagent.rs, harness/src/functions/send.rs	TurnRecord stores the optional registering session, and reactive child seeding propagates it.
Use owner-aware reactive spawning harness/src/functions/react.rs, harness/src/subagent.rs	Reactive spawns use typed requests and owner-aware inheritance when an owner session exists.
Notify reaction owners on failure harness/src/functions/react.rs, harness/src/turn_loop.rs	Failed reactions notify the owner session while preventing duplicate and self-notifications.

E2E progress watchdog

Layer / File(s)	Summary
Adopt stuck-timeout policies harness/tests/e2e/src/scenarios/*, harness/tests/e2e/src/report.rs	Scenario policies replace fixed timeout fields with stuck-timeout settings and increase the reactive automation token budget.
Track observable progress harness/tests/e2e/src/context.rs	Turn and metrics waits detect progress through root status and metrics snapshots, stopping session trees after inactivity.
Capture failed-run observations harness/tests/e2e/src/suite.rs	Execution captures partial metrics and transcripts after wait failures and recognizes no-progress failures.
Validate fallback documentation harness/tests/e2e/src/scenarios/reactive_automation/evaluate.rs	Fallback scoring distinguishes database and state mechanisms and tests their documentation requirements.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ReactionRegistration
  participant ReactiveSpawner
  participant OwnerSession
  participant ChildTurn
  ReactionRegistration->>ReactiveSpawner: submit typed spawn request
  ReactiveSpawner->>OwnerSession: load owner turn record
  OwnerSession-->>ReactiveSpawner: return inherited owner metadata
  ReactiveSpawner->>ChildTurn: seed reactive child
  ChildTurn-->>OwnerSession: send failure notification when execution fails

Loading

sequenceDiagram
  participant E2eSuite
  participant E2eContext
  participant SessionStatus
  participant SessionMetrics
  participant SessionTree
  E2eSuite->>E2eContext: wait using stuck timeout
  E2eContext->>SessionStatus: poll root status
  E2eContext->>SessionMetrics: poll metrics
  SessionStatus-->>E2eContext: status snapshot
  SessionMetrics-->>E2eContext: metrics snapshot
  E2eContext->>SessionTree: stop tree after no observable progress
  E2eContext-->>E2eSuite: return result or failure observation

Loading

Possibly related PRs

• iii-hq/workers#461: Both changes modify child seeding and inheritance in harness/src/subagent.rs.
• iii-hq/workers#552: Both changes modify child-failure notifications in finalize_failed.
• iii-hq/workers#577: Both changes affect turn finalization and child wake-up behavior.

Suggested reviewers: andersonleal, sergiofilhowz

Poem

I’m a rabbit guarding each new run,
With owner stamps beneath the sun.
Partial crumbs of metrics stay,
Stuck hops end the watchful day.
Quickstarts softly pass unseen—
And failures find their owning screen.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/harness-reactive-e2e

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

skill-check — worker

0 verified, 49 skipped (no docs/).

Layer	Result
structure	✓
vale	✓
ai	✓
render	✓

Four for four. Nicely done.