Harden v1.11.2 runtime and E2E regressions#777
Merged
Y1fe1Zh0u merged 7 commits intoJul 23, 2026
Conversation
The merged regression candidate exposed model lifecycle, group workspace, runtime terminal-state, logging, multimodal, scheduling, and acknowledgement failures that could not be accepted independently. Keep the fixes together as one deployable 3010 verification candidate based on upstream/v1.11.2-test. Constraint: Tool capability probes are diagnostic only; saved enabled tenant-valid models must remain runnable. Rejected: Keep the group ACK beside the live typing indicator | duplicates the same progress signal. Confidence: high Scope-risk: broad Reversibility: clean Directive: Do not restore supports_tool_calling as a Chat, Agent Runtime, Group planning, or context eligibility gate. Tested: backend 2056 tests; frontend 84 tests and production build; scoped Ruff; diff check; Qwen single-image and multi-image E2E on the pre-fix deployment. Not-tested: deployed fix candidate; controlled context overflow; deterministic provider protocol exhaustion; restart-boundary scheduling.
Exclude deleted or inactive group-only sessions from background compact scanning, preserve invalid compact-budget errors as deterministic terminal failures, and align runtime model guidance with the saved-enabled model policy. Constraint: Tool probes are diagnostic and must not gate saved enabled models. Rejected: Retry invalid compact budgets through reconciliation | the same capability error cannot succeed on retry Confidence: high Scope-risk: narrow Directive: Keep background group compaction scoped to live groups with at least one active eligible Agent. Tested: Backend 2057 tests; frontend 84 tests; frontend production build; Ruff and git diff check Not-tested: Live 3010 verification occurs after deployment
Live 3010 verification showed that RuntimeModelStepService computes the request budget while loading compact inputs, before the compactor's model-specific budget helper. Convert that capability error at the outer compactor boundary so the command worker records the original deterministic code instead of exhausting reconciliation retries. Constraint: Direct Chat compaction loads the complete business-request budget before evaluating the compact watermark. Rejected: Catch only in RuntimeModelStepService | the compactor port should normalize capability failures from any input loader implementation Confidence: high Scope-risk: narrow Tested: Backend 2058 tests; focused 66 Runtime compactor/worker tests; Ruff and git diff check Not-tested: Live 3010 retest occurs after deployment
Planning already resolves an active primary, fallback, or tenant-default model before entry scheduling. Reuse that pinned result, and convert deterministic scheduling validation failures into idempotent terminal group deliveries so product reconciliation can settle instead of retrying the same head row forever. Constraint: Planning checkpoints are immutable and child entry writes must remain all-or-nothing. Rejected: Require the resolved model to equal the Agent primary model | contradicts the existing fallback/default resolution contract. Confidence: high Scope-risk: narrow Directive: Retry infrastructure and delivery failures, but terminally project deterministic PlanningSchedulingError outcomes. Tested: 2059 backend tests; 58 related runtime tests; Ruff; git diff --check Not-tested: Live 3010 multi-Agent E2E before deployment
Multi-Agent greetings were entering the full Planning model path, producing invented analysis and public handoffs while the UI could lose its active-run poll between Planning and entry Runs. Recognize only exact greeting/check-in intents inside Planning, preserve the animation across the scheduling handoff, and reject self-targeted public handoffs before terminal delivery. Constraint: Complex requests that merely begin with a greeting must continue through the Planning model. Rejected: Bypass the Planning Run at message intake | would fork the multi-Agent dispatch lifecycle and its observability contract. Confidence: high Scope-risk: moderate Directive: Keep the fast path exact-match and planner-owned; do not broaden it to messages containing task content. Tested: 2064 backend tests; 85 frontend tests; Ruff; TypeScript and Vite production build Not-tested: Live 3010 multi-Agent greeting until deployment
A planned sequential group task could still be routed through private A2A when a model ignored the existing public-handoff guidance. Strengthen both Planning and Group Runtime instructions so every assigned Agent owns a public reply, plan transitions name the next public wake, and private delegated results cannot be substituted or attributed as group completion. Constraint: Preserve the existing group collaboration strategy and tool availability; this is a prompt-only correction. Rejected: Remove send_message_to_agent from planned Group Runs | broader runtime policy change was explicitly deferred. Confidence: medium Scope-risk: narrow Directive: Keep the prompt and runtime hard-gate questions separate; add enforcement only if prompt-only E2E still routes through private A2A. Tested: 2064 backend tests; targeted Planning and Model Step tests; Ruff; git diff --check Not-tested: Live 3010 sequential public-handoff E2E before deployment
Planning now normalizes vague multi-Agent requests into explicit Agent, action, output, and dependency directives before it schedules any work. Mention-local instructions remain bound to their named Agent, and repair attempts must re-check semantics against the original request instead of only repairing JSON. Constraint: Keep this fix prompt-only; do not alter group collaboration scheduling semantics. Rejected: Add deterministic parser enforcement | current request is to strengthen the planner prompt first. Confidence: high Scope-risk: narrow Directive: Do not weaken mention-local responsibility binding without a replacement semantic validator. Tested: 2064 backend tests; Ruff on changed files; git diff check. Not-tested: Live Provider behavior on 3010 has not been rerun with the new prompt.
Y1fe1Zh0u
marked this pull request as ready for review
July 23, 2026 13:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR consolidates the fixes found while running the PR #765–#774 regression suite against
v1.11.2-test.Root cause
The merged test branch exposed several contract mismatches between persisted model state, frontend caches, Runtime eligibility, Group delivery, background scanning, and UI status rendering. Planning also relied on soft semantic interpretation without first freezing explicit mention-local responsibility assignments, which allowed roles to be reversed or routed through private A2A.
The fixes keep the existing Runtime and Group collaboration architecture, but align each boundary with the persisted source of truth and make deterministic failures terminal and observable.
User impact
Validation
2064 passed85 passedgit diff --checkpassedd7030db3796b8254cc805f808df9dc62f2ea7a34Not tested in the final product click-through
These paths have automated or earlier E2E evidence in this branch, but were not repeated in the final product-owner click-through.
The intentionally deferred extreme boundaries remain: real giant multi-image context overflow, application-level uncaught HTTP 500 injection, one-success/one-failure Group execution, and announcement updates during an already-running Run.