Skip to content

Memory consolidation agent gets MultiAgentV2 spawn tools despite Feature::Collab disable; orphaned child streams unowned thread events to app-server clients #35781

Description

@randalmurphal

Summary

The Phase 2 memory consolidation agent is explicitly locked down with Feature::Collab disabled ("Consolidation runs as an internal worker and must not recursively delegate"), but MultiAgentV2 resolution bypasses that feature check entirely, so the consolidation agent receives the full V2 collab toolset (spawn_agent, wait, list_agents, ...) on every run. When its model actually used spawn_agent, the call raced the consolidation runtime's own teardown: the first spawn failed on a lookup of the (already removed) consolidation thread, and a second attempt created a child thread that was left parentless and streamed thread events at the app-server client with no ownership edge the client could ever resolve.

Deterministic part (verifiable by inspection, present at rust-v0.145.0 and current main)

  1. codex-rs/memories/write/src/phase2.rs (agent::get_config) builds the consolidation agent config:

    // Consolidation runs as an internal worker and must not recursively delegate.
    let _ = agent_config.features.disable(Feature::Collab);

    It does not disable Feature::MultiAgentV2 and does not set agents_enabled = false.

  2. codex-rs/core/src/config/mod.rs:

    pub(crate) fn multi_agent_version_for_model(&self, model_multi_agent_version: Option<MultiAgentVersion>) -> MultiAgentVersion {
        self.multi_agent_version_override()          // MultiAgentV2 feature / agents_enabled only
            .or(model_multi_agent_version)           // <-- model catalog capability wins here
            .unwrap_or_else(|| self.multi_agent_version_from_features())  // Collab consulted only in this V1 fallback
    }

    Feature::Collab is only consulted in the final fallback, which only gates V1.

  3. The consolidation default model is gpt-5.6-terra (DEFAULT_MEMORY_CONSOLIDATION_PREFERRED_MODEL, codex-rs/model-provider/src/provider.rs), and codex-rs/models-manager/models.json marks gpt-5.6-terra as multi_agent_version: v2.

Consequence: resolution short-circuits at the model's V2 capability before the Collab disable is ever consulted, so the consolidation agent gets V2 spawn tools despite the stated invariant. This is the same lockdown-leak shape as #30615 (sandbox mode) and #32429 (service tier), with a worse consequence.

Observed incident (once, codex-cli 0.145.0, app-server client, WSL2 Ubuntu 24.04, memories = true)

Root interactive thread 019fa8ef-5b0e-7fe2-81f4-f0aaaaf76b75 started 13:34:46Z; its rollout shows no collab tool calls in the window below. An ephemeral internal thread 019fa8ef-9159-… (no rollout file; UUIDv7 timestamp ≈ 13:35:00Z, ~14s after session start) appeared in the same app-server process, then:

13:36:03.296Z ERROR codex_core::tools::router: collab spawn failed: no thread with id: 019fa8ef-9159-7940-98ce-7ce762baec71
13:36:05.7Z   (child thread 019fa8f0-90fc-7611-b49a-a4b06761c9a2 created — UUIDv7 timestamp; no rollout file;
               streamed thread item notifications to the app-server client; no ownership item ever arrived)
13:36:37.749Z ERROR codex_core::tools::router: agent with id 019fa8ef-9159-7940-98ce-7ce762baec71 not found

Both ERROR lines are tool-router errors responded to a model, i.e. a hidden internal agent was making collab tool calls. The no thread with id <its own id> failure indicates the caller's thread had already been removed from the thread manager (consistent with shutdown_consolidation_agentremove_thread racing the still-executing turn). The second spawn attempt got far enough to create child 019fa8f0-…, which then had no registered parent: the client received its thread events but no spawn-ownership item can ever arrive for it, so a correct fail-closed client must quarantine and drop them.

Attribution note: the identification of 019fa8ef-9159 as the consolidation agent is by elimination, not direct observation — it is ephemeral, has no rollout, was created in-process at memory-startup-job time, and memories = true was enabled; review/compact threads persist rollouts and ran outside this window, guardian was inactive (approval policy never), and no client- or user-initiated thread matches. Everything else above is observed or verifiable from source.

Suggested fix

agent::get_config should pin multi-agent off in a way V2 resolution respects — e.g. disable Feature::MultiAgentV2 and/or set agents_enabled = false (making multi_agent_version_override() return Disabled) — instead of relying on Feature::Collab, which the model-capability path bypasses. Separately, the teardown race (a removed thread's turn continuing to execute collab tool calls that partially mutate state — here creating a parentless child) may warrant its own guard, since any late tool call from a torn-down internal agent can leak protocol-visible state at clients.

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLIIssues related to the Codex CLIapp-serverIssues involving app server protocol or interfacesbugSomething isn't workingmemorysubagentIssues involving subagents or multi-agent features

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions