Skip to content

Propagate execution profiles through Workflows/Keepalive and return worker-model telemetry #2742

Description

@stranske

Target: stranske/Workflows
Priority: P1
Depends on: issues 2, 3, and 5

Why

Workflows is close to supporting remote model choice but does not wire it through Keepalive. On verified origin/main, .github/workflows/reusable-codex-run.yml:55-64 accepts codex_model, resolves/falls back it at :700-746, runs Codex with --model at :925-979, and emits selected-model details at :990-1000. The Keepalive caller at .github/workflows/agents-keepalive-loop.yml:687-715 does not pass that input. Routing remains agent-label only (docs/keepalive/MULTI_AGENT_ROUTING.md:10-21), and .github/agents/registry.yml has no registry-backed execution profile.

Missing behavior: a local provider/model decision cannot reach the remote coding worker or return causally valid requested/resolved worker identity; evaluator telemetry can currently be mistaken for worker identity.

Scope

Add validated execution-profile propagation to the Workflows registry and Keepalive dispatch, emit operation-role-safe worker telemetry, and synchronize consumer templates. Local ingest and the bounded Sol/Terra/Luna plumbing trial are a separate dependent issue.

Non-Goals

  • Do not copy local learned weights into Workflows or create a second router/controller.
  • Do not accept arbitrary free-form model text from issue labels or PR comments.
  • Do not activate model-weight learning from the initial smoke trial.
  • Do not assume requested model equals resolved model when fallback occurs.
  • Do not implement local Orchestrator ingestion or run the model trial in this Workflows issue.
  • Scaffold-only completion does NOT count: adding a workflow input without registry validation, caller propagation, selected/fallback worker output, and consumer-template synchronization is incomplete for this Workflows issue.

Tasks

  • Add registry-backed allowed execution profile IDs and fallback chains to .github/agents/registry.yml or a companion versioned registry, including exact model, runner, capacity-pool references, safety defaults, and lifecycle state.
  • Add a validated profile input to .github/workflows/agents-keepalive-loop.yml and pass the corresponding exact model into .github/workflows/reusable-codex-run.yml; reject unknown profiles before worker execution.
  • Extend the runner's langsmith-fleet/v1 artifact to emit operation role worker, profile ID, requested model, selected/resolved model, fallback reason, and runner/CLI version separately from evaluator/verifier traces.
  • Update config/model_registry.json and config/llm_slots.json documentation so auxiliary judge models are not confused with coding-worker profiles.
  • Update templates/consumer-repo workflow/registry copies and Workflows freshness/sync tests from the canonical source.

Acceptance Criteria

  • A Workflows keepalive contract test proves a known profile reaches reusable-codex-run.yml and its exact model reaches the Codex command.
  • An unknown profile fails before any coding worker step starts and emits a bounded registry-validation reason.
  • A fallback fixture emits both requested and selected/resolved worker models plus fallback reason as a worker attempt; evaluator traces remain separate.
  • Template freshness tests prove the root workflow, consumer workflow, and registry contract remain synchronized.
  • Deliberate break: in the call currently rooted at .github/workflows/agents-keepalive-loop.yml:687-715, delete only the new execution_profile/codex_model forwarding line; run exact test .github/scripts/__tests__/keepalive-model-profile-contract.test.js::profile reaches codex runner via node --test .github/scripts/__tests__/keepalive-model-profile-contract.test.js --test-name-pattern='profile reaches codex runner'; observe AssertionError: expected selected profile model in reusable runner inputs; revert and show the exact test passes.

Implementation Notes

The dependent local trial issue must preserve CLI/version telemetry: July 9 requested Terra and Luna successfully through the app-bundled CLI after the Homebrew CLI rejected them as too old. This issue only establishes the remote Workflows contract.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions