Skip to content

counterfactual: Workers AI provider for the replay harness (flag-gated) so the CI check can actually run #8279

Description

@JSONbored

Problem

The #8222 CI check is budget-gated and currently needs an ollama URL reachable from GitHub runners — the ORB box's ollama is tailnet-internal, so the check can only ever post its skipped notice. A Workers AI-backed provider (the same account already running production reviews) makes the budget flag genuinely activatable.

Requirements

⚠️ Required pattern. Provider seam stays in the CLI wrapper (the core is provider-agnostic); explicit flag + credentials via CI secrets, never defaults; the #8219 spend contract's neuron budgeting maps onto Workers AI's own accounting; abstention semantics unchanged.

  • --provider workers-ai with account/token/model from env; refused loudly when creds absent.
  • Wire the workflow's provider selection; keep the ollama smoke path the local default.
  • Document the activation recipe (repo vars + secrets) on the backtest-calibration page.

Deliverables

  • Provider + tests; workflow updated; docs recipe
  • One recorded CI-run comparison on a real prompt-touching PR once activated

Links & Resources

#8222 (the check), scripts/counterfactual-replay.ts, the live Workers AI review path (src/services/ai-review.ts)

Boundaries

Flag-gated spend only; advisory posture unchanged.

maintainer-only — CI spend wiring.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions