feat(llm): ctx.llm capability seam (ar-plan PR #3, S1.a) - #155
Merged
Conversation
Provider-neutral LlmCapability interface centralizing per-model/provider capability facts. Three providers (MlxCapabilityProvider wraps pre-resolved MlxModelCapabilities; FirstPartyCapabilityProvider static model-id table; GatewayCapabilityProvider for bedrock/vertex/foundry/openai) + async factory createLlmCapability(modelId). Ctx envelope (ctx.llm) via async createCtx — async because MLX capability resolution probes the local model server. Migrate two consumers with optional ctx path falling back to old provider-if branches for byte-identical behavior (all current callers pass no ctx): - betas.ts modelSupportsStructuredOutputs(model, ctx?) - thinking.ts modelSupportsThinking(model, ctx?) ctx path replaces only the provider-conditional branch; 3P-override and ant-model-resolution pre-checks in thinking.ts stay consumer-side. Guard `ctx.llm.modelId === model` ensures ctx only diverges when ctx is present and matches the queried model. Conservative defaults for unknown model ids (appendix A.4): streaming + tool-calling on, vision + structured off, maxInput 200k, maxOutput 8192. 18 tests: 3-provider values, unknown-id defaults, MLX heuristic reuse (small ≤3B → toolCalling false, qwen3 keyword → structured, vl keyword → vision), factory routing, createCtx envelope. Tests use the real adapter (heuristic path, MLX down in CI) — no mock.module namespace-replace fragility. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 27, 2026
dahai80
added a commit
that referenced
this pull request
Aug 28, 2026
…173) First real prod consumer for the ctx.llm capability seam (ar-plan PR #155/#157 landed it byte-identical-off with 0 prod consumers — 555 LOC dead scaffolding). Per audit 1.1.2 decision "wire 1 real consumer". sideQuery (stateless, async, self-contained) now builds a Ctx and passes it to modelSupportsStructuredOutputs when: - output_format is in play (structured-output beta decision), AND - provider is cloud (firstParty/foundry/bedrock/vertex/openai). fusionMlx stays on the old provider-if path: its capability reads MLX model keywords (qwen3/llama3/...) and would enable the beta for models the old path blanket-denied — a deliberate capability-authoritative divergence deferred until the MLX beta-header behavior is verified. Cloud path is byte-identical (firstParty 6-clause list matches betas.ts; foundry/3P both return false). Fail-open: ctx build error -> undefined ctx -> old path (no new failure mode). Lazy: ctx only built when output_format truthy AND provider non-MLX. Globals getCwd()/getSessionId() (sync) supply cwd/sessionId without prop-drilling (sideQuery has 10 callers, all classifiers, stateless). Test: src/__tests__/utils/sideQueryCtxSeam.test.ts (7 tests, 38 expect): - firstParty 9 model ids with-ctx == no-ctx (byte-identical loop) - firstParty structured models return true - firstParty non-structured returns false - foundry capability-authoritative returns true - bedrock/vertex/openai byte-identical (both false) - fusionMlx gate: sideQuery passes ctx=undefined -> old path false (reference ctx would return true — divergence documented) - seam guard: mismatched ctx.llm.modelId falls back to old path Also fix pre-existing typecheck error in viewedTaskSelector.test.ts (fakes re-typed to narrowed InProcessTeammateTaskState/LocalAgentTaskState — .toBe() overload rejected widened TaskState). Behavior unchanged, 10/10 still pass. Per repo rule: fix failing cases even if unrelated. Gates: typecheck 0 / 721 tests / lint net-zero 4601 / build:dev 3762. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Provider-neutral LlmCapability interface centralizing per-model/provider capability facts (ar-plan §2 P1/S1.a). Eliminates scattered provider-if branches — consumers read
ctx.llm.supportsX().New
src/services/llm/capability.ts—LlmCapabilityinterface (8 methods) + 3 providers + asynccreateLlmCapabilityfactoryMlxCapabilityProvider— wraps pre-resolvedMlxModelCapabilities(asynccreate()probes local MLX server),supportsThinking()= false (MLX names lack sonnet-4/opus-4)FirstPartyCapabilityProvider— static model-id table mirroringgetModelMaxOutputTokens+modelSupports*(),supportsThinking()=!claude-3-, structured viafirstPartySupportsStructuredOutputGatewayCapabilityProvider— bedrock/vertex/foundry/openai;supportsStructuredOutput()= foundry-only,supportsThinking(): foundry!claude-3-elsesonnet-4|opus-4src/services/llm/ctx.ts—Ctxenvelope + asynccreateCtx(modelId, cwd, sessionId). Async because MLX capability resolution is async.Migrated (ctx-optional, byte-identical fallback)
src/utils/betas.tsmodelSupportsStructuredOutputs(model, ctx?)src/utils/thinking.tsmodelSupportsThinking(model, ctx?)ctx path replaces only the provider-conditional branch.
thinking.ts3P-override + ant-model-resolution pre-checks stay consumer-side (not provider-based). Guardctx.llm.modelId === modelensures ctx diverges only when ctx present AND matches queried model. All current callers pass no ctx → 100% old-path fallback.Conservative defaults (appendix A.4)
Unknown model ids: streaming + tool-calling on, vision + structured off, maxInput 200k, maxOutput 8192.
Test plan
bun run typecheck— 0 errorsbun test src/__tests__/services/llm/capability.test.ts— 18 passbun run lint— changed files clean (4 pre-existing warnings untouched)bun run build:dev— pass18 tests: 3-provider values, unknown-id defaults, MLX heuristic reuse (small ≤3B → toolCalling false, qwen3 keyword → structured, vl keyword → vision), factory routing (MLX-name/firstParty/foundry), createCtx envelope + seam-guard parity. Tests use the real adapter (heuristic path, MLX down in CI) — avoids
mock.modulenamespace-replace fragility that breakscreateFusionMlxFetchimporters.Byte-identical guarantee
Default-off seam. No caller passes ctx today → old provider-if paths execute unchanged. ctx path is parallel structure, verified by tests, ready for future PRs to thread
Ctxthrough the query pipeline.Co-Authored-By: Claude Fable 5 noreply@anthropic.com