feat(sdk,sdkx): pod-runtime L3 — llm redesign, tool middleware, script caps, catalog refresh [skip-tag:sdkx]#52
Merged
Conversation
…t caps, catalog refresh [skip-tag:sdkx]
Major rework grouped under the "Pod for Agents" pod-runtime gaps L3
plan (see docs/sdk-pod-runtime-gaps.md and docs/sdk-llm-redesign.md):
sdk/llm — redesigned around ModelSpec
* Introduce ModelSpec (Caps + Defaults + Limits) as the unified
model-fixed property set; ModelInfo.Caps kept as deprecated
alias with a warn-once auto-promote path (removal: v0.3.0).
* Three middleware layers, assembled one-shot by the resolver:
WithDefaults (fill nil) → WithCaps (drop / downgrade /
validate unsupported features) → WithLimits (clamp numeric
ceilings). Replaces the previous ad-hoc capsLLM wrapper.
* Capability surface extended: TopP, TopK, Vision, Audio, File,
ParallelTools, SystemPrompt, Streaming, JSONMode, JSONSchema.
System-prompt folding and streaming downgrade behaviors
happen automatically when the corresponding cap is disabled.
* Credential profiles (sdk/llm/profile.go): per-call profile
via context, profile-aware resolver cache, three-dimensional
cacheKey (provider, model, profile). InvalidateCache now
takes variadic WithProvider/WithModel/WithProfile options
with AND semantics.
* ModelDeprecation field on ModelInfo plus a one-shot resolver
telemetry warning per (provider, model). Catalog metadata
only — does NOT participate in SpecOverride merge so a
deployment cannot accidentally un-deprecate an upstream
declaration.
* File layout cleanup: caps/middleware split into capability.go,
spec.go, with_caps.go, with_defaults.go, with_limits.go;
types.go renamed to aliases.go; deprecated APIs collected in
deprecated.go with explicit v0.3.0 removal banners.
sdk/tool — registry middleware
* Dispatch/Middleware types and Registry.Use for cross-cutting
tool concerns (rate-limit, audit, allow-list). Composed
chain runs around coreDispatch.
* ToolMeta / ToolMetadata interface for static tool metadata.
sdk/script — runtime resource caps
* jsrt: WithMaxCallStackSize and WithMaxExecTime options.
* luart: WithMaxExecTime option. Memory limit removed —
gopher-lua does not expose a safe enforcement hook (see
luart.go comment for rationale).
sdkx/llm — catalog refresh against 2026-04-30 provider docs
* Anthropic: claude-opus-4-7 (new flagship), opus-4-6, sonnet-4-6,
haiku-4-5 with corrected family-first ID convention. JSONMode
enabled (Beta API path), JSONSchema disabled across the family.
* OpenAI: gpt-5.5 / gpt-5.5-pro (released 2026-04-23), gpt-5.4,
gpt-5.4-pro, gpt-5.4-mini, gpt-5.4-nano, gpt-5 family, plus
o3 / o3-pro / o4-mini reasoning models with sampling-control
caps disabled per o-series convention. o4-mini marked
Deprecation{RetiresAt: 2026-10-23, Replacement: gpt-5-mini}
per the 2026-04-22 OpenAI deprecation announcement. gpt-4.1
removed from catalog. defaultModel changed gpt-4o → gpt-5
(gpt-4o lineage is legacy / declining usage).
* MiniMax: M2.7, M2.7-highspeed, M2.5, M2.5-highspeed, M2.1,
M2 (legacy). All caps reflect the Anthropic-compat endpoint
surface — JSONMode/JSONSchema both disabled with documented
evidence (no response_format honoring on M2-series).
* ByteDance Doubao Seed 2.0, Qwen-Max/Plus/Turbo, DeepSeek V4
(flash + pro): all populated with current limits and caps.
[skip-tag:sdkx] is set so the auto-tag workflow only bumps sdk on
this push; sdkx will be tagged manually after a follow-up review of
catalog accuracy.
Made-with: Cursor
…oundary
Before this change, disabling CapParallelTools in a model catalog
was a silent no-op. The middleware comment claimed adapters MUST
consult `caps.Supports(CapParallelTools)` before forwarding the
parallel-tool toggle, but adapters sit BELOW WithCaps in the wrap
chain and only see GenerateOptions — they cannot read caps. As a
result, the DeepSeek catalog declaration that listed
CapParallelTools in DisabledCaps had zero effect at runtime.
Fix it by stripping the protocol-reserved Extra keys directly inside
WithCaps when the cap is disabled:
- "parallel_tool_calls" — OpenAI Chat Completions / Responses
(and OpenAI-compatible: Qwen,
ByteDance, MiniMax, DeepSeek)
- "disable_parallel_tool_use" — Anthropic Messages API
(and Anthropic-compatible: MiniMax
/anthropic endpoint)
Both keys are protocol-reserved namespaces, not free-form labels —
the previous "stripping is brittle" worry doesn't apply. Other
Extra map keys are preserved unmodified.
For new adapters introducing a non-standard parallel-tool toggle,
RegisterParallelToolExtraKey lets them extend the strip set from
their init() so the catalog cap stays a single source of truth.
Tests cover three contracts:
* Disabled cap strips both built-in keys and warns once.
* Enabled cap preserves the keys verbatim.
* RegisterParallelToolExtraKey picks up custom keys at runtime.
Made-with: Cursor
The Volcengine Ark SDK exposes parallel_tool_calls as the typed field model.CreateChatCompletionRequest.ParallelToolCalls (*bool). Our adapter previously did not consume opts.Extra at all, so callers had no way to control parallel tool calls on Doubao through this SDK — even though the underlying API supports it. Bridge opts.Extra["parallel_tool_calls"] (bool) onto the typed request field. This brings ByteDance to feature parity with the OpenAI adapter, and dovetails with the new CapParallelTools enforcement: when a future Doubao SKU drops parallel-tool support, declaring CapParallelTools disabled in the catalog is enough — the WithCaps middleware strips the Extra key before we get here, so this branch falls through and req.ParallelToolCalls stays nil, and Doubao serves the request with its own default behaviour. Made-with: Cursor
…ed field Anthropic carries disable_parallel_tool_use NESTED inside tool_choice (param.Opt[bool] on Auto / Any / Tool — not on None, where it would be meaningless), unlike OpenAI's top-level parallel_tool_calls. The adapter previously did not consume opts.Extra at all on this path, so callers had no way to disable parallel tool use on Anthropic through this SDK. Bridge opts.Extra["disable_parallel_tool_use"] (bool) onto whichever ToolChoice variant is selected. When the caller sets the toggle without an explicit ToolChoice, we default to OfAuto since omitted tool_choice = auto on Anthropic anyway. This also benefits the MiniMax adapter, which proxies through this exact code path via sdkx/llm/minimax/New() → anthropic.New(). The new CapParallelTools enforcement in WithCaps strips the Extra key before this branch runs when the catalog declares the cap disabled, keeping behaviour consistent with the OpenAI / ByteDance adapters. Qwen needed no change: it wraps the OpenAI adapter, so the parallel_tool_calls Extra key already passes through DashScope's OpenAI-compatible endpoint via option.WithJSONSet. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Major rework grouped under the Pod for Agents pod-runtime gaps L3 plan (see
docs/sdk-pod-runtime-gaps.mdanddocs/sdk-llm-redesign.md). Touches three SDK areas plus a fullsdkx/llmprovider catalog refresh.sdk/llm— redesigned aroundModelSpecModelSpec(Caps + Defaults + Limits) is now the unified model-fixed property set. OldModelInfo.Capskept as deprecated alias with warn-once auto-promote (removal: v0.3.0).WithDefaults(fill nil) →WithCaps(drop / downgrade / validate unsupported features) →WithLimits(clamp numeric ceilings). Replaces the previous ad-hoccapsLLMwrapper.TopP,TopK,Vision,Audio,File,ParallelTools,SystemPrompt,Streaming,JSONMode,JSONSchema. System-prompt folding and streaming downgrade kick in automatically when caps are disabled.profile.go): per-call profile via context, profile-aware resolver cache, three-dimensionalcacheKey {provider, model, profile}.InvalidateCachenow takes variadicWithProvider/WithModel/WithProfileoptions with AND semantics.ModelDeprecationfield onModelInfoplus a one-shot resolver telemetry warning per(provider, model). Catalog metadata only — does NOT participate inSpecOverridemerge so a deployment cannot accidentally un-deprecate an upstream declaration.capability.go,spec.go,with_caps.go,with_defaults.go,with_limits.go;types.go→aliases.go; deprecated APIs collected indeprecated.gowith explicit v0.3.0 removal banners.sdk/tool— registry middlewareDispatch/Middlewaretypes andRegistry.Usefor cross-cutting tool concerns (rate-limit, audit, allow-list). Composed chain runs aroundcoreDispatch.ToolMeta/ToolMetadatainterface for static tool metadata.sdk/script— runtime resource capsjsrt:WithMaxCallStackSizeandWithMaxExecTimeoptions.luart:WithMaxExecTimeoption. Memory limit removed —gopher-luadoes not expose a safe enforcement hook (seeluart.gocomment for rationale).sdkx/llm— catalog refresh against 2026-04-30 provider docsclaude-opus-4-7(new flagship), opus-4-6, sonnet-4-6, haiku-4-5 with corrected family-first ID convention. JSONMode enabled (Beta API path), JSONSchema disabled across the family.gpt-5.5/gpt-5.5-pro(released 2026-04-23), gpt-5.4 family, gpt-5 family, pluso3/o3-pro/o4-minireasoning models with sampling-control caps disabled per o-series convention.o4-minimarked deprecated (RetiresAt: 2026-10-23,Replacement: gpt-5-mini) per the 2026-04-22 OpenAI deprecation announcement.gpt-4.1removed from catalog.defaultModelchangedgpt-4o→gpt-5(gpt-4o lineage is legacy / declining usage).response_formathonoring on M2-series).Tag note
[skip-tag:sdkx]is set in the title (and commit subject) so the auto-tag workflow only bumpssdk/on this push.sdkx/will be tagged manually after a follow-up review of catalog accuracy.Test plan
make vet(sdk + sdkx + voice + GOWORK=off bench/examples/conformance)make test(full)sdk/llm,sdk/tool,sdk/script,sdkx/llm/...Made with Cursor