-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Support temperature/sampling parameters in plugin SKILL.md metadata #3056
Copy link
Copy link
Open
Labels
area:configurationConfig files, instruction files, settings, and environment variablesConfig files, instruction files, settings, and environment variablesarea:modelsModel selection, availability, switching, rate limits, and model-specific behaviorModel selection, availability, switching, rate limits, and model-specific behaviorarea:pluginsPlugin system, marketplace, hooks, skills, extensions, and custom agentsPlugin system, marketplace, hooks, skills, extensions, and custom agents
Metadata
Metadata
Assignees
Labels
area:configurationConfig files, instruction files, settings, and environment variablesConfig files, instruction files, settings, and environment variablesarea:modelsModel selection, availability, switching, rate limits, and model-specific behaviorModel selection, availability, switching, rate limits, and model-specific behaviorarea:pluginsPlugin system, marketplace, hooks, skills, extensions, and custom agentsPlugin system, marketplace, hooks, skills, extensions, and custom agents
Type
Fields
Give feedbackNo fields configured for Feature.
Describe the feature or problem you'd like to solve
Entire teams writing official tech docs but can't change the temperature or top-p
Proposed solution
Plugin authors building documentation-authoring skills need low-temperature inference to minimize fabrication and hallucination. Currently, there is no way for a plugin or skill to specify
temperature,top_p, or other sampling parameters -- not inplugin.json,agency.json, or SKILL.md frontmatter metadata.The only workaround is embedding prompt-level instructions like "behave as if temperature is 0.0-0.2," which is unreliable because the model may not honor behavioral temperature constraints the same way it honors API-level parameters.
Allow SKILL.md frontmatter
metadatato include sampling parameters that Copilot CLI passes through to the inference API:Or alternatively, support this at the plugin level in
plugin.json:{ "name": "author-pro", "modelParameters": { "temperature": 0.1, "top_p": 0.9 } }Either approach would let plugin authors tune inference behavior for their use case -- low temperature for factual documentation, higher temperature for creative brainstorming, etc.
Prompt-level behavioral constraints -- "Respond as if temperature is set to 0.1." Works partially but is not equivalent to API-level control. Models still exhibit higher variance than a true low-temperature API call.
MCP server with sampling -- Build an MCP server that uses the
sampling/createMessageprotocol method with temperature parameters. This is architecturally heavier and adds latency, but could work if sampling parameters are passed through to the inference backend.Direct API calls via MCP -- Build an MCP server that calls Azure OpenAI directly with explicit temperature. This works but requires separate API credentials, deployment management, and cost -- defeating the purpose of an integrated plugin system.
Example prompts or workflows
tasktool already supports amodelparameter for sub-agents. Extending this pattern to includetemperaturewould be consistent.sampling.requested/sampling.completed) already exist in the session events schema, suggesting the infrastructure may partially exist.Additional context