Skip to content

adds model preference support for mcp server sampling requests#373

Merged
bkrabach merged 1 commit intomicrosoft:mainfrom
bkrabach:sampling-model-selection
Mar 4, 2025
Merged

adds model preference support for mcp server sampling requests#373
bkrabach merged 1 commit intomicrosoft:mainfrom
bkrabach:sampling-model-selection

Conversation

@bkrabach
Copy link
Copy Markdown
Collaborator

@bkrabach bkrabach commented Mar 4, 2025

Example:

from mcp.types import ModelPreferences
from mcp_extensions import send_sampling_request

# Set model preferences
model_preferences = ModelPreferences(
    # Can use hints to prefer models with specific names
    # hints=[
    #     ModelHint(
    #         # Prefer models where name starts with `name` value (so `o3` would _include_ `o3-mini`)')
    #         name="gpt-4o",
    #     )
    # ],
    # Setting speed priority to 1 to choose a faster model, like gpt-4o
    speedPriority=1,
    # If needing a reasoning model, set intelligence priority to 1 instead
    # intelligencePriority=1,
)

sampling_result = await send_sampling_request(
        fastmcp_server_context=ctx,
        system_prompt="<your prompt>",
        messages=messages,
        model_preferences=model_preferences,
        max_tokens=1024,
    )

@bkrabach bkrabach merged commit 31845cd into microsoft:main Mar 4, 2025
@bkrabach bkrabach deleted the sampling-model-selection branch March 4, 2025 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant