Skip to content

Conversation

@felixarntz
Copy link
Member

Fixes #117

This finally resolves a long-standing issue where the default model chosen for a provider would be sort of "random", or rather the first model returned by the provider's API listings.

The solution is to implement model sorting per provider via reasonable heuristics, without getting too opinionated:

  • prefer flagship models (and then the common "lower/cheaper tier" variant, if applicable)
  • prefer later versions
  • prefer non-preview/non-experimental

Keep in mind that the sorting is per provider. There is no issue with this in terms of favoring any specific provider, the sorting is just for their models internally.

For easy testing of the sorted model IDs per provider, do this:

  • Temporarily add a var_dump($modelId); for each $modelId iterated over in PromptBuilder::generateMapFromCandidates (e.g. line 1216).
  • Use the built-in CLI tool, without specifying a model or model preference, but specifying --providerId (with each "anthropic", "google", "openai").

@felixarntz felixarntz added this to the 0.2.1 milestone Nov 17, 2025
@felixarntz felixarntz added the [Type] Bug An existing feature does not function as intended label Nov 17, 2025
@github-actions
Copy link

github-actions bot commented Nov 17, 2025

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: felixarntz <flixos90@git.wordpress.org>
Co-authored-by: JasonTheAdams <jason_the_adams@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@felixarntz felixarntz changed the title Sort models per provider based on reasonable heuristics to prefer flagship models and recent versions Sort models per provider based on reasonable heuristics to avoid random default model Nov 17, 2025
Copy link
Member

@JasonTheAdams JasonTheAdams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wow! I was literally thinking through doing something like this as I was getting my day going this morning! Thanks for busting this out!

Just a couple comments/questions for the moment.

Copy link
Member

@JasonTheAdams JasonTheAdams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! I think it will really improve the default behavior for these Providers!

@felixarntz felixarntz merged commit e9e335b into trunk Nov 17, 2025
5 checks passed
@felixarntz felixarntz deleted the fix/useless-default-models branch November 17, 2025 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Bug An existing feature does not function as intended

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Account for OpenAI's search models

3 participants