Skip to content

Add developer settings mode with the ability to set desired provider and model per feature#486

Merged
dkotter merged 28 commits into
WordPress:developfrom
dkotter:feature/developer-mode
May 7, 2026
Merged

Add developer settings mode with the ability to set desired provider and model per feature#486
dkotter merged 28 commits into
WordPress:developfrom
dkotter:feature/developer-mode

Conversation

@dkotter
Copy link
Copy Markdown
Collaborator

@dkotter dkotter commented Apr 29, 2026

What?

Add a new "developer mode" option on the settings page that when toggled on, shows a new settings section on each feature that allows you to choose the desired provider and model.

Why?

While the average user may not care about the actual model being used for a feature (and likely won't know the difference between the dozens of model options) there are certainly users and situations where setting the desired provider (and optionally model) is desired and beneficial.

Since we don't want to overwhelm users with settings they don't care about, we're hiding these new options behind a "developer mode" that can be toggled on and off.

How?

  • Add a new dropdown menu on our settings page that when clicked, shows a Developer Tools menu with the ability to turn on the selection of provider and model
  • When in this mode, output a new developer settings section for each feature. This section only shows if a feature is toggled on
  • Within this new settings section, add a dropdown that allows you to choose one of your connected providers
  • When a provider is selected, a new dropdown shows that allows you to choose a model. These models are filtered to the selected provider and filtered to the feature. For example, in the Image Generation feature, you should only see image generation models. This data is delivered by a new REST endpoint as well as a new capability argument added to our feature classes
  • Add new helper functions and methods to set the provider (and model, if set) when a request is made. This overrides the defaults we set

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Sonnet 4.6
Used for: Initial review and plan; executed on that plan and wrote tests; final review and tweaking done by me

Testing Instructions

  1. Pull this PR down and run npm i && npm run build
  2. Go to the settings page: Settings > AI
  3. Ensure you see a new three dot icon in the top toolbar
  4. Click on this. Ensure a dropdown is shown with a Model Selection option. Click this and ensure it gets an active state
  5. Toggle on an experiment or feature
  6. Ensure a dropdown shows to select a provider
  7. Select a provider
  8. Ensure a new dropdown shows to select a model
  9. Select a model from the list
  10. Test the feature/experiment out and ensure it still works. We don't currently have built-in logging in order to easily see what provider and model was used but I do have a dev tool that logs this to the debug log that I'm happy to share if desired

Screenshots or screencast

Developer mode toggle Developer mode menu Provider and Model fields
AI plugin settings screen showing the new developer mode icon AI plugin settings screen showing the new developer menu open AI plugin settings screen showing the new provider and model fields

Changelog Entry

Added - New developer mode that can be toggled on the settings page.
Added - When developer mode is toggled on, show new settings allowing you to set the desired provider and model for a feature.

Open WordPress Playground Preview

dkotter added 14 commits April 28, 2026 13:27
…lt that to text_generation. Override this in alt text and image generation to be vision and image_generation
… mode is on, get all providers and models. When a feature is turned on, output a select input and populate that with providers. When a provider is selected, show a models dropdown and populate that with models for that provider. Ensure these are saved anytime they change
…ion with the other setting fields. Adjust styles a bit
…Add a method that will use this data, if it exists, on the prompt builder
@dkotter dkotter added this to the 1.0.0 milestone Apr 29, 2026
@dkotter dkotter self-assigned this Apr 29, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>
Co-authored-by: itsgajendraSingh <gajendrasingh@git.wordpress.org>
Co-authored-by: raftaar1191 <raftaar1191@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 73.65591% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.14%. Comparing base (4c8cf7f) to head (87bb0a5).

Files with missing lines Patch % Lines
includes/REST/Models_Controller.php 70.93% 25 Missing ⚠️
includes/helpers.php 77.41% 7 Missing ⚠️
includes/Abstracts/Abstract_Ability.php 76.92% 3 Missing ⚠️
includes/Admin/Dashboard/AI_Status_Widget.php 0.00% 3 Missing ⚠️
...bilities/Excerpt_Generation/Excerpt_Generation.php 0.00% 2 Missing ⚠️
includes/Abilities/Image/Alt_Text_Generation.php 0.00% 2 Missing ⚠️
includes/Abilities/Refine_Notes/Refine_Notes.php 0.00% 2 Missing ⚠️
includes/Abilities/Summarization/Summarization.php 0.00% 2 Missing ⚠️
...es/Abilities/Title_Generation/Title_Generation.php 0.00% 2 Missing ⚠️
includes/Abilities/Review_Notes/Review_Notes.php 0.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #486      +/-   ##
=============================================
+ Coverage      68.10%   69.14%   +1.03%     
- Complexity      1059     1066       +7     
=============================================
  Files             65       66       +1     
  Lines           5039     5175     +136     
=============================================
+ Hits            3432     3578     +146     
+ Misses          1607     1597      -10     
Flag Coverage Δ
unit 69.14% <73.65%> (+1.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dkotter dkotter requested a review from jeffpaul April 30, 2026 21:15
@itsgajendraSingh
Copy link
Copy Markdown

Hi @dkotter,

I tried testing this PR using the “Try it in Playground” button, but it doesn’t seem to be working.

When Playground opens, it shows a “Blueprint execution failed” error and isn’t able to download/install the plugin from the CI artifact URL.

Screenshot 2026-05-02 at 7 23 05 PM Screenshot 2026-05-02 at 7 23 27 PM

@raftaar1191
Copy link
Copy Markdown
Contributor

@dkotter PlayGround button is not working

image image

@dkotter
Copy link
Copy Markdown
Collaborator Author

dkotter commented May 3, 2026

@itsgajendraSingh @raftaar1191 The Playground link should work now.

Note the GitHub Action that adds that Playground link is a bit flakey and doesn't seem to always work when a new commit is pushed (seems to fairly often update the referenced build even if that build doesn't exist). I'd recommend for best results to just pull the PR down and test locally.

Copy link
Copy Markdown
Member

@jeffpaul jeffpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not immediately clear when clicking the new gear icon and its color/state changing is doing (even with the "Developer mode" on-hover text). So perhaps we switch to the vertical ellipses like Gutenberg and add one item there to select/check/enable for "Model selection" with description of "Select the specific provider and model per feature"?

Image

@dkotter
Copy link
Copy Markdown
Collaborator Author

dkotter commented May 6, 2026

Its not immediately clear when clicking the new gear icon and its color/state changing is doing (even with the "Developer mode" on-hover text). So perhaps we switch to the vertical ellipses like Gutenberg and add one item there to select/check/enable for "Model selection" with description of "Select the specific provider and model per feature"?

This has been updated now (also updated screenshots in the PR description). I did add this menu item in a DEVELOPER TOOLS group but can remove that group if we don't want it or can rename it if there's a better phrase

@dkotter dkotter requested a review from jeffpaul May 6, 2026 16:16
Copy link
Copy Markdown
Member

@jeffpaul jeffpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model dropdown appears to be missing its down carret and then when a model is selected the X looks misaligned as well:

Image

dkotter added 2 commits May 6, 2026 10:54
…select. Add a class to our reset button so we can target that specifically for style changes to avoid having those styles leak
@dkotter
Copy link
Copy Markdown
Collaborator Author

dkotter commented May 6, 2026

The model dropdown appears to be missing its down carret and then when a model is selected the X looks misaligned as well:

Misalignment was my fault, overaggressive CSS that was impacting that which I've fixed now.

For the missing caret, this is actually expected here. This page is using DataForm fields and in diving into those, when using a select input, if there are more than 10 items in that select, it automatically converts to an adaptiveSelect component that has built-in search functionality.

So what you're seeing here is you can click into the input and start typing and it will show you options that match. But that means it's slightly different than a normal select that just shows you dropdown items (and that will render that caret).

That said, for now I've forced these inputs to always use the normal select component instead of adaptiveSelect so now will always be a normal select input (with the caret) regardless of how many options it has. Can revert that back though if we think the search input is nicer (though I had the same issue, wasn't clear to me it actually is a search input)

Copy link
Copy Markdown
Member

@jeffpaul jeffpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having all of this behind the loading animation, can the Provider label be immediate and the loading animation is the placeholder just for that dropdown?

Image

… hiding everything behind the loading spinner
@dkotter
Copy link
Copy Markdown
Collaborator Author

dkotter commented May 6, 2026

Instead of having all of this behind the loading animation, can the Provider label be immediate and the loading animation is the placeholder just for that dropdown?

Updated now to show the label with the spinner icon below instead of hiding everything with that spinner

jeffpaul
jeffpaul previously approved these changes May 7, 2026
Copy link
Copy Markdown
Member

@jeffpaul jeffpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, happy to see this merge into 0.9.0 today if you're good with that

@dkotter dkotter modified the milestones: 1.0.0, 0.9.0 May 7, 2026
@dkotter dkotter merged commit 5a83a76 into WordPress:develop May 7, 2026
17 checks passed
@github-project-automation github-project-automation Bot moved this from Needs review to Done in WordPress AI Planning & Roadmap May 7, 2026
@dkotter dkotter deleted the feature/developer-mode branch May 7, 2026 16:22
@dkotter dkotter mentioned this pull request May 7, 2026
31 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

4 participants