Skip to content

Add image refinement flow to our image generation screens#292

Draft
dkotter wants to merge 12 commits intoWordPress:developfrom
dkotter:feature/image-edits
Draft

Add image refinement flow to our image generation screens#292
dkotter wants to merge 12 commits intoWordPress:developfrom
dkotter:feature/image-edits

Conversation

@dkotter
Copy link
Collaborator

@dkotter dkotter commented Mar 10, 2026

Important

Image editing only works with the Google Provider and requires some changes to be released there first. You can manually copy those changes over to test (which is what I've done) but easiest to wait for those changes to be released.

What?

Closes #279

Partially closes #270

Add a "Refine" flow to our existing image generation screens

Why?

It's likely that an image you generate doesn't match exactly what you want the first time. While you can edit the prompt and try again, sometimes the image generated is pretty close and needs just a few refinements. This flow is now supported when generating images, both in the editor and directly in the Media Library.

How?

  • Update our generate-image Ability to support passing in a reference base-64 encoded image. If that is passed, we add that as a file to our prompt
  • In our image generation UI, add a "refine" state that allows you to enter a refinement prompt and send that along with the generated image (this was previously added in Inline image generation #235 but removed)
  • When a refinement comes back, show the original image next to the refined image for easy comparison
  • When importing an image, keep track of each prompt used so we can store that with the image. Previously we stored the original prompt only, now we store that plus any refinement prompts

Use of AI Tools

Pulled my original code from #235 for most of this. Used GPT-5.3 Codex in Cursor to refine and fix a few things with manual review, testing and cleanup done by me

Testing Instructions

  1. Install the AI Provider for Google plugin
  2. Pull this PR down and run npm install && npm run build
  3. Add valid Google credentials
  4. Turn on the Image Generation experiment
  5. Add an image block into a post and click on the Generate Image button
  6. Add a prompt to generate an image
  7. Click the Refine Image button and add another prompt
  8. Choose to keep that image, refine again or generate again (which will send the original image along with the refinement prompt, basically a "try again" approach)
  9. Ensure all those buttons work as expected and the image imports with proper history
  10. Do the same thing with the stand-alone image generation screen at Media > Generate Image

Screenshots

Modal showing how to generate a new image Generated image showing available options, including the new Refine image button Adding a refinement prompt to a generated image Showing the result of the refinement step with the original image next to the refined image Open WordPress Playground Preview

dkotter added 7 commits March 9, 2026 16:10
…n passed, add that as a reference image before making our image generation request
…mage for easier comparison. In that state, ensure refining further refines the refined image instead of the original. And ensure generating another image sends the original image along with the refinement prompt
@dkotter dkotter added this to the 0.5.0 milestone Mar 10, 2026
@dkotter dkotter self-assigned this Mar 10, 2026
@codecov
Copy link

codecov bot commented Mar 10, 2026

Codecov Report

❌ Patch coverage is 82.75862% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.13%. Comparing base (74dcf0e) to head (ab60a51).

Files with missing lines Patch % Lines
includes/Abilities/Image/Generate_Image.php 82.75% 5 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #292      +/-   ##
=============================================
+ Coverage      57.72%   58.13%   +0.40%     
- Complexity       567      572       +5     
=============================================
  Files             36       36              
  Lines           2933     2952      +19     
=============================================
+ Hits            1693     1716      +23     
+ Misses          1240     1236       -4     
Flag Coverage Δ
unit 58.13% <82.75%> (+0.40%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Normalize Media Library image generation to editor flow Refine image generation loading state and fix layout shift in Generate Image modal

1 participant