Refactor and redesign ImageEditor component

[pngwn]

This is a not working draft of the image editor. I had it half working until i let o3 run wild and then i broke everything.

o3 has done more to set this country back than Margaret Thatcher.

[pngwn]

Anyway you can do two things:

• upload and image and edit it with crop/ resize (its a bit rough, few constraints atm)
• draw

You cannot do both.

My main question is the interplay between crop and resize.

Should crop refocus on the resized image? It probably should since it is an image crop.
How should we handle the menu UIs? I like the contextual menus but I'm not about the alignment, weird atm.

demo/image_editor should give a half working UI.

[pngwn]

The python API won't do anything and I don't expect it to work with gradio right now, just dealing with finalising the UI work.

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
🦄	Changes	detected!	Details

---

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/16c2f8d4a041d382b40c2a0001c01bc066ebfb0b/gradio-5.20.1-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@16c2f8d4a041d382b40c2a0001c01bc066ebfb0b#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/16c2f8d4a041d382b40c2a0001c01bc066ebfb0b/gradio-client-1.13.1.tgz

Use Lite from this PR

<script type="module" src="https://gradio-lite-previews.s3.amazonaws.com/16c2f8d4a041d382b40c2a0001c01bc066ebfb0b/dist/lite.js""></script>

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
@gradio/markdown-code	patch
gradio	patch

With the following changelog entry.

fix: latex rendering of markdown

⚠️ The changeset file for this pull request has been modified manually, so the changeset generation bot has been disabled. To go back into automatic mode, delete the changeset file.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[abidlabs]

@pngwn I wasn't able to do a lot of testing since the image upload and drawing features are the broken state, even separately for me:

• When uploadn in image, there's a white gap in the top-left corner
• When drawing, eraser icon doesn't work, nor does changing the color of the brush

Should crop refocus on the resized image? It probably should since it is an image crop.

I agree, and just to confirm, cropping can be undo-ed?

How should we handle the menu UIs? I like the contextual menus but I'm not about the alignment, weird atm.

I'm not sure which menu UIs we are referring to? Can you elaborate or share a screenshot?

[pngwn]

Ah, that is slightly more broken than intended but I see the issue locally too.

I'll repost with more clarity shortly.

[hannahblair]

• can the zoom % font colour be the same as the icons? also can it have the | separator on the right?

[pngwn]

@hannahblair I will consider your request.

Jk, yeah I'll change that!

[abidlabs]

Very very cool @pngwn! Just going through and testing, I noticed a few things:

• (1a) In the initial state, the gr.ImageEditor seems to allow both drawing and uploading an image. So for example, when I click on the gr.ImageEditor, I can draw a stroke, and then as soon as I finish that stroke, the upload dialog screen pops open. Drawing should not be enabled in this screen

Screen.Recording.2025-03-07.at.1.56.11.PM.mov

• (1b) I've mentioned this before, but I find the gray upload box inside the the ImageEditor component very stuffy. I understand that it needs to be there after you've uploaded an image or if you are in draw mode, but if in the initial screen, imo it should look stylistically identical to the Image component on the right (no internal boxes):

• (1c) I also don't think the zoom % (or perhaps the entire zoom toolbar) needs to be shown before an image is uploaded. The 32% was a bit confusing to see

• (2a) I'm a little confused by the buttons on the side toolbar. First of all what does this button do?

The alt text says maximize but it seems to have no effect. If it's really maximizing the input image, then I suggest moving it to the zoom toolbar.

• (2b) After I upload an image, the sub-tools change from the various ways to input an image, to the cropping/resizing sub-tools. I found that confusing. I think these two sub-tools should belong to a different tool, so that it's possible to go back and change the background image using any of the original sources.

• (2c) Imo, after you upload an image, it should automatically change to the draw tool since that's the most common workflow: upload an image -> annotate it.

Actually interestingly, after you upload an image right now, the tool doesn't change, but you have the ability to draw on the uploaded image. However, this drawing is very jerky and buggy. Only when you actually click on the draw tool can you draw reliably, see this:

Screen.Recording.2025-03-07.at.2.18.24.PM.mov

(still testing...)

[abidlabs]

(3) nit: initially I was confused when I started drawing because I thought the brush color would be black because of the black dot sub-tool underneath the paint sub-tool. I wonder if we can expose the currently selected mask color in the toolbar?

[abidlabs]

(4) also nit: but the apply icon looks a bit out of place compared to the other icons in the toolbar:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        ie = gr.ImageEditor()
        io = gr.Image()
    b = gr.Button()
    ie.apply(lambda x: x["composite"], ie, io)

demo.launch()

But I should note that all the events are working great in my tests!

[abidlabs]

Love the erasing UX!

Screen.Recording.2025-03-07.at.2.28.23.PM.mov

[abidlabs]

(5) The cropping and resizing don't apply to the annotations. Is that expected? I would have expected everything to be cropped/resized:

Screen.Recording.2025-03-07.at.2.39.44.PM.mov

(you'll also notice a flaky behavior the first time the cropping tool was used). The code here is just:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        ie = gr.ImageEditor()
        io = gr.Image()
    b = gr.Button()
    b.click(lambda x: x["composite"], ie, io)

demo.launch()

[abidlabs]

The zooming works great, exactly as expected

[freddyaboulton]

Heroic work @pngwn ! Works great. Noticed some small issues. Apologies if they are repeats/made obsolete by @hannahblair 's upcoming UI enhancements

• The layers ui is not compatible with dark mode

• The blank canvas text is not visible in dark mode (and it looks off-center). This is image_editor_events demo

• After I change colors, I can't immediately drag it across the canvas to draw a line.
  

• A bit niche but the canvas and upload text don't react to resizing

• Can't programmatically set the background or layers via an event. See the image_editor_layers demo.

• The resizing works well but the UI is not intuitive to me. I don't understand what that grid means. Maybe the default is scale is checked and if unchecked the grid appears? I think most people are interested in resizing while preserving the aspect ratio.

• If you can't pan, then you end up drawing. It was confusing behavior for me.

• The webcam and paste buttons just open the file upload
  

• +1 to adding the undo action buttons back. I think they will be sorely missed. Ok to release without.

[hannahblair]

@pngwn can you check that i resolved this merge conflict correctly