Refactor and redesign `ImageEditor` component #10635

pngwn · 2025-02-19T21:49:33Z

Fixes:

Misc Fixes/ work:

pngwn · 2025-02-19T21:52:09Z

Anyway you can do two things:

upload and image and edit it with crop/ resize (its a bit rough, few constraints atm)
draw

You cannot do both.

My main question is the interplay between crop and resize.

Should crop refocus on the resized image? It probably should since it is an image crop.
How should we handle the menu UIs? I like the contextual menus but I'm not about the alignment, weird atm.

demo/image_editor should give a half working UI.

pngwn · 2025-02-19T21:54:16Z

The python API won't do anything and I don't expect it to work with gradio right now, just dealing with finalising the UI work.

gradio-pr-bot · 2025-02-19T21:55:46Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
🦄	Changes	detected!	Details

Install Gradio from this PR

pip install https://gradio-pypi-previews.s3.amazonaws.com/32d560f2533cf7136be71e0af2f5891e276862e6/gradio-5.23.3-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@32d560f2533cf7136be71e0af2f5891e276862e6#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-npm-previews.s3.amazonaws.com/32d560f2533cf7136be71e0af2f5891e276862e6/gradio-client-1.14.1.tgz

Use Lite from this PR

<script type="module" src="https://gradio-lite-previews.s3.amazonaws.com/32d560f2533cf7136be71e0af2f5891e276862e6/dist/lite.js""></script>

gradio-pr-bot · 2025-02-19T21:56:00Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`@gradio/atoms`	`minor`
`@gradio/icons`	`minor`
`@gradio/image`	`minor`
`@gradio/imageeditor`	`minor`
`@gradio/multimodaltextbox`	`minor`
`@gradio/upload`	`minor`
`@self/app`	`minor`
`gradio`	`minor`
`website`	`minor`

Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

Refactor and redesign ImageEditor component

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

abidlabs · 2025-02-19T23:48:55Z

@pngwn I wasn't able to do a lot of testing since the image upload and drawing features are the broken state, even separately for me:

When uploadn in image, there's a white gap in the top-left corner
When drawing, eraser icon doesn't work, nor does changing the color of the brush

Should crop refocus on the resized image? It probably should since it is an image crop.

I agree, and just to confirm, cropping can be undo-ed?

How should we handle the menu UIs? I like the contextual menus but I'm not about the alignment, weird atm.

I'm not sure which menu UIs we are referring to? Can you elaborate or share a screenshot?

pngwn · 2025-02-20T11:31:57Z

Ah, that is slightly more broken than intended but I see the issue locally too.

I'll repost with more clarity shortly.

hannahblair · 2025-03-05T19:13:25Z

can the zoom % font colour be the same as the icons? also can it have the | separator on the right?

pngwn · 2025-03-07T09:56:37Z

@hannahblair I will consider your request.

Jk, yeah I'll change that!

abidlabs · 2025-03-07T22:18:59Z

Very very cool @pngwn! Just going through and testing, I noticed a few things:

(1a) In the initial state, the gr.ImageEditor seems to allow both drawing and uploading an image. So for example, when I click on the gr.ImageEditor, I can draw a stroke, and then as soon as I finish that stroke, the upload dialog screen pops open. Drawing should not be enabled in this screen

Screen.Recording.2025-03-07.at.1.56.11.PM.mov

(1b) I've mentioned this before, but I find the gray upload box inside the the ImageEditor component very stuffy. I understand that it needs to be there after you've uploaded an image or if you are in draw mode, but if in the initial screen, imo it should look stylistically identical to the Image component on the right (no internal boxes):

(1c) I also don't think the zoom % (or perhaps the entire zoom toolbar) needs to be shown before an image is uploaded. The 32% was a bit confusing to see
(2a) I'm a little confused by the buttons on the side toolbar. First of all what does this button do?

The alt text says maximize but it seems to have no effect. If it's really maximizing the input image, then I suggest moving it to the zoom toolbar.

(2b) After I upload an image, the sub-tools change from the various ways to input an image, to the cropping/resizing sub-tools. I found that confusing. I think these two sub-tools should belong to a different tool, so that it's possible to go back and change the background image using any of the original sources.

(2c) Imo, after you upload an image, it should automatically change to the draw tool since that's the most common workflow: upload an image -> annotate it.

Actually interestingly, after you upload an image right now, the tool doesn't change, but you have the ability to draw on the uploaded image. However, this drawing is very jerky and buggy. Only when you actually click on the draw tool can you draw reliably, see this:

Screen.Recording.2025-03-07.at.2.18.24.PM.mov

(still testing...)

abidlabs · 2025-03-07T22:24:49Z

(3) nit: initially I was confused when I started drawing because I thought the brush color would be black because of the black dot sub-tool underneath the paint sub-tool. I wonder if we can expose the currently selected mask color in the toolbar?

abidlabs · 2025-03-07T22:26:15Z

(4) also nit: but the apply icon looks a bit out of place compared to the other icons in the toolbar:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        ie = gr.ImageEditor()
        io = gr.Image()
    b = gr.Button()
    ie.apply(lambda x: x["composite"], ie, io)

demo.launch()

But I should note that all the events are working great in my tests!

abidlabs · 2025-03-07T22:28:38Z

Love the erasing UX!

Screen.Recording.2025-03-07.at.2.28.23.PM.mov

abidlabs · 2025-03-07T22:41:34Z

(5) The cropping and resizing don't apply to the annotations. Is that expected? I would have expected everything to be cropped/resized:

Screen.Recording.2025-03-07.at.2.39.44.PM.mov

(you'll also notice a flaky behavior the first time the cropping tool was used). The code here is just:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        ie = gr.ImageEditor()
        io = gr.Image()
    b = gr.Button()
    b.click(lambda x: x["composite"], ie, io)

demo.launch()

abidlabs · 2025-03-07T22:43:08Z

The zooming works great, exactly as expected

freddyaboulton · 2025-03-10T22:47:11Z

Heroic work @pngwn ! Works great. Noticed some small issues. Apologies if they are repeats/made obsolete by @hannahblair 's upcoming UI enhancements

The layers ui is not compatible with dark mode

The blank canvas text is not visible in dark mode (and it looks off-center). This is image_editor_events demo

After I change colors, I can't immediately drag it across the canvas to draw a line.
A bit niche but the canvas and upload text don't react to resizing

Can't programmatically set the background or layers via an event. See the image_editor_layers demo.
The resizing works well but the UI is not intuitive to me. I don't understand what that grid means. Maybe the default is scale is checked and if unchecked the grid appears? I think most people are interested in resizing while preserving the aspect ratio.

If you can't pan, then you end up drawing. It was confusing behavior for me.

The webcam and paste buttons just open the file upload
+1 to adding the undo action buttons back. I think they will be sorely missed. Ok to release without.

js/upload/src/Upload.svelte

### Description This pull request updates the `get_node_path` function in the `gradio/utils.py` file to handle scenarios where the `which` command is missing. The function has been improved by splitting the error handling into two separate `try` blocks for better granularity and robustness. ### Changes Made - Split the `try` block into two separate `try` blocks for handling Windows and Unix-like systems. - Added error handling for `FileNotFoundError` in addition to `subprocess.CalledProcessError`. ### Rationale The original implementation did not properly handle the case when the `which` command is missing. This update ensures the function gracefully handles such scenarios and continues to check other possible locations for the `node` executable.

* fix: latex rendering of markdown * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: Dawood Khan <dawoodkhan82@gmail.com> Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

abidlabs · 2025-04-07T22:03:21Z

js/upload/src/Upload.svelte

@@ -58,7 +62,7 @@
 		}
 		return "." + type;
 	};
-
+	console.log({ filetype });


stray consoles like this

abidlabs · 2025-04-07T22:32:49Z

@pngwn I pushed some changes to fix the Python formatting and tests.

I spent quite a bit of time trying to understand why the video e2e test is failing. Unfortunately I couldn't quite figure it out. It seems like the test chromium browser that is used by playwright doesn't handle video uploads properly. You can check this by running the demo/video_component_events/run.py demo and then using

npx playwright codegen http://127.0.0.1:7860

to manually upload a video. In my experience, the video doesn't get uploaded correctly so we don't get the play/pause video controls for playwright to "click on" and trigger the play/pause events. I can't figure out what's different about this browser that prevents the videos from being uploaded and the part I really don't understand is why this test passes on main.

If you can't figure it out by tomorrow, I think it's okay to remove these tests since I've tested the Video and other file-based components on a regular browser and I can confirm they do work correctly both locally and on Spaces.

pngwn · 2025-04-08T16:35:48Z

Okay @abidlabs I have fixed both the obscured text and the issues wth certain image not showing crop handles (many images did it, initialisation issue).

When CI passes, I shall murj.

…gradio into image-editor-rewrite

pngwn · 2025-04-08T21:21:22Z

Only took 5 hours.

thanks o3

7c3e4a6

add changeset

318b73b

pngwn added 3 commits March 7, 2025 15:04

more

618c514

make events work

8e09825

moreeeeeee

954f090

abidlabs marked this pull request as ready for review March 7, 2025 21:47

abidlabs changed the title ~~thanks o3~~ Refactor and redesign ImageEditor component Mar 7, 2025

hannahblair and others added 4 commits March 11, 2025 01:51

prettier

383821e

nb

73f428f

Merge branch 'main' into image-editor-rewrite

a0010de

add changeset

2c71b78

hannahblair reviewed Mar 11, 2025

View reviewed changes

js/upload/src/Upload.svelte Outdated Show resolved Hide resolved

pablospe and others added 3 commits March 11, 2025 09:55

fix: latex rendering of markdown (#10765)

8380fa8

* fix: latex rendering of markdown * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: Dawood Khan <dawoodkhan82@gmail.com> Co-authored-by: Abubakar Abid <abubakar@huggingface.co>

more

5b1d78f

abidlabs added 3 commits April 7, 2025 14:13

fix pytest

99a388e

changes

ef2d791

push

2919ce1

abidlabs reviewed Apr 7, 2025

View reviewed changes

js/upload/src/Upload.svelte Outdated

@@ -58,7 +62,7 @@

}

return "." + type;

};

console.log({ filetype });

Copy link

Member

abidlabs Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stray consoles like this

pngwn added 6 commits April 8, 2025 00:26

fiiiiiix

30be635

tweaks

c8e6706

Merge branch 'main' into image-editor-rewrite

4abca24

fix

2a5819e

fix thingy

38ec4a3

tweaks

428fedf

pngwn and others added 12 commits April 8, 2025 17:37

changeset

f00da01

add changeset

983acf4

Merge branch 'main' into image-editor-rewrite

9bf66ad

fix

a783787

Merge branch 'main' into image-editor-rewrite

5fc062a

fix

527324e

website build issue

3c6af02

add changeset

77b3a92

add layeroptions to docs

ead8112

Merge branch 'image-editor-rewrite' of https://github.com/gradio-app/…

0ba287f

…gradio into image-editor-rewrite

formatting

06172c8

fix

32d560f

pngwn merged commit 2f68c9d into main Apr 8, 2025
22 of 23 checks passed

pngwn deleted the image-editor-rewrite branch April 8, 2025 21:02

gradio-pr-bot mentioned this pull request Apr 8, 2025

chore: update versions #10931

Merged

This was referenced Apr 10, 2025

[ImageEditor] - Allow users to set the mask opacity #5644

Closed

[ImageMask] - Brush missing #10236

Closed

Refactor and redesign ImageEditor component #10635

Refactor and redesign ImageEditor component #10635

Uh oh!

Conversation

pngwn commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pngwn commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pngwn commented Feb 19, 2025

Uh oh!

gradio-pr-bot commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🪼 branch checks and previews

Uh oh!

gradio-pr-bot commented Feb 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦄 change detected

This Pull Request includes changes to the following packages.

With the following changelog entry.

Something isn't right?

Uh oh!

abidlabs commented Feb 19, 2025

Uh oh!

pngwn commented Feb 20, 2025

Uh oh!

hannahblair commented Mar 5, 2025

Uh oh!

pngwn commented Mar 7, 2025

Uh oh!

abidlabs commented Mar 7, 2025

Uh oh!

abidlabs commented Mar 7, 2025

Uh oh!

abidlabs commented Mar 7, 2025

Uh oh!

abidlabs commented Mar 7, 2025

Uh oh!

abidlabs commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

abidlabs commented Mar 7, 2025

Uh oh!

freddyaboulton commented Mar 10, 2025

Uh oh!

Uh oh!

abidlabs Apr 7, 2025

Choose a reason for hiding this comment

Uh oh!

abidlabs commented Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pngwn commented Apr 8, 2025

Uh oh!

Uh oh!

pngwn commented Apr 8, 2025

Uh oh!

Uh oh!

Refactor and redesign `ImageEditor` component #10635

Refactor and redesign `ImageEditor` component #10635

pngwn commented Feb 19, 2025 •

edited

Loading

pngwn commented Feb 19, 2025 •

edited

Loading

gradio-pr-bot commented Feb 19, 2025 •

edited

Loading

gradio-pr-bot commented Feb 19, 2025 •

edited

Loading

abidlabs commented Mar 7, 2025 •

edited

Loading

abidlabs commented Apr 7, 2025 •

edited

Loading