Some tweaks to ImageEditor (#6538)

* fixes * changelog * add changeset * changes * ie fixes * details * templates * Update CHANGELOG.md * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
gradio-app · Nov 22, 2023 · 1479261 · 1479261
1 parent a424fdb
commit 1479261
Show file tree

Hide file tree

Showing 4 changed files with 79 additions and 16 deletions.
diff --git a/.changeset/real-things-push.md b/.changeset/real-things-push.md
@@ -0,0 +1,5 @@
+---
+"gradio": patch
+---
+
+feat:Some tweaks to `ImageEditor`
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -250,8 +250,8 @@ Gradio 4.0 is a new major version, and includes breaking changes from 3.x. Here'
 * Removes `deprecation.py` -- this was designed for internal usage so unlikely to break gradio apps
 * Moves save to cache methods from component methods to standalone functions in processing_utils
 * Renames `source` param in `gr.Audio` and `gr.Video` to `sources`
-* Removes `show_edit_button` param from `gr.Audio``
-
+* Removes `show_edit_button` param from `gr.Audio`
+* The `tool=` argument in `gr.Image()` has been removed. As of `gradio==4.5.0`, we have a new `gr.ImageEditor` component that takes its place. The `ImageEditor` component is a streamlined component that allows you to do basic manipulation of images. It supports setting a background image (which can be uploaded, pasted, or recorded through a webcam), as well the ability to "edit" the background image by using a brush to create strokes and an eraser to erase strokes in layers on top of the background image. See the **Migrating to Gradio 4.0** section below.
 
 **Other changes related to the `gradio` library**:
 
@@ -271,11 +271,11 @@ Gradio 4.0 is a new major version, and includes breaking changes from 3.x. Here'
 
 * When using the gradio Client libraries in 3.x with any component that returned JSON data (including `gr.Chatbot`, `gr.Label`, and `gr.JSON`), the data would get saved to a file and the filepath would be returned. Similarly, you would have to pass input JSON as a filepath. Now, the JSON data is passed and returned directly, making it easier to work with these components using the clients. 
 
-**Migrating to Gradio 4.0**
+### Migrating to Gradio 4.0
 
 Here are some concrete tips to help migrate to Gradio 4.0:
 
-* **Using `allowed_paths`**
+#### **Using `allowed_paths`**
 
 Since the working directory is now not served by default, if you reference local files within your CSS or in a `gr.HTML` component using the `/file=` route, you will need to explicitly allow access to those files (or their parent directories) using the `allowed_paths` parameter in `launch()`
 
@@ -313,7 +313,7 @@ demo.launch(allowed_paths=["."])
 ```
 
 
-* **Using `concurrency_limit` instead of `concurrency_count`**
+#### **Using `concurrency_limit` instead of `concurrency_count`**
 
 Previously, in Gradio 3.x, there was a single global `concurrency_count` parameter that controlled how many threads could execute tasks from the queue simultaneously. By default `concurrency_count` was 1, which meant that only a single event could be executed at a time (to avoid OOM errors when working with prediction functions that utilized a large amount of memory or GPU usage). You could bypass the queue by setting `queue=False`. 
 
@@ -328,6 +328,63 @@ To summarize migration:
 * For events that take significant resources (like the prediction function of your machine learning model), and you only want `X` executions of this function at a time, you should set `concurrency_limit=X` parameter in the event trigger.(Previously you would set a global `concurrency_count=X`.)
 
 
+**The new `ImageEditor` component**
+
+In Gradio 4.0, the `tool=` argument in `gr.Image()` was removed. It has been replaced, as of Gradio 4.5.0, with a new `gr.ImageEditor()` component. The `ImageEditor` component is a streamlined component that allows you to do basic manipulation of images. It supports setting a background image (which can be uploaded, pasted, or recorded through a webcam), as well the ability to "edit" the background by using a brush to create strokes and an eraser to erase strokes in layers on top of the background image.
+
+The `ImageEditor` component is much more performant and also offers much more flexibility to customize the component, particularly through the new `brush` and `eraser` arguments, which take `Brush` and `Eraser` objects respectively. 
+
+Here are some examples of how you might migrate from `Image(tool=...)` to `gr.ImageEditor()`. 
+
+* To create a sketchpad input that supports writing black strokes on a white background, you might have previously written:
+
+```py
+gr.Image(source="canvas", tools="sketch")
+```
+
+Now, you should write:
+
+```py
+gr.ImageEditor(sources=(), brush=gr.Brush(colors=["#000000"]))
+```
+
+Note: you can supply a list of supported stroke colors in `gr.Brush`, as well as control whether users can choose their own colors by setting the `color_mode` parameter of `gr.Brush` to be either `"fixed"` or `"defaults"`.
+
+* If you want to create a sketchpad where users can draw in any color, simply omit the `brush` parameter. In other words, where previously, you would do:
+
+```py
+gr.Image(source="canvas", tools="color-sketch")
+```
+
+Now, you should write:
+
+```py
+gr.ImageEditor(sources=())
+```
+
+
+* If you want to allow users to choose a background image and then draw on the image, previously, you would do:
+
+```py
+gr.Image(source="upload", tools="color-sketch")
+```
+
+Now, this is the default behavior of the `ImageEditor` component, so you should just write:
+
+```py
+gr.ImageEditor()
+```
+
+Unlike the `Image` component, which passes the input image as a single value into the prediction function, the `ImageEditor` passes a dictionary consisting of three key-value pairs:
+
+* the key `"background"`, whose value is the background image
+* the key `"layers"`, which consists of a list of values, with the strokes in each layer corresponding to one list element.
+* the key `"composite"`, whose value is to the complete image consisting of background image and all of the strokes.
+
+The type of each value can be set by the `type` parameter (`"filepath"`, `"pil"`, or `"numpy"`, with the default being `"numpy"`), just like in the `Image` component.
+
+Please see the documentation of the `gr.ImageEditor` component for more details: https://www.gradio.app/docs/imageeditor
+
 ### Features
 
 - [#6184](https://github.com/gradio-app/gradio/pull/6184) [`86edc0199`](https://github.com/gradio-app/gradio/commit/86edc01995d9f888bac093c44c3d4535fe6483b3) - Remove gr.mix.  Thanks [@abidlabs](https://github.com/abidlabs)!
@@ -4575,4 +4632,4 @@ We've introduced a lot of new components in `3.0`, including `Model3D`, `Dataset
 - [@ronvoluted](https://github.com/ronvoluted) made their first contribution in [PR 1050](https://github.com/gradio-app/gradio/pull/1050)
 - [@radames](https://github.com/radames) made their first contribution in [PR 1074](https://github.com/gradio-app/gradio/pull/1074)
 - [@freddyaboulton](https://github.com/freddyaboulton) made their first contribution in [PR 1085](https://github.com/gradio-app/gradio/pull/1085)
-- [@liteli1987gmail](https://github.com/liteli1987gmail) & [@chenglu](https://github.com/chenglu) made their first contribution in [PR 4767](https://github.com/gradio-app/gradio/pull/4767)
+- [@liteli1987gmail](https://github.com/liteli1987gmail) & [@chenglu](https://github.com/chenglu) made their first contribution in [PR 4767](https://github.com/gradio-app/gradio/pull/4767)
diff --git a/gradio/components/image_editor.py b/gradio/components/image_editor.py
@@ -195,7 +195,6 @@ def __init__(
     def convert_and_format_image(
         self,
         file: FileData | None,
-        crop_size: tuple[int | float, int | float] | str | None = None,
     ) -> np.ndarray | _Image.Image | str | None:
         if file is None:
             return None
@@ -214,8 +213,10 @@ def convert_and_format_image(
         with warnings.catch_warnings():
             warnings.simplefilter("ignore")
             im = im.convert(self.image_mode)
-        if crop_size and not isinstance(crop_size, str):
-            im = image_utils.crop_scale(im, int(crop_size[0]), int(crop_size[1]))
+        if self.crop_size and not isinstance(self.crop_size, str):
+            im = image_utils.crop_scale(
+                im, int(self.crop_size[0]), int(self.crop_size[1])
+            )
         return image_utils.format_image(
             im,
             cast(Literal["numpy", "pil", "filepath"], self.type),
@@ -234,7 +235,7 @@ def preprocess(self, x: EditorData | None) -> EditorValue | None:
         if x is None:
             return x
 
-        bg = self.convert_and_format_image(x.background, crop_size=self.crop_size)
+        bg = self.convert_and_format_image(x.background)
         layers = (
             [self.convert_and_format_image(layer) for layer in x.layers]
             if x.layers

diff --git a/gradio/templates.py b/gradio/templates.py
@@ -46,7 +46,7 @@ def __init__(
 
 class Sketchpad(components.ImageEditor):
     """
-    Sets: image_mode="L", sources=(), crop_size=(28, 28), brush=Brush(colors=["#000000"])
+    Sets: sources=(), brush=Brush(colors=["#000000"], color_mode="fixed")
     """
 
     is_template = True
@@ -59,7 +59,7 @@ def __init__(
         width: int | None = None,
         image_mode: Literal[
             "1", "L", "P", "RGB", "RGBA", "CMYK", "YCbCr", "LAB", "HSV", "I", "F"
-        ] = "L",
+        ] = "RGBA",
         sources: Iterable[Literal["upload", "webcam", "clipboard"]] = (),
         type: Literal["numpy", "pil", "filepath"] = "numpy",
         label: str | None = None,
@@ -77,13 +77,13 @@ def __init__(
         mirror_webcam: bool = True,
         show_share_button: bool | None = None,
         _selectable: bool = False,
-        crop_size: tuple[int | float, int | float] | str | None = (28, 28),
+        crop_size: tuple[int | float, int | float] | str | None = None,
         transforms: Iterable[Literal["crop"]] = ("crop",),
         eraser: Eraser | None = None,
         brush: Brush | None = None,
     ):
         if not brush:
-            brush = Brush(colors=["#000000"])
+            brush = Brush(colors=["#000000"], color_mode="fixed")
         super().__init__(
             value=value,
             height=height,
@@ -182,7 +182,7 @@ def __init__(
 
 class ImageMask(components.ImageEditor):
     """
-    Sets:
+    Sets: brush=Brush(colors=["#000000"], color_mode="fixed")
     """
 
     is_template = True
@@ -223,7 +223,7 @@ def __init__(
         brush: Brush | None = None,
     ):
         if not brush:
-            brush = Brush(colors=["#000000"])
+            brush = Brush(colors=["#000000"], color_mode="fixed")
         super().__init__(
             value=value,
             height=height,