Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in #7680

dfl · 2024-03-12T19:22:58Z

Description

preserve file format during upload, default to PNG only when no other format is available.
adds format parameter to Image, AnnotatedImage, and Plot components.

Closes: #7486

Tests

I'm not able to get the test suite working properly on my mac. Having problems with dependencies.

gradio-pr-bot · 2024-03-12T19:23:42Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
🦄	Changes	detected!	Details

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/41636f7ca124b44a596d9fa67e10c26f42982823/gradio-4.21.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@41636f7ca124b44a596d9fa67e10c26f42982823#subdirectory=client/python"

freddyaboulton

Thanks @dfl! Overall looks great. Left some comments that we should address. And can you please add unit tests? Thanks

freddyaboulton · 2024-03-12T23:26:57Z

gradio/processing_utils.py

@@ -78,23 +84,31 @@ def get_pil_metadata(pil_image):

 def encode_pil_to_bytes(pil_image, format="png"):
    with BytesIO() as output_bytes:
-        pil_image.save(output_bytes, format, pnginfo=get_pil_metadata(pil_image))
+        if format == "png":
+            params = {"pnginfo": get_pil_metadata(pil_image)}


I think we can close #7657 then

freddyaboulton · 2024-03-12T23:28:38Z

gradio/image_utils.py

@@ -51,13 +51,17 @@ def format_image(
        )


-def save_image(y: np.ndarray | PIL.Image.Image | str | Path, cache_dir: str):
+def save_image(


The type hint for format is wrong (it can't be None)

freddyaboulton · 2024-03-12T23:40:55Z

gradio/components/annotated_image.py

            )
            base_img_path = str(utils.abspath(base_file))
        elif isinstance(base_img, PIL.Image.Image):
            base_file = processing_utils.save_pil_to_cache(
-                base_img, cache_dir=self.GRADIO_CACHE
+                base_img, cache_dir=self.GRADIO_CACHE, format=self.format


We need to update the type hint of format in save_pil_to_cache to allow for an arbitrary string

freddyaboulton · 2024-03-12T23:45:15Z

gradio/image_utils.py

@@ -51,13 +51,17 @@ def format_image(
        )


-def save_image(y: np.ndarray | PIL.Image.Image | str | Path, cache_dir: str):


Right now format_image will give precedence for the current format of the image instead of the format argument. It should be the other way around.

gleb-akhmerov · 2024-03-17T06:43:35Z

gradio/components/image.py

@@ -199,7 +202,7 @@ def postprocess(
            return None
        if isinstance(value, str) and value.lower().endswith(".svg"):
            return FileData(path=value, orig_name=Path(value).name)
-        saved = image_utils.save_image(value, self.GRADIO_CACHE)
+        saved = image_utils.save_image(value, self.GRADIO_CACHE, self.format or "png")


Will this lead to loss of image quality for lossy formats?

I suppose it depends on what the default JPG encoding settings are?

gradio-pr-bot · 2024-03-18T21:39:49Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`gradio`	`minor`

Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

freddyaboulton · 2024-03-19T14:41:50Z

This should be good for review now

abidlabs · 2024-03-19T18:01:07Z

gradio/components/plot.py

@@ -56,6 +57,7 @@ def __init__(
        """
        Parameters:
            value: Optionally, supply a default plot object to display, must be a matplotlib, plotly, altair, or bokeh figure, or a callable. If callable, the function will be called whenever the app loads to set the initial value of the component.
+            format: File format to save matplotlib plots as, such as 'jpg' or 'png'. If set to None, will default to 'png'.


Suggested change

format: File format to save matplotlib plots as, such as 'jpg' or 'png'. If set to None, will default to 'png'.

format: File format in which to send plot data to the frontend, such as 'jpg' or 'png'. If set to None, will default to 'png'.

This only applies to Matplotlib plots so I'll keep that part and make the other suggestions in wording

abidlabs · 2024-03-19T18:05:24Z

gradio/components/annotated_image.py

@@ -67,6 +68,7 @@ def __init__(
        """
        Parameters:
            value: Tuple of base image and list of (annotation, label) pairs.
+            format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. If set to None, will use "png". This parameter only takes effect when numpy arrays or PIL Images are returned from the prediction function.


Suggested change

format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. If set to None, will use "png". This parameter only takes effect when numpy arrays or PIL Images are returned from the prediction function.

format: Format to save image before it is returned to the frontend, such as 'png' or 'jpeg'. This parameter only takes effect when the base image is returned from the prediction function as a numpy array or a PIL Image. The format should be supported by the PIL library.

abidlabs · 2024-03-19T18:05:47Z

gradio/components/annotated_image.py

@@ -49,6 +49,7 @@ def __init__(
        ]
        | None = None,
        *,
+        format: str | None = None,


Suggested change

format: str | None = None,

format: str = "png",

abidlabs · 2024-03-19T18:06:42Z

gradio/components/annotated_image.py

@@ -141,12 +144,12 @@ def postprocess(
            base_img = np.array(PIL.Image.open(base_img))
        elif isinstance(base_img, np.ndarray):
            base_file = processing_utils.save_img_array_to_cache(
-                base_img, cache_dir=self.GRADIO_CACHE
+                base_img, cache_dir=self.GRADIO_CACHE, format=self.format or "png"


If we make the change where format is, by default, png, we won't have to handle the case where it has to be None here and below

Yea good point

abidlabs · 2024-03-19T18:08:15Z

gradio/components/gallery.py

@@ -76,6 +77,7 @@ def __init__(
        """
        Parameters:
            value: List of images to display in the gallery by default. If callable, the function will be called whenever the app loads to set the initial value of the component.
+            format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. If set to None, will use "png". This parameter only takes effect when numpy arrays or PIL Images are returned from the prediction function.


Suggested change

format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. If set to None, will use "png". This parameter only takes effect when numpy arrays or PIL Images are returned from the prediction function.

format: Format to save images before they are returned to the frontend, such as 'png' or 'jpeg'. This parameter only applies to images that are returned from the prediction function as numpy arrays or PIL Images. The format should be supported by the PIL library.

abidlabs · 2024-03-19T18:08:36Z

gradio/components/gallery.py

@@ -50,6 +50,7 @@ def __init__(
        | Callable
        | None = None,
        *,
+        format: str | None = None,


Suggested change

format: str | None = None,

format: str = "png",

abidlabs · 2024-03-19T18:08:51Z

gradio/components/annotated_image.py

            )
            base_img_path = str(utils.abspath(base_file))
        elif isinstance(base_img, PIL.Image.Image):
            base_file = processing_utils.save_pil_to_cache(
-                base_img, cache_dir=self.GRADIO_CACHE
+                base_img, cache_dir=self.GRADIO_CACHE, format=self.format or "png"


abidlabs · 2024-03-19T18:09:00Z

gradio/components/gallery.py

@@ -176,12 +179,12 @@ def _save(img):
                img, caption = img
            if isinstance(img, np.ndarray):
                file = processing_utils.save_img_array_to_cache(
-                    img, cache_dir=self.GRADIO_CACHE
+                    img, cache_dir=self.GRADIO_CACHE, format=self.format or "png"


If we make the change where format is, by default, png, we won't have to handle the case where it has to be None here and below

abidlabs · 2024-03-19T18:09:05Z

gradio/components/gallery.py

                )
                file_path = str(utils.abspath(file))
            elif isinstance(img, PIL.Image.Image):
                file = processing_utils.save_pil_to_cache(
-                    img, cache_dir=self.GRADIO_CACHE
+                    img, cache_dir=self.GRADIO_CACHE, format=self.format or "png"


abidlabs · 2024-03-19T18:12:55Z

gradio/components/image.py

@@ -43,6 +43,7 @@ def __init__(
        self,
        value: str | PIL.Image.Image | np.ndarray | None = None,
        *,
+        format: str | None = None,


Suggested change

format: str | None = None,

format: str = "png",

abidlabs · 2024-03-19T18:13:39Z

gradio/components/image.py

@@ -69,6 +70,7 @@ def __init__(
        """
        Parameters:
            value: A PIL Image, numpy array, path or URL for the default value that Image component is going to take. If callable, the function will be called whenever the app loads to set the initial value of the component.
+            format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. This parameter has no effect on SVG files. If set to None, image will keep uploaded format.


Suggested change

format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. This parameter has no effect on SVG files. If set to None, image will keep uploaded format.

format: Format to save image, e.g. 'png' or 'jpeg'. The format should be supported by the PIL library. This parameter has no effect on SVG files or if the file is being returned to the frontend as a str filepath.

abidlabs · 2024-03-19T18:21:03Z

Just some nits to improve the docstrings, but otherwise lgtm @dfl and @freddyaboulton!

freddyaboulton · 2024-03-19T18:35:40Z

Thanks for the suggestions @abidlabs ! I made those modifications and merged in the latest main branch.

abidlabs

Thanks @freddyaboulton! Let's get this in

preserve file format during upload add format to image component

freddyaboulton · 2024-03-19T20:10:15Z

Thanks @dfl ! You'll be credited in the changelog!

freddyaboulton reviewed Mar 12, 2024

View reviewed changes

freddyaboulton mentioned this pull request Mar 12, 2024

preserve jpeg exif data; closes #7657 #7665

Closed

gleb-akhmerov reviewed Mar 17, 2024

View reviewed changes

freddyaboulton self-assigned this Mar 18, 2024

freddyaboulton changed the title ~~Preserve image format~~ Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in Mar 19, 2024

freddyaboulton requested review from abidlabs, whitphx, aliabid94, aliabd, dawoodkhan82, pngwn and hannahblair March 19, 2024 14:41

abidlabs reviewed Mar 19, 2024

View reviewed changes

abidlabs approved these changes Mar 19, 2024

View reviewed changes

abidlabs enabled auto-merge (squash) March 19, 2024 18:36

dfl and others added 13 commits March 19, 2024 15:11

WIP re: gradio-app#7486. Make less PNG-centric.

b1808a3

preserve file format during upload add format to image component

more format preservation re: gradio-app#7486

3801873

lint

8efda0b

lint; add comments; update Plot component

28d8584

add'l for Plot imaging defaults

95cd365

Working tests

34af192

add changeset

6699108

Add code

1ca8116

test

031b99d

Add code

cd43f45

add changeset

88219f1

Docstring improvements

c3115bd

Fix tests

41636f7

freddyaboulton force-pushed the preserve-image-format branch from c02537a to 41636f7 Compare March 19, 2024 19:11

abidlabs merged commit 853d945 into gradio-app:main Mar 19, 2024
7 checks passed

pngwn mentioned this pull request Mar 19, 2024

chore: update versions #7669

Merged

freddyaboulton mentioned this pull request Apr 3, 2024

ensure ImageEditor events work as expected #7845

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in #7680

Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in #7680

dfl commented Mar 12, 2024

gradio-pr-bot commented Mar 12, 2024 •

edited

freddyaboulton left a comment

freddyaboulton Mar 12, 2024

freddyaboulton Mar 12, 2024

freddyaboulton Mar 12, 2024

freddyaboulton Mar 12, 2024

gleb-akhmerov Mar 17, 2024 •

edited

dfl Mar 17, 2024

gradio-pr-bot commented Mar 18, 2024 •

edited

Something isn't right?

freddyaboulton commented Mar 19, 2024

abidlabs Mar 19, 2024

freddyaboulton Mar 19, 2024

abidlabs Mar 19, 2024 •

edited

abidlabs Mar 19, 2024

abidlabs Mar 19, 2024

freddyaboulton Mar 19, 2024

abidlabs Mar 19, 2024 •

edited

abidlabs Mar 19, 2024

abidlabs Mar 19, 2024

abidlabs Mar 19, 2024

abidlabs Mar 19, 2024

abidlabs Mar 19, 2024

abidlabs Mar 19, 2024 •

edited

abidlabs commented Mar 19, 2024 •

edited

freddyaboulton commented Mar 19, 2024

abidlabs left a comment

freddyaboulton commented Mar 19, 2024

		@@ -51,13 +51,17 @@ def format_image(
		)


		def save_image(y: np.ndarray \| PIL.Image.Image \| str \| Path, cache_dir: str):

	format: File format to save matplotlib plots as, such as 'jpg' or 'png'. If set to None, will default to 'png'.
	format: File format in which to send plot data to the frontend, such as 'jpg' or 'png'. If set to None, will default to 'png'.

	format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. If set to None, will use "png". This parameter only takes effect when numpy arrays or PIL Images are returned from the prediction function.
	format: Format to save image before it is returned to the frontend, such as 'png' or 'jpeg'. This parameter only takes effect when the base image is returned from the prediction function as a numpy array or a PIL Image. The format should be supported by the PIL library.

	format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. If set to None, will use "png". This parameter only takes effect when numpy arrays or PIL Images are returned from the prediction function.
	format: Format to save images before they are returned to the frontend, such as 'png' or 'jpeg'. This parameter only applies to images that are returned from the prediction function as numpy arrays or PIL Images. The format should be supported by the PIL library.

	format: Format used to save images, such as 'jpeg' or 'png'. The jpeg format uses less memory at the cost of image quality. This parameter has no effect on SVG files. If set to None, image will keep uploaded format.
	format: Format to save image, e.g. 'png' or 'jpeg'. The format should be supported by the PIL library. This parameter has no effect on SVG files or if the file is being returned to the frontend as a str filepath.

Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in #7680

Add format parameter to gr.Image, gr.Gallery, gr.AnnotatedImage, gr.Plot to control format to save image files in #7680

Conversation

dfl commented Mar 12, 2024

Description

Tests

gradio-pr-bot commented Mar 12, 2024 • edited

🪼 branch checks and previews

freddyaboulton left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gleb-akhmerov Mar 17, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gradio-pr-bot commented Mar 18, 2024 • edited

🦄 change detected

This Pull Request includes changes to the following packages.

With the following changelog entry.

Something isn't right?

freddyaboulton commented Mar 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abidlabs Mar 19, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abidlabs Mar 19, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abidlabs Mar 19, 2024 • edited

Choose a reason for hiding this comment

abidlabs commented Mar 19, 2024 • edited

freddyaboulton commented Mar 19, 2024

abidlabs left a comment

Choose a reason for hiding this comment

freddyaboulton commented Mar 19, 2024

gradio-pr-bot commented Mar 12, 2024 •

edited

gleb-akhmerov Mar 17, 2024 •

edited

gradio-pr-bot commented Mar 18, 2024 •

edited

abidlabs Mar 19, 2024 •

edited

abidlabs Mar 19, 2024 •

edited

abidlabs Mar 19, 2024 •

edited

abidlabs commented Mar 19, 2024 •

edited