Enable streaming audio in python client #5248

freddyaboulton · 2023-08-16T21:30:48Z

Description

Update the FileSerializable so that it can handle streaming responses introduced in #5077

import gradio_client as grc
client = grc.Client("<local-url-of-streaming-audio-demo>")
job = client.submit("/Users/freddy/Pictures/bark_demo.mp4", fn_index=1)

# The entire audio stream
job.result()

# Each individual chunk
job.outputs()

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Tests

PRs will only be merged if tests pass on CI. To run the tests locally, please set up your Gradio environment locally and run the tests: bash scripts/run_all_tests.sh
You may need to run the linters: bash scripts/format_backend.sh and bash scripts/format_frontend.sh

gradio-pr-bot · 2023-08-16T21:31:04Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
	Visual tests	all good!	Build review
🦄	Changes	detected!	Details

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/387da3b5846278da4ddbb56e0f2b669fa1925129/gradio-3.40.1-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@387da3b5846278da4ddbb56e0f2b669fa1925129#subdirectory=client/python"

gradio-pr-bot · 2023-08-16T21:31:33Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`gradio`	`minor`
`gradio_client`	`minor`

With the following changelog entry.

Enable streaming audio in python client

The gradio_client now supports streaming file outputs 🌊

No new syntax! Connect to a gradio demo that supports streaming file outputs and call predict or submit as you normally would.
import gradio_client as grc
client = grc.Client("gradio/stream_audio_out")

# Get the entire generated audio as a local file
client.predict("/Users/freddy/Pictures/bark_demo.mp4", api_name="/predict")

job = client.submit("/Users/freddy/Pictures/bark_demo.mp4", api_name="/predict")

# Get the entire generated audio as a local file
job.result()

# Each individual chunk
job.outputs()

⚠️ The changeset file for this pull request has been modified manually, so the changeset generation bot has been disabled. To go back into automatic mode, delete the changeset file.

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

freddyaboulton · 2023-08-16T21:50:02Z

gradio/blocks.py

-                    "is_stream": True,
-                }
+                if data[i]:
+                    data[i]["is_file"] = False


Preserve the orig_name so that clients have an easier time downloading the byte stream

But isn't ["name"] assigned to the same value as before in the next line?

Yea we're using "name" to mean the url from which to download the stream so I'm keeping that the same. Just preserving orig_name (if it's there) so that we can give a meaningful name when we go to download the byte stream to a file.

freddyaboulton · 2023-08-16T21:51:49Z

gradio/blocks.py

-                }
+                if data[i]:
+                    data[i]["is_file"] = False
+                    data[i]["name"] = f"{session_hash}/{run}/{output_id}"


@aliabid94 Thoughts on including the stream endpoint name in the route so that it doesn't have to be hardcoded in the client?

abidlabs · 2023-08-17T02:52:22Z

client/python/gradio_client/serializing.py

+                    self.stream = self._setup_stream(
+                        root_url + "stream/" + x["name"], hf_token=hf_token
+                    )
+                    chunk = next(self.stream)


Sorry can you explain why these two lines are repeated from above?

It was to determine when the stream changed but there's a better/clearer way. Will push up now!

abidlabs · 2023-08-17T03:36:10Z

client/python/gradio_client/utils.py

+        for data in r.iter_bytes():
+            arr += data
+            yield data
+    yield arr


Am I understanding correctly that what's happening is that we are iterating chunk by chunk and then finally yielding the full array so that .predict() returns the full data array? 2 questions:

(1) How come outputs() doesn't include this final full data array?
(2) Will r.iter_bytes() work if the streaming response returns binary data (as in #5238) instead of file data?

(1) outputs only includes the values returned from the server during the process_generating event
(2) yea the stream enpoint only serves bytes so it should work when #5238 is merged

abidlabs · 2023-08-17T03:36:51Z

Really nice @freddyaboulton! Just a few questions to understand the implementation -- will test more thoroughly tomorrow.

abidlabs

Very nice @freddyaboulton! LGTM

abidlabs · 2023-08-21T19:15:33Z

I'll go ahead and merge this in so that we can consolidate things before the release

freddyaboulton and others added 3 commits August 16, 2023 17:36

Add code

120a7fd

Remove file

509dda6

add changeset

ebfbfae

freddyaboulton force-pushed the enable-streaming-in-client branch from 7c3ce18 to ebfbfae Compare August 16, 2023 21:36

freddyaboulton added t: feat A change that implements a feature t: highlight A change that we wish to highlight labels Aug 16, 2023

add changeset

8783d98

freddyaboulton removed the t: feat A change that implements a feature label Aug 16, 2023

Update chilly-fans-make.md

7879a0d

freddyaboulton commented Aug 16, 2023

View reviewed changes

freddyaboulton added 4 commits August 16, 2023 18:01

lint

3c0ee45

Lint

51329b8

Add ffmpeg

5e5f4eb

Lint

416f818

freddyaboulton marked this pull request as ready for review August 16, 2023 22:42

freddyaboulton requested review from aliabid94, abidlabs, pngwn, aliabd, dawoodkhan82 and hannahblair August 16, 2023 22:43

abidlabs reviewed Aug 17, 2023

View reviewed changes

freddyaboulton and others added 2 commits August 17, 2023 11:27

Cleaner way to handle stream change

0337b8f

Merge branch 'main' into enable-streaming-in-client

cc1f16c

freddyaboulton self-assigned this Aug 21, 2023

Fix windows test

48cc495

abidlabs approved these changes Aug 21, 2023

View reviewed changes

abidlabs mentioned this pull request Aug 21, 2023

Enable streaming audio in JS client #5281

Open

abidlabs merged commit 390624d into main Aug 21, 2023
20 checks passed

abidlabs deleted the enable-streaming-in-client branch August 21, 2023 19:15

pngwn mentioned this pull request Aug 21, 2023

chore: update versions #5177

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable streaming audio in python client #5248

Enable streaming audio in python client #5248

freddyaboulton commented Aug 16, 2023

gradio-pr-bot commented Aug 16, 2023 •

edited

gradio-pr-bot commented Aug 16, 2023 •

edited

Enable streaming audio in python client

Something isn't right?

freddyaboulton Aug 16, 2023

abidlabs Aug 17, 2023 •

edited

freddyaboulton Aug 21, 2023

freddyaboulton Aug 16, 2023

abidlabs Aug 17, 2023

freddyaboulton Aug 17, 2023

abidlabs Aug 17, 2023

freddyaboulton Aug 17, 2023

abidlabs commented Aug 17, 2023

abidlabs left a comment

abidlabs commented Aug 21, 2023

Enable streaming audio in python client #5248

Enable streaming audio in python client #5248

Conversation

freddyaboulton commented Aug 16, 2023

Description

🎯 PRs Should Target Issues

Tests

gradio-pr-bot commented Aug 16, 2023 • edited

🪼 branch checks and previews

gradio-pr-bot commented Aug 16, 2023 • edited

🦄 change detected

This Pull Request includes changes to the following packages.

With the following changelog entry.

Enable streaming audio in python client

Something isn't right?

freddyaboulton Aug 16, 2023

Choose a reason for hiding this comment

abidlabs Aug 17, 2023 • edited

Choose a reason for hiding this comment

freddyaboulton Aug 21, 2023

Choose a reason for hiding this comment

freddyaboulton Aug 16, 2023

Choose a reason for hiding this comment

abidlabs Aug 17, 2023

Choose a reason for hiding this comment

freddyaboulton Aug 17, 2023

Choose a reason for hiding this comment

abidlabs Aug 17, 2023

Choose a reason for hiding this comment

freddyaboulton Aug 17, 2023

Choose a reason for hiding this comment

abidlabs commented Aug 17, 2023

abidlabs left a comment

Choose a reason for hiding this comment

abidlabs commented Aug 21, 2023

gradio-pr-bot commented Aug 16, 2023 •

edited

gradio-pr-bot commented Aug 16, 2023 •

edited

abidlabs Aug 17, 2023 •

edited