New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow caching examples with streamed output #5295
Conversation
🪼 branch checks and previews
Install Gradio from this PR pip install https://gradio-builds.s3.amazonaws.com/18265c19a93495cee775923ada7ca13c8a5e1a72/gradio-3.40.1-py3-none-any.whl Install Gradio Python Client from this PR pip install "gradio-client @ git+https://github.com/gradio-app/gradio@18265c19a93495cee775923ada7ca13c8a5e1a72#subdirectory=client/python" |
🦄 change detectedThis Pull Request includes changes to the following packages.
With the following changelog entry.
Maintainers or the PR author can modify the PR title to modify this entry.
|
…p/gradio into cache_streamed_content
gradio/helpers.py
Outdated
return x | ||
|
||
fn = get_final_item | ||
else: | ||
fn = self.fn | ||
# create a fake dependency to process the examples and get the predictions | ||
from gradio.events import StreamableOutput |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: better to put the import at the top of the function, or if possible, at the top of the file.
Edit: ignore my comment here @aliabid94. I chatted with @freddyaboulton and although there may a better way to fix this, it would require making some changes to the client that we can keep for 4.0 (@freddyaboulton's implemented them separately on this branch) |
lgtm @aliabid94, if we could add a unit test for the chunk concatenation, that'd be great. |
Done |
* changes * changes * add changeset * add changeset * chages * Update silver-clowns-brush.md * changes * chagers * changes * Update silver-clowns-brush.md * change * change * change * changes * chages * changes * add changeset * changes * changes * changes --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: pngwn <hello@pngwn.io>
* disable sanitize * add changeset * Fix devcontainer postCreateCommand (#5289) * add chmod commands to grant execute permissions to each script before running them * increase the memory limit * Increase memory requirement to 8 * Setting a minimum specification for codespace machines * Rotate axes labels on LinePlot, BarPlot, and ScatterPlot (#5305) * add chmod commands to grant execute permissions to each script before running them * increase the memory limit * Increase memory requirement to 8 * Setting a minimum specification for codespace machines * Added label_angle parameter to LinePlot * add the x_label_angle and y_label_angle to the postprocess * Add x_label_angle and y_label_angle to ScatterPlot * Add x_label_angle and y_label_angle to BarPlot * Remove postCreateCommand update from this PR, it is in its own PR * Remove custumization on the devcontainer, it is on another PR * Remove extra line on the devcontainer, it is on another PR * add changeset * update the parameter definition in the docstring --------- Co-authored-by: Abubakar Abid <abubakar@huggingface.co> Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * `make_waveform()` twitter video resolution fix (#5296) * waveform res fix * add changeset * add changeset * Improve audio streaming (#5238) * changes * changes * add changeset * add changeset * chages * Update silver-clowns-brush.md * changes * chagers * changes * Update silver-clowns-brush.md * change * change --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: pngwn <hello@pngwn.io> * Tweaks to `icon` parameter in `gr.Button()` (#5285) * button * add changeset * fix * add changeset * add changeset * types * type fix * formatting * fix based on review * fix unit tests * stories --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * test * test --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: aliabid94 <aabid94@gmail.com> Co-authored-by: pngwn <hello@pngwn.io> Co-authored-by: Abubakar Abid <abubakar@huggingface.co> * Allow caching examples with streamed output (#5295) * changes * changes * add changeset * add changeset * chages * Update silver-clowns-brush.md * changes * chagers * changes * Update silver-clowns-brush.md * change * change * change * changes * chages * changes * add changeset * changes * changes * changes --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: pngwn <hello@pngwn.io> * Create event listener table for components on docs (#5298) * changes * changes * remove languages from code docs * fix img path in guide * rework event listeners * add changeset * remove console log * name it event arguments * Update js/_website/src/lib/components/EventListeners.svelte Co-authored-by: Abubakar Abid <abubakar@huggingface.co> * Update js/_website/src/lib/components/EventListeners.svelte Co-authored-by: Abubakar Abid <abubakar@huggingface.co> * requested changes --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: Abubakar Abid <abubakar@huggingface.co> * merge * fix (#5308) * only start listening for events after the components are mounted (#5312) * only start listening for events after the components are mounted * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * Restores missing part of bottom border on file component (#5313) * removes missing part of bottom border on file component * add changeset * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * chore: update versions (#5177) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Test Blocks.svelte footer copy (#5307) * test test * fix test * tweak * tweak test name * ensure login form has correct styles (#5324) * fix changelog generation (#5328) * Add ignore scripts to pnpm i in pypi action (#5316) * add json generation to prepare * add changeset * remove pnpm prepare * add changeset * bring back prepare, add ignore scripts * delete changeset * add to right commadn * add frozen lockfile to gh_action_pypi * remove unused gh_action_pypi script * remove unused scripts --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * ensure dropdown stays open when identical data is passed in (#5323) * ensure dropwdown stays open when new data is passed in * add changeset * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * Fix ci again again again (#5329) * fix changelog generation * fix changelog generation * fix * chore: update versions (#5326) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix ci again again again (#5330) * fix changelog generation * fix changelog generation * fix * fix script * Fix ci again again again (#5331) * fix changelog generation * fix changelog generation * fix * fix script * fix script * Fix ci again again again (#5332) * fix changelog generation * fix changelog generation * fix * fix script * fix script * fix vars * Fix ci again again again (#5333) * fix changelog generation * fix changelog generation * fix * fix script * fix script * fix vars * fix vars * Fix Checkbox select dispatch (#5340) * Fix bug * add changeset --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * Minor bug fix sweep (#5284) * changes * changes * add changeset * changes * changes * fix box changes on website * add changeset * changes * changes * Revert "changes" This reverts commit 189b4e8. * chanegs * changes * changes * changes * changes * add changeset * Update fancy-bats-deny.md --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: aliabd <ali.si3luwa@gmail.com> Co-authored-by: Abubakar Abid <abubakar@huggingface.co> * Fix: wrap avatar-image in a div to clip its shape (#5319) * fix: wrap avatar-image in a div to clip its shape and add data url support * add changeset * revert get file from data url * add changeset --------- Co-authored-by: Dawood Khan <dawoodkhan82@gmail.com> Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> * chore: update versions (#5341) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * add kwarg to markdown * add changeset * config --------- Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com> Co-authored-by: Faiga Alawad <faiga.alawd91@gmail.com> Co-authored-by: Abubakar Abid <abubakar@huggingface.co> Co-authored-by: aliabid94 <aabid94@gmail.com> Co-authored-by: pngwn <hello@pngwn.io> Co-authored-by: Ali Abdalla <ali.si3luwa@gmail.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Hannah <hannahblair@users.noreply.github.com> Co-authored-by: Freddy Boulton <alfonsoboulton@gmail.com> Co-authored-by: Keldos <Keldos.ljw@gmail.com>
This PR allows setting cache_examples=True when a function generates streamed content. It works by capturing the yielded content, and for any StreamingOutput components, concatening the output binary content and saving it as a file.
In the process of doing this, I realized that when loading from cache, we use the serialize method, which unnecessarily converts the content into base64 in this case. I added an
allow_links
argument which will let the cached examples return links, making them faster. (This was also done because the the audio playback was giving me issues when trying to set the src of the audio element to a base64 without specifying the mimetype).