Delete user state when they close the tab. Add an unload event for the demo and a delete_callback on gr.State to let developers control how resources are cleaned up

[freddyaboulton]

Description

This PR adds some features to address the problem of state being accumulated in the gradio server indefinitely.

Changes:

• Once a user leaves closes the tab, state stored for that session in the server is deleted.
• Add a time_to_live parameter to gr.State. This is how long the state will be held onto in the server after its most recent creation/update. This is for cases where users minimize the tab and forget about it.
• Add a delete_callback parameter to gr.State to let developers do any additional clean up actions when state is deleted, e.g. delete tensors from GPU memory. This parameter accepts a function that takes one parameter, the state value.
• Add a Blocks.unload method to set an "event" that is triggered when a user closes the page. The goal is to perform any additional clean up not pertaining to gr.State when a user leaves the page. The state is automatically deleted even if no unload event is attached to the demo. This event only accepts one parameter, the function to run. No inputs and outputs are accepted because there is no longer a client connection to receive or send data to.

To test, try the demo/state_cleanup or demo/unload_event_test demos.

Closes: #7517

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Tests

1. PRs will only be merged if tests pass on CI. To run the tests locally, please set up your Gradio environment locally and run the tests: bash scripts/run_all_tests.sh

2. You may need to run the linters: bash scripts/format_backend.sh and bash scripts/format_frontend.sh

[gradio-pr-bot]

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
🦄	Changes	detected!	Details

---

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/fb4ff449b2cad1ba38e5117261cb6a76ac43e2cc/gradio-4.24.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@fb4ff449b2cad1ba38e5117261cb6a76ac43e2cc#subdirectory=client/python"

[gradio-pr-bot]

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
@gradio/client	minor
gradio	minor
gradio_client	minor

With the following changelog entry.

Automatically delete state after user has disconnected from the webpage

Gradio now automatically deletes gr.State variables stored in the server's RAM when users close their browser tab.
The deletion will happen 60 minutes after the server detected a disconnect from the user's browser.
If the user connects again in that timeframe, their state will not be deleted.

Additionally, Gradio now includes a Blocks.unload() event, allowing you to run arbitrary cleanup functions when users disconnect (this does not have a 60 minute delay).
You can think of the unload event as the opposite of the load event.

with gr.Blocks() as demo:
    gr.Markdown(
"""# State Cleanup Demo
🖼️ Images are saved in a user-specific directory and deleted when the users closes the page via demo.unload.
""")
    with gr.Row():
        with gr.Column(scale=1):
            with gr.Row():
                img = gr.Image(label="Generated Image", height=300, width=300)
            with gr.Row():
                gen = gr.Button(value="Generate")
            with gr.Row():
                history = gr.Gallery(label="Previous Generations", height=500, columns=10)
                state = gr.State(value=[], delete_callback=lambda v: print("STATE DELETED"))

    demo.load(generate_random_img, [state], [img, state, history]) 
    gen.click(generate_random_img, [state], [img, state, history])
    demo.unload(delete_directory)


demo.launch(auth=lambda user,pwd: True,
            auth_message="Enter any username and password to continue")

⚠️ The changeset file for this pull request has been modified manually, so the changeset generation bot has been disabled. To go back into automatic mode, delete the changeset file.

Something isn't right?

• Maintainers can change the version label to modify the version bump.
• If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

[freddyaboulton]

I did the experiment where I killed my network connection and the state did not get deleted when I reconnected.

[abidlabs]

Nice! How's that possible though? An asyncio.CancelledError should be raised when the network disconnects right?

[freddyaboulton]

Maybe chrome keeps the connections alive but doesn't allow new requests to go through?

[abidlabs]

Yes that's what I was thinking as well

[freddyaboulton]

cc @pseudotensor - I think you had asked for automatic state deletion.

[pseudotensor]

Nice, I can remove my work-around involving a web socket if this works.

I'm still waiting for the k8 nginx proxy stuff to be fixed before moving forward with gradio 4 in general. Also the audio streaming regression isn't fixed yet.

[abidlabs]

Nice to include this in the clients as well!

[abidlabs]

It looks like the python unit tests were timing out? Rerunning..

[freddyaboulton]

@abidlabs I just pushed up your suggestion (state deleted 60 minutes after client disconnects) and made other minor tweaks (like highlight changelog)!

[abidlabs]

Super cool PR! LGTM @freddyaboulton

[freddyaboulton]

Thanks for the review @abidlabs !!!