Queueing [tracking and feedback]

[pngwn]

We a few issues left regarding the new queue and it would be good to track them together.

Bugs

• [Priority] Reconnect when the ws connection is lost #2043
• Queue upstream when loading apps via gr.Interface.load() #1316
• Gracefully Scaling Down on Spaces With the new Queue #2019
• Can't embed multiple spaces on the same page if spaces use different queue implementation #2114
• Colab hosted GradIO doesnt receive output and hangs if process is longer then 60 seconds #2111
• Websocket connection fails inside browser console for hosted version #2096

UI/UX improvements

• A component that shows the status of the queue before a prediction is submitted #435
• Send ETA before Processing in Queue #2004
• Enable a maximum eta for the queue  #1862
• Improve Queue ETA calculation via exponential weighted average #1984
• Multi-tasking on mobile (iOS/Android) seems to silently remove user from the queue #2075
• Enable queuing by default everywhere #2081

Faster inference for users

• GPU Input Batching #1597
• Keep WS Connections Open After Processing #2001

Testing and Documentation

• Test Performance of the New Queue after Redesign #1813
• Write Tests for Queueing #1983
• Write an Advanced Guide about Queue, Concurrency, Batching in Gradio #2016

If you are a Gradio user, we'd love for you to try out the new queue, which is available in the latest version of Gradio (gradio>=3.3). Example Space with queue enabled: https://huggingface.co/spaces/farukozderim/test-new-queue

Share any feedback below!

[abidlabs]

Closing this tracking issue and milestone as we're not tackling the remaining issues "as a group". Will leave the individual issues open as we'll get around to solving them eventually.