Non blocking streaming endpoint

[dougsouza]

Hello @miguelgrinberg, thanks for your great tutorial on video streaming with flask. I've been working on a web app that streams some live video (from a webcam). The stream works great but when I need to call other API endpoints it doesn't work. It seems that my flask app is busy with the streaming endpoint and doesn't process any other request. I've done some research, tried disabling debug mode and using gunicorn and gevent but nothing worked so far. Do you have any idea how I can solve this?

Thanks!

[miguelgrinberg]

You may have done something wrong, because gevent should have solved your problem.

But let's rewind a bit. These are your options to support concurrent requests:

• With the Flask development web server, you can add threaded=True to your app.run() call, or --with-threads if you use the Flask CLI.

• With Gunicorn. Install it with pip, then start the server with

    gunicorn --threads 5 --workers 1 --bind 0.0.0.0:5000 app:app
  

  This will give you five concurrent requests (i.e. one stream and four more, or two streams and three more, etc.). You can change the 5 to the number that addresses your needs.

• With Gunicorn and gevent. Install both with pip, then run with

    gunicorn --worker-class gevent --workers 1 --bind 0.0.0.0:5000 app:app
  

  This should give you up to 1000 concurrent requests.

• Instead of gevent, you can also use eventlet. Install with pip, then run with

    gunicorn --worker-class eventlet --workers 1 --bind 0.0.0.0:5000 app:app
  

  This should also give you up to 1000 concurrent requests.

[dougsouza]

@miguelgrinberg, indeed. I added a second endpoint to your code to test and it works. I'll figure it out what I am doing wrong. Thanks!

[dougsouza]

@miguelgrinberg, the request does get processed, but it takes a lot of time (like 5 seconds, sometimes more) any idea where the bottleneck might be? I do some (heavy) image processing that I moved to your background thread, like this:

class Camera(BaseCamera):
    
    @staticmethod
    def frames():
        camera = cv2.VideoCapture(0)
        if not camera.isOpened():
            raise RuntimeError('Could not start camera.')
        styles = Styles() # this loads lots of stuff in GPU memory
        current_style = 'la_muse'

        while True:
            # read current frame
            _, frame = camera.read()
             # this does some (heavy) GPU computation
            frame = styles.convert_to_style(frame[:, :, ::-1], current_style)[:, :, ::-1]
            frame = cv2.resize(frame, (640, 480))
            frame = cv2.imencode('.jpg', frame)[1].tobytes()
            # encode as a jpeg image and return it
            yield frame

[miguelgrinberg]

Did you try the gunicorn approach without gevent? The async options require cooperative multitasking, they are not a great match for tasks that do heavy computing. Try with --threads 5, I think that should work better since it will work with regular threads, which the OS knows how to schedule in and out of the CPU.

[dougsouza]

It doesn't work. I make lots of calls to other endpoints, they take a huge time to process, if I refresh the browser they all come instantly (because the streaming has stopped for a second, I assume). I don't know what else to do. I'm considering splitting the app in two servers, one for the web app itself and regular APIs and other just for the streaming, and I would have to find a way to interact with the streaming server other than using API calls, of course.

[miguelgrinberg]

You can always restrict your camera thread and your streaming generator to deliver frames at a specific rate instead of going as fast as possible, that should give other tasks time to run. Just add a sleep inside their loops to slow them down a bit.

[dougsouza]

Finally, it worked. Thank you very much!!

[miguelgrinberg]

This issue will be automatically closed due to being inactive for more than six months. Seeing that I haven't responded to your last comment, it is quite possible that I have dropped the ball on this issue and I apologize about that. If that is the case, do not take the closing of the issue personally as it is an automated process doing it, just reopen it and I'll get back to you.