Using granian for flask

[LohithBlaze]

Hi,

When we use granian for flask services, it looks like granian doesnt switch the threads and the flask is only serving one request at a time.

is there something I'm missing here? here is the granian configuration used.
granian.Granian(target="main:app", address="0.0.0.0", interface=granian.server.Interfaces.WSGI, port=4001, log_level=granian.server.LogLevels.debug,
workers=workers,threads = threads, websockets=False ,reload=True,respawn_failed_workers=True,
backlog=100000).serve()

Should I increase the blocking-threads configuration to something greater than 1? is it due to this configuration?

[gi0baro]

threads are the rust ones, so the ones involved in network I/O.
blocking-threads are the ones involved into Python code execution.

Thus you probably want to increase the blocking ones and leave the other value as per default. Also, the backlog is about TCP connections, so such a large value won't help you. The default value should be enough for the vast majority of use cases (gunicorn has 2048 if I remember correctly)

[LohithBlaze]

the flask services has calls to other internal micro services and also DB calls, so consider I want to handle 1000 - 2000 RPS, does it make sense to increase the blocking threads or does 2048 is enough? the worker count is cpu_count * 2 + 1 and the threads count is worker_count * 2.

[gi0baro]

Again, the backlog are TCP connections waiting to be processed, increasing that value won't change anything if you can't process that amount of requests in a reasonable amount of time. So IMHO 2048 is perfectly fine.

Worker exceeding numbers of CPU won't necessarily help, as it just might increase the response time. I'd personally run tests starting with a number of workers matching CPU cores.
Also, both threads and blocking threads are already per worker, so when you set those numbers you should keep that in mind. Depending on your average response time spawning too many threads might just increase the response time as well. But again: you need to experiment with your own application, there's no magic number or formula out there.