API for cpu bound task

[ileixe]

Hi

I'd like to ask introducing new API to support pure CPU intensive (which is not blocking I/O) task. In ML domain (from FastAPI), each ML serving server is trying to solve this problem introducing specific APIs (basically based on multiprocessing). I think if starlette can provide general API to support such task, it would affect greatly to entire community powered by FastAPI.

To be specific, I want to ask it's viable to use anyio.to_process(). As starlette have been used anyio.to_thread() , I think it makes sense to use symmetric API to support multiprocess.

Any advice would be appreciated.

Thanks.

[alex-oleshkevich]

I think the background task processors like RQ or Celery are better suited for this.
Having it as a part of Starlette can lead to high resource usage because in concurrent environment you can spawn a lot of subprocesses and we would need some mechanism to control process concurrency which is out of scope of web framework.

[adriangb]

Agreed. A web framework is not the right place to handle this. I recommend introducing some sort of abstraction around how thing get executed and using that to control reliable execution (e.g. just run it and block the event loop in tests), subprocess execution (maybe this is what your first production version does) or putting work into some sort of distributed queue (maybe if you need to scale up).

[ileixe]

Ok makes sense. Thanks for the answers.