You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With the support for MMS, there is a question of how we can support parallel inference at the same time across multiple models. The main blocker for this is Python's GIL, which blocks true parallelism within the same process.
The scope of this issue is to explore different alternatives (e.g. using multiprocessing) to support parallel inference.
The text was updated successfully, but these errors were encountered:
axsaucedo
changed the title
Explore parallel inference
Parallel Inference
May 6, 2021
With the support for MMS, there is a question of how we can support parallel inference at the same time across multiple models. The main blocker for this is Python's GIL, which blocks true parallelism within the same process.
The scope of this issue is to explore different alternatives (e.g. using
multiprocessing
) to support parallel inference.The text was updated successfully, but these errors were encountered: