You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current ml_api forces workers=1 because it calls out to native code and there are a lot of static bridging variables exist between Python and C. Need to have a way to enable concurrency > 1 when running prediction. Ideally it should be done in a way to maximize GPU utilization.
The text was updated successfully, but these errors were encountered:
Current
ml_api
forcesworkers=1
because it calls out to native code and there are a lot of static bridging variables exist between Python and C. Need to have a way to enable concurrency > 1 when running prediction. Ideally it should be done in a way to maximize GPU utilization.The text was updated successfully, but these errors were encountered: