Multiple model with one gpu

[tianhuat]

If I have multiple models but with one GPU, can tensorflow serving allows multiple model to share one GPU? Also how do I enable the usage of GPU - as currently seems like only CPU is used though I have GPU installed (and is used with tensorflow training/inference without problem).

[arghyaganguly]

@tianhuat , for serving with GPU please check :- serving with gpu
We can definitely serve multiple models on a single GPU.
Please check link1 and link2 for more information.

[tianhuat]

On this link, #335, it shows errors for concurrent requests of multiple models on single GPU, has it been fixed?

[arghyaganguly]

#335 is referencing to an earlier version of tf(1.1) and tf serving.
As per comment#335,this was fixed.
Please try out with the latest stable versions and let us know if the issue exists.

[arghyaganguly]

This issue has been with the "awaiting response" tag for past 5 days.
Closing this as of now.
Please feel free to reopen with comment if you have questions based on the above trace.