Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate Nvidia's TensorRT #121

Closed
ukclivecox opened this issue Mar 20, 2018 · 3 comments
Closed

Investigate Nvidia's TensorRT #121

ukclivecox opened this issue Mar 20, 2018 · 3 comments

Comments

@ukclivecox
Copy link
Contributor

The ability for users to serve their models using the optimization framework Nvidia's TensorRT could be useful.

At present it comes hardwired with a set REST server. Would need to modify this to use our microservice prediction API.
@ukclivecox ukclivecox changed the title Investigate NvidiaRT Investigate Nvidia's TensorRT Mar 20, 2018
@ukclivecox
Copy link
Contributor Author

https://developers.googleblog.com/2018/03/tensorrt-integration-with-tensorflow.html

This shows more understanding is needed as to what are the blockers for use of TensorRT before we do any work within seldon-core

@jlewi
Copy link

jlewi commented Jul 2, 2018

In Kubeflow we would like to create an example for serving with GPUs: kubeflow/examples#145

@ukclivecox
Copy link
Contributor Author

We have an example notebook for TensorRT now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jlewi @ukclivecox