Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create wrapper for PyTorch models #82

Closed
ahousley opened this issue Feb 6, 2018 · 5 comments
Closed

Create wrapper for PyTorch models #82

ahousley opened this issue Feb 6, 2018 · 5 comments
Labels
help wanted

Comments

@ahousley
Copy link
Member

ahousley commented Feb 6, 2018

There are issues to create wrappers for R and Spark, so adding PyTorch here too as it was raised the community Slack earlier today. This would be a great first PR for anyone looking to contribute!

  • expose PyTorch models with a thin REST server or gRPC server that respects the internal model API
  • Build Docker image
@jlewi
Copy link

jlewi commented Jul 2, 2018

For the Kubeflow 0.3.0 release I think we'd like to flush out our PyTorch story which includes finding a good way to serve PyTorch models.

kubeflow/kubeflow#1117

@ukclivecox
Copy link
Contributor

We have an IBM FfDL Example. We could close this or work with Kubeflow on a focused example.

@dsdinter
Copy link

FYI - I am working on a Kubeflow Pytorch example leveraging Seldon.
Link to the PR.

@ukclivecox
Copy link
Contributor

@dsdinter That's great - we'll follow your updates. Feel free to reach out if there are any questions.

@ukclivecox
Copy link
Contributor

Kubeflow example exists. So closing.

agrski pushed a commit that referenced this issue Dec 2, 2022
@ukclivecox
Prometheus Inference Metrics (#82)
592e1f7 
* initial wiring for http reverse proxy

* server config updates

* lint

* remove enable proxy

* docker compose updates

* Update envoy processing so each route has a unqiue name and is not set to model name.

* Update set state method to take an expected state to transition from

* add missing Docker compose files

* lint

* Add model lock/unlock

* lint

* Add test for unexpected state change

* lint

* Add cluster based weighting to split traffic

* review comments

* remove extract model from path method

* remove debug stmt

* Add prometheus metrics to http reverse proxy

* add grpc metrics

* Make namespace name safe for prometheus metric prefix

* Update server configs for metrocs

* install prometheus and monitors

* Fix ports - always need to direct at revere proxy ports as envoy will notuse svc ports but direct pod ports from headless svc

* rename metrics and add a latency sum metric

* review comment fixes

* small typo - space

* update metrics

* review fixes

* Review fixes

* review fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jlewi @ahousley @dsdinter @ukclivecox