New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add HuggingFace runtime out-of-the-box support #3395
Conversation
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
33b071a
to
909d7e8
Compare
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
81ea759
to
1d0bf3f
Compare
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
@@ -44,6 +44,8 @@ type PredictorSpec struct { | |||
Triton *TritonSpec `json:"triton,omitempty"` | |||
// Spec for ONNX runtime (https://github.com/microsoft/onnxruntime) | |||
ONNX *ONNXRuntimeSpec `json:"onnx,omitempty"` | |||
// Spec for HuggingFace runtime (https://github.com/huggingface) | |||
HuggingFace *HuggingFaceRuntimeSpec `json:"huggingface,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually we are trying to deprecate these individually defined runtime types as we are moving users to the model spec instead. @sivanantha321 has a PR for this #3144
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I am aware of that PR and thought about it as well. IMO we should include this for consistency (better UX) and then deprecate all the individually defined runtimes together later as part of #3144.
@terrytangyuan Looks great! can you help add an e2e test? |
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: terrytangyuan, yuzisun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
* feat: Add HuggingFace runtime out-of-the-box support Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Update violation list Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Add to charts Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Regenerate everything Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> --------- Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> Signed-off-by: Tim Kleinloog <tkleinloog@deeploy.ml>
* feat: Add HuggingFace runtime out-of-the-box support Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Update violation list Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Add to charts Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Regenerate everything Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> --------- Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> Signed-off-by: Tim Kleinloog <tkleinloog@deeploy.ml>
* feat: Add HuggingFace runtime out-of-the-box support Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Update violation list Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Add to charts Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Regenerate everything Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> --------- Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> Signed-off-by: Tim Kleinloog <tkleinloog@deeploy.ml>
* feat: Add HuggingFace runtime out-of-the-box support Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Update violation list Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Add to charts Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Regenerate everything Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> --------- Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
* feat: Add HuggingFace runtime out-of-the-box support Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Update violation list Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Add to charts Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Regenerate everything Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * fix lint Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * Empty-Commit Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> --------- Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Part of #3393.