Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Follow up work for ServingRuntime support #1924

Closed
yuzisun opened this issue Nov 24, 2021 · 1 comment · Fixed by #1948, #1926 or #1943
Closed

Follow up work for ServingRuntime support #1924

yuzisun opened this issue Nov 24, 2021 · 1 comment · Fixed by #1948, #1926 or #1943
Assignees
@Suresh-Nakkeran
Labels
kfserving/control-plane kind/feature
Projects
KServe 0.8

Comments

@yuzisun
Copy link
Member

yuzisun commented Nov 24, 2021

/kind feature

Describe the solution you'd like

  • Add template variable support for enabling passing in isvc name to runtime container.
  • Add more default ClusterServingRuntimes
  • Adjust the InferenceService mutator to convert old schema into the new schema.
  • Add ability to disable a Framework/ModelType, in a ServingRuntime and add associated logic?
  • Add E2E tests using new format
  • Eventually add default ClusterServingRuntimes as a part of the default installation (perhaps generate separate yaml file for
    default runtimes as including the runtimes in the main default install causes that race condition with the CRD not existing yet).

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
@yuzisun
Copy link
Member Author

yuzisun commented Nov 24, 2021

/cc @pvaneck @Suresh-Nakkeran @andyi2it

@yuzisun yuzisun mentioned this issue Nov 25, 2021
Merged
@yuzisun yuzisun added this to To do in KServe 0.8 via automation Nov 27, 2021
@yuzisun yuzisun moved this from To do to In progress in KServe 0.8 Nov 27, 2021
kserve-oss-bot pushed a commit that referenced this issue Dec 11, 2021
@Suresh-Nakkeran @yuzisun
Follow up work on ServingRuntime support - PR1 #1924 (#1926)
db4d528 
* added built-in cluster servingruntime for existing framework

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* replace placeholders in runtime container

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* convert isvc old schema to new schema

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* added triton as supported framework for backward compatibility

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* assigning predictor spec issue fix

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* incorporated review comments on serving runtimes

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* predictor model name issue fix

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* update image tag and protocol version

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* isvc update issue fix

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* fix e2e test failure issues

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* - remove gpu clusterservingruntimes
 - update image tag if gpu enabled or runtime version provided
 - update mlserver, tensorflow image versions

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* added ut for new utils funcs

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* updated model supported versions

Signed-off-by: Suresh Nakkeran <suresh.n@ideas2it.com>

* Fix protocol version

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

Co-authored-by: Dan Sun <dsun20@bloomberg.net>
@yuzisun yuzisun moved this from In progress to Done in KServe 0.8 Dec 12, 2021
This was referenced Dec 21, 2021
Merged
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Suresh-Nakkeran Suresh-Nakkeran

Labels
kfserving/control-plane kind/feature
Projects
No open projects
KServe 0.8
Done
Milestone
No milestone
3 participants
@yuzisun @Suresh-Nakkeran @kserve-oss-bot