Support OpenAIModel in ModelRepository #3590
Conversation
|
Tagging the people with most knowledge on openai support: @tessapham @cmaddalozzo @yuzisun |
grandbora
force-pushed
the
openai-patch
branch
4 times, most recently
from
056f8fe
to
da410d1
Compare
Signed-off-by: grandbora <grandbora@fb.com>
|
/assign @jinchihe |
Signed-off-by: grandbora <grandbora@fb.com>
Signed-off-by: grandbora <grandbora@fb.com>
Signed-off-by: grandbora <grandbora@fb.com>
Signed-off-by: grandbora <grandbora@fb.com>
|
Looks like some of the test failures are due to different python version I was using when developing. I'll make sure tests pass in python 3.9. |
Signed-off-by: grandbora <grandbora@fb.com>
Signed-off-by: grandbora <grandbora@fb.com>
Signed-off-by: grandbora <grandbora@fb.com>
Signed-off-by: grandbora <grandbora@fb.com>
|
I tested this branch with a transformer I ran locally. I was able to get a response from the openai legacy completion endpoint. @yuzisun |
|
/lgtm |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: cmaddalozzo, grandbora, yuzisun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
What this PR does / why we need it:
Adds
OpenAIModelsupport toModelRepository.In #3477 we introduced openai endpoints to kserve. These endpoints are registered when there are any models of type
OpenAIModelin theModelRepository. In order to take advantage of the openai endpoints, ModelRepository should supportOpenAIModeltype.Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #
Type of changes
Please delete options that are not relevant.
Feature/Issue validation/testing:
cd python/kserve && make test