feat(ray): implement model local run and adopt latest task spec #196
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
heiruwu
force-pushed
the
heiru/INS-5462
branch
from
0f395ae to
f83e110
Compare
heiruwu
force-pushed
the
heiru/INS-5462
branch
from
af52c4a to
ded09da
Compare
Because - We are going to remove data serialization between `model-backend` and `ray-server` to improve the DX for implementing a custom model - To support local model image self-test, ray application must be able to accept payload in JSON object for ease of use This commit - adopt latest contract for ray application - redesign and rewrite IO help functions
Because - Our protogen `model` module import name will collide with local `model.py` This commit - avoid module name collision by temporarily remove cwd path from `sys.path`
Because - Often time `instill-sdk` package is updated while other dependencies version stay the same This commit - separate sdk installation into another layer to make the build process more effiecient
Because - executable path is not always `""` This commit - rename conflicting `model.py` file name inside container
Because - The generated field names for protobuf message does not aligned between golang and python - Task protobuf definitions are going to be retired This commit - use `MessageToDict` instead to parse task inputs
Because - update ray proto to use `__call__` as default trigger entry to allow http request - wrong visioninput `image` type This commit - update proto and fix wrong vision input type - fix VisionInput image type
Because - python cli needs to provide model image self-test function This commit - add `run` command to provide local self-test
Because
- currently, model image can only install instill-sdk from pypi service,
which makes changes must be merged into main branch
->
- with these changes, we can make dist to build local dev wheel and
instill build --sdk-wheel {path-to-wheel} to force model image to use
dev version sdk, which makes the dev process much easier
This commit
- support local sdk wheel for building model image
heiruwu
force-pushed
the
heiru/INS-5462
branch
from
9490d6a to
517ba48
Compare
joremysh
pushed a commit
that referenced
this pull request
Sep 12, 2024
🤖 I have created a release *beep* *boop* --- ## [0.12.0](v0.11.0...v0.12.0) (2024-09-11) ### Features * **artifact:** update get_file_catalog input parameter default values ([#203](#203)) ([f5da73f](f5da73f)) * **ray:** implement model local run and adopt latest task spec ([#196](#196)) ([5b67b56](5b67b56)) * **ray:** support multimodal embedding input ([#204](#204)) ([e150ad1](e150ad1)) ### Bug Fixes * **dockerfile:** revert storing cache dir ([#207](#207)) ([297f1f1](297f1f1)) * **ray:** append mime type for image output ([#208](#208)) ([9e38b70](9e38b70)) * **ray:** fix bounding box output type ([#205](#205)) ([22242df](22242df)) * **ray:** fix multimodal chat input ([#210](#210)) ([37aa13a](37aa13a)) * **ray:** fix number of sample type ([#206](#206)) ([c87527d](c87527d)) * **ray:** replace user-agent header ([#209](#209)) ([e18b217](e18b217)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because
This commit
resolves INS-5462
resolves INS-5458
resolves INS-5215
resolves INS-5456
resolves INS-5463
resolves INS-5464