Skip to content

Model bundles v2 python API #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Mar 28, 2023
Merged

Model bundles v2 python API #89

merged 5 commits into from
Mar 28, 2023

Conversation

phil-scale
Copy link
Contributor

@phil-scale phil-scale commented Mar 28, 2023 •
edited
Loading

7 new methods:

  • create_model_bundle_from_callable_v2
  • create_model_bundle_from_runnable_image_v2
  • create_model_bundle_from_zip_v2
  • get_model_bundle_v2
  • get_latest_model_bundle_v2
  • list_model_bundles_v2
  • clone_model_bundle_with_changes_v2

New documentation currently deployed at https://scaleapi.github.io/launch-python-client/api/client/#launch.client.LaunchClient.create_model_bundle_from_callable_v2

Other than static type-checking, I think the best way to test for now is to add them to integration tests, which is coming in a follow-up PR.

[sc-707019]

@shortcut-integration
Copy link

This pull request has been linked to Shortcut Story #707019: Users can create Model Bundles according to post-LIRA APIs.

@phil-scale phil-scale requested a review from a team March 28, 2023 01:16
@phil-scale phil-scale self-assigned this Mar 28, 2023
@yixu34 yixu34 requested a review from Anirudh-Scale March 28, 2023 18:31
yixu34
yixu34 approved these changes Mar 28, 2023
View reviewed changes
launch/client.py
self,
*,
model_bundle_name: str,
load_predict_fn: Callable[[LaunchModel_T], Callable[[Any], Any]],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Anirudh-Scale You may want to take a look at this PR. I think it's still orthogonal to the Q2-ish cloudpickle API changes you're thinking of, but note that this create_model_bundle_from_callable_v2 is being created. What this means for the rest of the @scaleapi/launch-eng crew - just be aware that we may be deprecating load_predict_fn in favor of something else at some point in the future.

launch/client.py

- ``model_bundle_id``: The ID of the created model bundle.
"""
nonnull_requirements = requirements or []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tell me you're a math major without telling me you're a math major 😁

launch/model_bundle.py

# TODO(yi): These docstrings are currently perfunctory. I'm not sure we even want to expose most of these
# fields. We need to overhaul our types :sadge:


class ModelBundleFrameworkType(str, Enum):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long term, I think the ideal solution would be to have a common package (a la botocore) that has common definitions like this that are shared between server and client? But this is fine for now. cc @scaleapi/train-eng

@phil-scale phil-scale merged commit 467b4eb into launch-v1 Mar 28, 2023
phil-scale added a commit that referenced this pull request Apr 4, 2023
@phil-scale @yixu34
@adlam-scale @Anirudh-Scale @malcolmgreaves
Launch v1 (#66)
518d9db 
* Add autogenerated client into launch v1 (#63)

* add autogenerated client into launch v1

* flake8 conf

* pylint and mypy too

* isort

* Squashed commit of the following:

commit 9533e47
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 23:12:49 2022 -0800

    update deps

commit 14d1f11
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 23:07:38 2022 -0800

    fix pylint check

commit f6c159d
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 23:05:20 2022 -0800

    fix

commit 26825f6
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 22:58:44 2022 -0800

    fix tests

commit f2c0db8
Merge: 443cb0c 7bdf37b
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 14:33:23 2022 -0800

    Merge remote-tracking branch 'origin/master' into phil/launch-v1

commit 443cb0c
Author: Phil Chen <phil.chen@scale.com>
Date:   Wed Dec 14 21:58:33 2022 -0800

    fix

commit 7dee94f
Author: Phil Chen <phil.chen@scale.com>
Date:   Wed Dec 14 18:27:13 2022 -0800

    fixes

commit 44efbf4
Author: Phil Chen <phil.chen@scale.com>
Date:   Fri Dec 2 12:13:55 2022 -0800

    fix

commit 7b74884
Author: Phil Chen <phil.chen@scale.com>
Date:   Fri Dec 2 11:03:45 2022 -0800

    fix

commit 451de71
Author: Phil Chen <phil.chen@scale.com>
Date:   Fri Dec 2 01:05:26 2022 -0800

    basic auth and fixes

commit 607332e
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:47:43 2022 -0800

    update reqs

commit ce11dff
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:40:10 2022 -0800

    black

commit ae4e040
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:38:29 2022 -0800

    fix mypy

commit 58c62ce
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:34:14 2022 -0800

    change to deployment_state

commit a582155
Merge: ff24539 aa0c0b2
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:29:16 2022 -0800

    Merge branch 'launch-v1' into phil/launch-v1

commit aa0c0b2
Author: Phil Chen <92065453+phil-scale@users.noreply.github.com>
Date:   Thu Dec 1 14:28:48 2022 -0800

    Add autogenerated client into launch v1 (#63)

    * add autogenerated client into launch v1

    * flake8 conf

    * pylint and mypy too

    * isort

commit ff24539
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:28:15 2022 -0800

    address comment

commit a53c23d
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 09:25:55 2022 -0800

    black

commit b5d4787
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 09:14:52 2022 -0800

    launch v1 client updates

* fixes

* Use new autogenerated python client for v1 (#67)

* Phil/upload schemas (#68)

* update client

* try again

* fix import

* fixes

* wip

* fixes

* try fix again

* fix

* fixes

* add pydantic schema support

* add test

* fix

* fix flat schemas

* black and isort

* Launch v1 client updates for batch jobs and callbacks (#69)

* add autogen client

* update client to use new API for callbacks and batch jobs

* fix

* Update clone-bundle route (#70)

* Update clone bundle function

* Revert unnecessary change

* pr comments

* mypy

* black

* ✨ Updated v1 CLI ✨  (#71)

* updated v1 python client

* specify encoding

* use utf-8

* add ID to list endpoints and get endpoint

* Use spinner while loading in cli and edit endpoints from CLI (#72)

* clone_bundle_with_changes server route expects new_app_config, not app_config

* update reqs

* use base64 and json to encode batch job payloads (#74)

* fix base64 encoding

* Add CLI command for getting batch job status (#75)

* Use timedelta for printing batch job duration

* fix sync endpoint request

* fix sync endpoint result on error

* Add timeouts to `EndpointResponseFuture` (#77)

* Send requests to launch endpoint (#76)

Adding `send` command, ie: `launch tasks send`

* print more info in launch CLI

* Clean up unused code and use ruff instead of flake8 (#79)

* use ruff instead of flake8

* update lint cfgs

* fix typing

* rm pipeline test

* Add features to `launch endpoints list` (#80)

* add orderby, descending and num_gpus

* reverting changes

* typing_extensions for py37

* remove double print (#82)

* filter bundles and endpoints by name in CLI (#84)

* New documentation site (#81)

* Use mkdocs and write docs for launch

* updates

* add site_url

* add overview of concepts

* fix

* some more fixes

* Update CLI help string to 118 instead of 78 chars wide

* test python code in docs and docstrings

* update team tags to infra

* update literal to typing_extensions

* add some reqs

* fix

* add sleep to callbacks

* updates

* update deps

* address comments

* fix pylint

* Callback custom auth (#85)

* update clients

* Update client to support custom callback auth

* update docs

* update docs with mtls default example

* add copy extension

* Add endpoint to client

* Update docs and external client for priority API change (#87)

* new docs

* format

* format isort

* new changes

* new formatted changes

* formatting

* updated client

* changing from str to bool

* removing site

* making sure?

* update to v1

* Update autogen client with v2 bundles (#88)

* Update autogenerated python client

* fix tests

* Model bundles v2 python API (#89)

* Add model bundles v2 API

* add get list and clone routes too

* use types and make docstrings more informative

* fixes

* fix status

* Documentation for model bundles v2 (#90)

* Add documentation for model bundles

* address comments

* fix

* launch v2 bundles API (#91)

* v2 bundles api copy

* Formated with black

* isort and fix 2 mypy errors

launch/connection.py:14: error: Incompatible default for argument "endpoint" (default has type "None", argument has type "str")  [assignment]
launch/client.py:1811: error: Incompatible default for argument "urls" (default has type "None", argument has type "List[str]")  [assignment]

* Fix package import

* Added readiness_initial_delay_seconds

* updating docs

* Added create TritonEnhancedRunnableImageFlavor bundle

* fix

* fix

* fix

* rm space

* update docs

* fix index docs to new apis

---------

Co-authored-by: Phil Chen <phil.chen@scale.com>

* update mkdocs readme

* use logging lib

---------

Co-authored-by: Yi Xu <yi.xu@scale.com>
Co-authored-by: Adrian Lam <72415932+adlam-scale@users.noreply.github.com>
Co-authored-by: Anirudh Jain <88404749+Anirudh-Scale@users.noreply.github.com>
Co-authored-by: Malcolm Greaves <malcolmgreaves@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yixu34 yixu34 yixu34 approved these changes

@Anirudh-Scale Anirudh-Scale Awaiting requested review from Anirudh-Scale

Assignees

@phil-scale phil-scale

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@phil-scale @yixu34