Skip to content

Use spinner while loading in cli and edit endpoints from CLI #72

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Feb 7, 2023
Merged

Use spinner while loading in cli and edit endpoints from CLI #72

merged 3 commits into from
Feb 7, 2023

Conversation

phil-scale
Copy link
Contributor

@phil-scale phil-scale commented Feb 6, 2023 •
edited
Loading

30-minute update to make Launch CLI look nice and to allow users to edit endpoints via CLI

❯ launch endpoints list
Endpoints
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Endpoint ID              ┃ Endpoint name   ┃ Bundle name   ┃ Status ┃ Endpoint type ┃ Min Workers ┃ Max Workers ┃ Available Workers ┃ Unavailable Workers ┃ Metadata ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ end_cfgmv0253f3g03cp5keg │ demo-endpoint-a │ test-bundle-a │ READY  │ async         │ 0           │ 1           │ 0                 │ 0                   │ {}       │
└──────────────────────────┴─────────────────┴───────────────┴────────┴───────────────┴─────────────┴─────────────┴───────────────────┴─────────────────────┴──────────┘
❯ launch endpoints edit demo-endpoint-a
? Model bundle:  Current bundle (test-bundle-a)
? Cpus:  1
? Gpus:  0
? Memory:  8Gi
? Storage (optional):
? Gpu type (optional):
? Min workers:  0
? Max workers:  1
? Per worker:  10
? Post-inference hooks:  done
? Default callback url (optional):

@phil-scale phil-scale requested a review from a team February 6, 2023 21:24
@phil-scale phil-scale changed the title use spinner while loading in cli Use spinner while loading in cli and edit endpoints from CLI Feb 6, 2023
seanshi-scale
seanshi-scale reviewed Feb 6, 2023
View reviewed changes
Copy link
Contributor

@seanshi-scale seanshi-scale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait are the cli edit endpoint parts pushed?

@phil-scale
Copy link
Contributor Author

wait are the cli edit endpoint parts pushed?

oops thanks for the catch

seanshi-scale
seanshi-scale approved these changes Feb 6, 2023
View reviewed changes
launch/cli/endpoints.py
)

with spinner(f"Editing endpoint '{endpoint_name}'"):
client.edit_model_endpoint(model_endpoint=model_endpoint, **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want(/need) to print out a nice error message if the user passes in parameters that end up failing server-side validation? (or does the error message seem good enough)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that would be a good TODO - error messages are hard 😢

@phil-scale phil-scale merged commit ecbd7b0 into launch-v1 Feb 7, 2023
phil-scale added a commit that referenced this pull request Apr 4, 2023
@phil-scale @yixu34
@adlam-scale @Anirudh-Scale @malcolmgreaves
Launch v1 (#66)
518d9db 
* Add autogenerated client into launch v1 (#63)

* add autogenerated client into launch v1

* flake8 conf

* pylint and mypy too

* isort

* Squashed commit of the following:

commit 9533e47
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 23:12:49 2022 -0800

    update deps

commit 14d1f11
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 23:07:38 2022 -0800

    fix pylint check

commit f6c159d
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 23:05:20 2022 -0800

    fix

commit 26825f6
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 22:58:44 2022 -0800

    fix tests

commit f2c0db8
Merge: 443cb0c 7bdf37b
Author: phil-scale <phil.chen@scale.com>
Date:   Mon Dec 19 14:33:23 2022 -0800

    Merge remote-tracking branch 'origin/master' into phil/launch-v1

commit 443cb0c
Author: Phil Chen <phil.chen@scale.com>
Date:   Wed Dec 14 21:58:33 2022 -0800

    fix

commit 7dee94f
Author: Phil Chen <phil.chen@scale.com>
Date:   Wed Dec 14 18:27:13 2022 -0800

    fixes

commit 44efbf4
Author: Phil Chen <phil.chen@scale.com>
Date:   Fri Dec 2 12:13:55 2022 -0800

    fix

commit 7b74884
Author: Phil Chen <phil.chen@scale.com>
Date:   Fri Dec 2 11:03:45 2022 -0800

    fix

commit 451de71
Author: Phil Chen <phil.chen@scale.com>
Date:   Fri Dec 2 01:05:26 2022 -0800

    basic auth and fixes

commit 607332e
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:47:43 2022 -0800

    update reqs

commit ce11dff
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:40:10 2022 -0800

    black

commit ae4e040
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:38:29 2022 -0800

    fix mypy

commit 58c62ce
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:34:14 2022 -0800

    change to deployment_state

commit a582155
Merge: ff24539 aa0c0b2
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:29:16 2022 -0800

    Merge branch 'launch-v1' into phil/launch-v1

commit aa0c0b2
Author: Phil Chen <92065453+phil-scale@users.noreply.github.com>
Date:   Thu Dec 1 14:28:48 2022 -0800

    Add autogenerated client into launch v1 (#63)

    * add autogenerated client into launch v1

    * flake8 conf

    * pylint and mypy too

    * isort

commit ff24539
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 14:28:15 2022 -0800

    address comment

commit a53c23d
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 09:25:55 2022 -0800

    black

commit b5d4787
Author: Phil Chen <phil.chen@scale.com>
Date:   Thu Dec 1 09:14:52 2022 -0800

    launch v1 client updates

* fixes

* Use new autogenerated python client for v1 (#67)

* Phil/upload schemas (#68)

* update client

* try again

* fix import

* fixes

* wip

* fixes

* try fix again

* fix

* fixes

* add pydantic schema support

* add test

* fix

* fix flat schemas

* black and isort

* Launch v1 client updates for batch jobs and callbacks (#69)

* add autogen client

* update client to use new API for callbacks and batch jobs

* fix

* Update clone-bundle route (#70)

* Update clone bundle function

* Revert unnecessary change

* pr comments

* mypy

* black

* ✨ Updated v1 CLI ✨  (#71)

* updated v1 python client

* specify encoding

* use utf-8

* add ID to list endpoints and get endpoint

* Use spinner while loading in cli and edit endpoints from CLI (#72)

* clone_bundle_with_changes server route expects new_app_config, not app_config

* update reqs

* use base64 and json to encode batch job payloads (#74)

* fix base64 encoding

* Add CLI command for getting batch job status (#75)

* Use timedelta for printing batch job duration

* fix sync endpoint request

* fix sync endpoint result on error

* Add timeouts to `EndpointResponseFuture` (#77)

* Send requests to launch endpoint (#76)

Adding `send` command, ie: `launch tasks send`

* print more info in launch CLI

* Clean up unused code and use ruff instead of flake8 (#79)

* use ruff instead of flake8

* update lint cfgs

* fix typing

* rm pipeline test

* Add features to `launch endpoints list` (#80)

* add orderby, descending and num_gpus

* reverting changes

* typing_extensions for py37

* remove double print (#82)

* filter bundles and endpoints by name in CLI (#84)

* New documentation site (#81)

* Use mkdocs and write docs for launch

* updates

* add site_url

* add overview of concepts

* fix

* some more fixes

* Update CLI help string to 118 instead of 78 chars wide

* test python code in docs and docstrings

* update team tags to infra

* update literal to typing_extensions

* add some reqs

* fix

* add sleep to callbacks

* updates

* update deps

* address comments

* fix pylint

* Callback custom auth (#85)

* update clients

* Update client to support custom callback auth

* update docs

* update docs with mtls default example

* add copy extension

* Add endpoint to client

* Update docs and external client for priority API change (#87)

* new docs

* format

* format isort

* new changes

* new formatted changes

* formatting

* updated client

* changing from str to bool

* removing site

* making sure?

* update to v1

* Update autogen client with v2 bundles (#88)

* Update autogenerated python client

* fix tests

* Model bundles v2 python API (#89)

* Add model bundles v2 API

* add get list and clone routes too

* use types and make docstrings more informative

* fixes

* fix status

* Documentation for model bundles v2 (#90)

* Add documentation for model bundles

* address comments

* fix

* launch v2 bundles API (#91)

* v2 bundles api copy

* Formated with black

* isort and fix 2 mypy errors

launch/connection.py:14: error: Incompatible default for argument "endpoint" (default has type "None", argument has type "str")  [assignment]
launch/client.py:1811: error: Incompatible default for argument "urls" (default has type "None", argument has type "List[str]")  [assignment]

* Fix package import

* Added readiness_initial_delay_seconds

* updating docs

* Added create TritonEnhancedRunnableImageFlavor bundle

* fix

* fix

* fix

* rm space

* update docs

* fix index docs to new apis

---------

Co-authored-by: Phil Chen <phil.chen@scale.com>

* update mkdocs readme

* use logging lib

---------

Co-authored-by: Yi Xu <yi.xu@scale.com>
Co-authored-by: Adrian Lam <72415932+adlam-scale@users.noreply.github.com>
Co-authored-by: Anirudh Jain <88404749+Anirudh-Scale@users.noreply.github.com>
Co-authored-by: Malcolm Greaves <malcolmgreaves@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seanshi-scale seanshi-scale seanshi-scale approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@phil-scale @seanshi-scale