-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Examples for online inference for LLaVA #2671
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bhimar
reviewed
Sep 26, 2023
cli/foundation-models/system/inference/image-text-to-text/image-text-to-text-online-endpoint.sh
Outdated
Show resolved
Hide resolved
bhimar
reviewed
Sep 26, 2023
...undation-models/system/inference/image-text-to-text/image-text-to-text-online-endpoint.ipynb
Outdated
Show resolved
Hide resolved
bhimar
approved these changes
Sep 26, 2023
...undation-models/system/inference/image-text-to-text/image-text-to-text-online-endpoint.ipynb
Outdated
Show resolved
Hide resolved
...undation-models/system/inference/image-text-to-text/image-text-to-text-online-endpoint.ipynb
Outdated
Show resolved
Hide resolved
PhaniShekhar
approved these changes
Sep 26, 2023
PhaniShekhar
approved these changes
Sep 27, 2023
rdondera-microsoft
added a commit
that referenced
this pull request
Sep 27, 2023
This reverts commit 3ce6d0a.
fredms
pushed a commit
that referenced
this pull request
Sep 27, 2023
* Rough version of ITT notebook. * Working version of the notebook. * Minor improvements. * Rough version of the CLI demo. * Reformat with black. * Remove mode parameter in call to prepare_data.py. * Make code cells easier to read. * Minor readability improvement. * Format with black. * Minor fixes and reformatting. * Reformat with black.
fredms
pushed a commit
that referenced
this pull request
Sep 27, 2023
* Rough version of ITT notebook. * Working version of the notebook. * Minor improvements. * Rough version of the CLI demo. * Reformat with black. * Remove mode parameter in call to prepare_data.py. * Make code cells easier to read. * Minor readability improvement. * Format with black. * Minor fixes and reformatting. * Reformat with black.
rdondera-microsoft
added a commit
that referenced
this pull request
Sep 27, 2023
qjxu
pushed a commit
that referenced
this pull request
Sep 27, 2023
* Add automation test for featurestore cli notebooks * Sample updates for AML Spark managed VNet GA. (#2656) * Sample updates for AML Spark managed VNet GA. * Updated notebook title. Triggering build to test. * Updated workspace name to include timestamp. Triggering build to test. * Shortened VNet workspace name to meet validation. * README.md clean-up and fixes for AML Spark samples. * README.md clean-up and fixes for AML Spark samples. * chore: Remove 'Send IcM on Failure' step from tutorial workflows (#2670) * chore: Remove 'Send IcM on Failure' step that hasn't been in use for a while * chore: Run tutorials/readme.py * chore: Remove unused local 'generate-icm' action * update cron schedules to avoid endpoint deletion from automated resource cleanup (#2672) * change starting hour for workflow * update cron for text mc mlflow * use python 3.8.* (#2678) * use python 3.8.18 * Update cleanup.yml * update usepython v4 * Update workstation_env.yml (#2680) * Examples for online inference for LLaVA (#2671) * Rough version of ITT notebook. * Working version of the notebook. * Minor improvements. * Rough version of the CLI demo. * Reformat with black. * Remove mode parameter in call to prepare_data.py. * Make code cells easier to read. * Minor readability improvement. * Format with black. * Minor fixes and reformatting. * Reformat with black. * Update workstation_env.yml (#2681) delete extraneous space * update vision FT examples corresponding to recent component update (#2650) * updating changes with ME component * updating MMOD as well * update MMOD cli * separate section for model evaluation component * remove ds ort true for classification * black check * Remove v1 references (#2675) * Remove v1 samples from v2 repo * Remove v1 from table of contents * Remove v1 test files * Remove v1 test files * Remove v1 folder from README * Remove v1 folder * Remove v1 tests * Delete v1 files * Restore custom-container directory * Restore custom-container/triton directory --------- Co-authored-by: Yogi Pandey <20666257+ynpandey@users.noreply.github.com> Co-authored-by: kdestin <101366538+kdestin@users.noreply.github.com> Co-authored-by: Rehaan Bhimani <rbhimani@microsoft.com> Co-authored-by: Ada <61294872+adrosa@users.noreply.github.com> Co-authored-by: Sheri Gilley <sgilley@microsoft.com> Co-authored-by: rdondera-microsoft <98922913+rdondera-microsoft@users.noreply.github.com> Co-authored-by: shubhamiit <41925087+shubhamiit@users.noreply.github.com> Co-authored-by: Diondra <16376603+diondrapeck@users.noreply.github.com>
fredms
added a commit
that referenced
this pull request
Sep 27, 2023
* Enable automation test for all feature store notebooks * Update feature store uai * Update uai name * Add test for 5th notebook * Convert 5th notebook to py * Fix the syntax error * Update random generator * Update redis name * Change conda file used for test * Update to use dynamic version * Fix the format * Use the same conda for all notebooks * Check mounted folder * Exclude 5th notebook * Update feature_version to version * Sample updates for AML Spark managed VNet GA. (#2656) * Sample updates for AML Spark managed VNet GA. * Updated notebook title. Triggering build to test. * Updated workspace name to include timestamp. Triggering build to test. * Shortened VNet workspace name to meet validation. * README.md clean-up and fixes for AML Spark samples. * README.md clean-up and fixes for AML Spark samples. * chore: Remove 'Send IcM on Failure' step from tutorial workflows (#2670) * chore: Remove 'Send IcM on Failure' step that hasn't been in use for a while * chore: Run tutorials/readme.py * chore: Remove unused local 'generate-icm' action * update cron schedules to avoid endpoint deletion from automated resource cleanup (#2672) * change starting hour for workflow * update cron for text mc mlflow * use python 3.8.* (#2678) * use python 3.8.18 * Update cleanup.yml * update usepython v4 * Update workstation_env.yml (#2680) * Examples for online inference for LLaVA (#2671) * Rough version of ITT notebook. * Working version of the notebook. * Minor improvements. * Rough version of the CLI demo. * Reformat with black. * Remove mode parameter in call to prepare_data.py. * Make code cells easier to read. * Minor readability improvement. * Format with black. * Minor fixes and reformatting. * Reformat with black. * Update workstation_env.yml (#2681) delete extraneous space * update vision FT examples corresponding to recent component update (#2650) * updating changes with ME component * updating MMOD as well * update MMOD cli * separate section for model evaluation component * remove ds ort true for classification * black check * Remove v1 references (#2675) * Remove v1 samples from v2 repo * Remove v1 from table of contents * Remove v1 test files * Remove v1 test files * Remove v1 folder from README * Remove v1 folder * Remove v1 tests * Delete v1 files * Restore custom-container directory * Restore custom-container/triton directory * Update the notebook format --------- Co-authored-by: Yogi Pandey <20666257+ynpandey@users.noreply.github.com> Co-authored-by: kdestin <101366538+kdestin@users.noreply.github.com> Co-authored-by: Rehaan Bhimani <rbhimani@microsoft.com> Co-authored-by: Ada <61294872+adrosa@users.noreply.github.com> Co-authored-by: Sheri Gilley <sgilley@microsoft.com> Co-authored-by: rdondera-microsoft <98922913+rdondera-microsoft@users.noreply.github.com> Co-authored-by: shubhamiit <41925087+shubhamiit@users.noreply.github.com> Co-authored-by: Diondra <16376603+diondrapeck@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Example notebook and CLI configuration files for running inference with LLaVA model (version 7B). Both examples delete the endpoint as the final step, but for reviewing purposes endpoints were kept at
https://ml.azure.com/endpoints/realtime/image-text-to-text-1695662461/detail?wsid=/subscriptions/dbd697c3-ef40-488f-83e6-5ad4dfb78f9b/resourcegroups/rdondera/providers/Microsoft.MachineLearningServices/workspaces/llava&tid=72f988bf-86f1-41af-91ab-2d7cd011db47 (CLI)
and
https://ml.azure.com/endpoints/realtime/image-text-to-text-1695663850/detail?wsid=/subscriptions/dbd697c3-ef40-488f-83e6-5ad4dfb78f9b/resourcegroups/rdondera/providers/Microsoft.MachineLearningServices/workspaces/llava&tid=72f988bf-86f1-41af-91ab-2d7cd011db47 (notebook)
The model used was published to an internal registry using the model publish pipeline.
Checklist