Skip to content

Conversation

@chuyang-deng
Copy link
Contributor

@chuyang-deng chuyang-deng commented Jul 28, 2020

Issue #, if available:
#1464

Description of changes:
Use image_uris.retrieve() for ModelMonitor default image uri

Testing done:
unit tests

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

  • I have read the CONTRIBUTING doc
  • I used the commit message format described in CONTRIBUTING
  • I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
  • I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have checked that my tests are not configured for a specific region or account (if appropriate)
  • I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: 021937b
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-notebook-tests
  • Commit ID: 021937b
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: 021937b
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

logger = logging.getLogger(__name__)

ECR_URI_TEMPLATE = "{registry}.dkr.{hostname}/{repository}:{tag}"
MONITOR_URI_TEMPLATE = "{registry}.dkr.{hostname}/{repository}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer the code in this file be as framework/algorithm-agnostic as possible. so let's remove the tag part from the template, and then figure out how to structure the JSON instead?

this is a bit tough since I hadn't even considered that there might be URIs without tags when I started this...you could use "" for the version? or add some logic to handle the case where "versions" isn't a valid key.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the model-monitor.json file as well as the retrieve() logic to let model monitor use the generic template.

We could update the "versioins" later when model-monitor images with tags are released.

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: 6b0b83f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-notebook-tests
  • Commit ID: 6b0b83f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: 6b0b83f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

"versions": {
"": {
"registries": {
"eu-north-1": "895015795356",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpicky nitpick: alphabetize the regions

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-zwei-pr
  • Commit ID: 6b0b83f
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: 7d72166
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-notebook-tests
  • Commit ID: 7d72166
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: 7d72166
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-zwei-pr
  • Commit ID: 7d72166
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: 954c956
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-notebook-tests
  • Commit ID: 954c956
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-zwei-pr
  • Commit ID: 954c956
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: 954c956
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@laurenyu laurenyu merged commit 4632611 into aws:zwei Jul 29, 2020
zhaoqizqwang added a commit that referenced this pull request Nov 20, 2025
* Trinity (#1765)

* Test commit of new ModelBulider with V2 dependencies eliminated

* ModelBuilder.build() working for tgi server

* Support for torchserve, tgi, djl, triton, tei, mms, and smd

* Build fixed for jumpstart, auto-container detection logic fixed

* Model Builder cleaned and added more unit tests

* Deploy happy-case for Triton and JumpStart (TGI)

* Better auto-container detection and user experience

* Remove debug print statements

* Removing bad unit tests

* Style and cleanup changes

* Removing unrelated unit tests for now (will migrate in the future to V3)

* Fix installation and Sentinel issue

* Attempt to fix installation issues #2

* Sentinel bug fix third attempt, support for tgi server

* PR Check installation issue fix attempt number four

* Resolving missing dependency issues

* Update dependencies for sagemaker_train

* serve folder flattening, support for local mode

* Fully supporting local mode and in process mode with integ tests

* ModelBuilder and Pipeline merged

* Better deploy logic and fixed bad imports

* Trinity updated (#1769)

* Test commit of new ModelBulider with V2 dependencies eliminated

* ModelBuilder.build() working for tgi server

* Support for torchserve, tgi, djl, triton, tei, mms, and smd

* Build fixed for jumpstart, auto-container detection logic fixed

* Model Builder cleaned and added more unit tests

* Deploy happy-case for Triton and JumpStart (TGI)

* Better auto-container detection and user experience

* Remove debug print statements

* Removing bad unit tests

* Style and cleanup changes

* Removing unrelated unit tests for now (will migrate in the future to V3)

* Fix installation and Sentinel issue

* Attempt to fix installation issues #2

* Sentinel bug fix third attempt, support for tgi server

* PR Check installation issue fix attempt number four

* Resolving missing dependency issues

* Update dependencies for sagemaker_train

* serve folder flattening, support for local mode

* Fully supporting local mode and in process mode with integ tests

* ModelBuilder and Pipeline merged

* Better deploy logic and fixed bad imports

* Fixes for build() and deploy(), ModelTrainer support

---------

Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>

* Update processor and model trainer (#1772)

* Test pipeline v3 (#1773)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Resolve conflict (#1774)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix ProcessingJob pipeline integration (#1778)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables (#1779)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Test pipeline v3 (#1781)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer.train() request (#1782)


* Fix ModelTrainer request

* Clean up code

* Cleaned up files and removed unneeded files (#1780)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Fix pipeline example notebooks

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer request

* Clean up code

* Fix v3 pipeline examples

* Trinity updated (#1787)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* remote function, model card (#1786)

* Add model card to v3

Add unit tests passed. Compared the new classes with V2 classes

* fixed circular dep issue for modelcard testing

* remote function working preoperly now

---------

Co-authored-by: jzhaoqwa <52220743+zhaoqizqwang@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Fixed optimize import error (#1791)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring (#1792)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Add model registry to v3 (#1794)

* Update model registry and examples

* tuner fix (#1796)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Add transformation job and example (#1797)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer request

* Clean up code

* Fix v3 pipeline examples

* Update model registry and examples

* Add transformation job example

* Trinity update for model monitor (#1789)

* model monitoring test and fix

* model monitor changes and notebook (needs final run through)

* remove unused notebook

* add actual notebook

* file dir change

* clarify added and model monitor fixed

* Trinity hyperparametertuner (#1798)

* tuner fix

* example for hyperparameter tuning

* rmvd unneeded files

* added back

---------

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Add pipeline_models key to ModelBuilder to support building model with multiple containers (#1799)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer request

* Clean up code

* Fix v3 pipeline examples

* Update model registry and examples

* Add transformation job example

* Add feature to ModelBuild for building model with multiple containers (pipeline_models)

* fix code

* Continue fix pipeline model

* Model monitor in pipeline example updated, experiment config example added (#1801)

* model monitoring test and fix

* model monitor changes and notebook (needs final run through)

* remove unused notebook

* add actual notebook

* file dir change

* clarify added and model monitor fixed

* model monitor in pipeline example updated, experiment config example added

* Support TuningStep in Pipeline (#1803)


* Support TuningStep in pipeline

* enables repack_model with js compatability (#1806)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Fix model monitor example notebook, add experiment module example (#1809)

* model monitoring test and fix

* model monitor changes and notebook (needs final run through)

* remove unused notebook

* add actual notebook

* file dir change

* clarify added and model monitor fixed

* model monitor in pipeline example updated, experiment config example added

* fix model monitor example notebook, add experiment module

* Add FrameworkProcessor to support Pytorch preprocessing (#1810)


* Add Framework Processor to support Pytorch models

* step decorator fix (#1812)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Fix import error and eemove unused examples (#1814)


* Fix import error and eemove unused examples

* Support for FrameWork Models (#1808)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Support for XGBoost, SKLearn, and other framework models

* Update ML Ops examples (#1815)

* Update ML Ops examples

* Update examples

* update transform job pipeline notebook (#1817)

* extra model card changes

* local container fix (#1820)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* SA Bugbash fixes (#1822)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Support for XGBoost, SKLearn, and other framework models

* WIP: Updates to model builder and serve components

* WIP: Latest V3 ModelBuilder updates before upstream merge

* Test pipeline v3 (#1824)

* Update ML Ops examples

* Update examples

* Remove unused files

* clean up files

* Move over jumpstart utils

* feat: add jumpstart model_builder telemetry (#1784)

* add jumpstart model_builder telemetry

* chore: use is_jumpstart_model_id to detect JS use case

* feat: add tagging for JS model builder created from trainer

---------

Co-authored-by: aviruthen <91846056+aviruthen@users.noreply.github.com>

* Bug fixes for local mode and tensorflow-serving (#1829)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Support for XGBoost, SKLearn, and other framework models

* WIP: Updates to model builder and serve components

* WIP: Latest V3 ModelBuilder updates before upstream merge

* Support for local mode on all model servers, support for tensorflow-serving

* Updated Autogen SageMaker-Core

* Bug Fixes for JumpStart, ModelTrainer ModelBuilder flow

* Bug fixes, code cleanup, and adding integration tests

* Migrating utils (#1832)

* fixed integ test imports

* fixed pyproj

* toml file fix

* fixed import paths and sm-train

* import update in example notebook

* modelbuilder

* modelbuilder utils reference deleted

* Import bug fixes for jumpstart

* fixes

* pyproj fix

* resource requirement fix

* Bug fixes for jumpstart and training

* model builder fix

* model builder merge

* notebooks fix and modeltrainer

* notebook role fix

---------

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>
Co-authored-by: aviruthen <91846056+aviruthen@users.noreply.github.com>

* Update MLOps integ tests and ModeBuilder model key(#1839)

* Add v3 inference example notebooks (#1840)

* Cleaned up example notebooks (#1841)

* Removed import warnings and organized example notebooks

* Updated folder name

* Removed legacy folder (#1843)

* Removed import warnings and organized example notebooks

* Updated folder name

* Removed legacy folder

* Remove unused files


* Remove files that depend on predictor classes

* Update Training tox.ini for V3 release

* Updating tox.ini and codebuild yaml for PR checks (#1850)

* Trinity (#1765)

* Test commit of new ModelBulider with V2 dependencies eliminated

* ModelBuilder.build() working for tgi server

* Support for torchserve, tgi, djl, triton, tei, mms, and smd

* Build fixed for jumpstart, auto-container detection logic fixed

* Model Builder cleaned and added more unit tests

* Deploy happy-case for Triton and JumpStart (TGI)

* Better auto-container detection and user experience

* Remove debug print statements

* Removing bad unit tests

* Style and cleanup changes

* Removing unrelated unit tests for now (will migrate in the future to V3)

* Fix installation and Sentinel issue

* Attempt to fix installation issues #2

* Sentinel bug fix third attempt, support for tgi server

* PR Check installation issue fix attempt number four

* Resolving missing dependency issues

* Update dependencies for sagemaker_train

* serve folder flattening, support for local mode

* Fully supporting local mode and in process mode with integ tests

* ModelBuilder and Pipeline merged

* Better deploy logic and fixed bad imports

* Trinity updated (#1769)

* Test commit of new ModelBulider with V2 dependencies eliminated

* ModelBuilder.build() working for tgi server

* Support for torchserve, tgi, djl, triton, tei, mms, and smd

* Build fixed for jumpstart, auto-container detection logic fixed

* Model Builder cleaned and added more unit tests

* Deploy happy-case for Triton and JumpStart (TGI)

* Better auto-container detection and user experience

* Remove debug print statements

* Removing bad unit tests

* Style and cleanup changes

* Removing unrelated unit tests for now (will migrate in the future to V3)

* Fix installation and Sentinel issue

* Attempt to fix installation issues #2

* Sentinel bug fix third attempt, support for tgi server

* PR Check installation issue fix attempt number four

* Resolving missing dependency issues

* Update dependencies for sagemaker_train

* serve folder flattening, support for local mode

* Fully supporting local mode and in process mode with integ tests

* ModelBuilder and Pipeline merged

* Better deploy logic and fixed bad imports

* Fixes for build() and deploy(), ModelTrainer support

---------

Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>

* Update processor and model trainer (#1772)

* Test pipeline v3 (#1773)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Resolve conflict (#1774)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix ProcessingJob pipeline integration (#1778)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables (#1779)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Test pipeline v3 (#1781)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer.train() request (#1782)


* Fix ModelTrainer request

* Clean up code

* Cleaned up files and removed unneeded files (#1780)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Fix pipeline example notebooks

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer request

* Clean up code

* Fix v3 pipeline examples

* Trinity updated (#1787)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* remote function, model card (#1786)

* Add model card to v3

Add unit tests passed. Compared the new classes with V2 classes

* fixed circular dep issue for modelcard testing

* remote function working preoperly now

---------

Co-authored-by: jzhaoqwa <52220743+zhaoqizqwang@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Fixed optimize import error (#1791)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring (#1792)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Add model registry to v3 (#1794)

* Update model registry and examples

* tuner fix (#1796)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Add transformation job and example (#1797)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer request

* Clean up code

* Fix v3 pipeline examples

* Update model registry and examples

* Add transformation job example

* Trinity update for model monitor (#1789)

* model monitoring test and fix

* model monitor changes and notebook (needs final run through)

* remove unused notebook

* add actual notebook

* file dir change

* clarify added and model monitor fixed

* Trinity hyperparametertuner (#1798)

* tuner fix

* example for hyperparameter tuning

* rmvd unneeded files

* added back

---------

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Add pipeline_models key to ModelBuilder to support building model with multiple containers (#1799)

* Update processor and model trainer

* Add Pipeline Variable class to sagemaker core

* Update PipelineVariable usage and imports

* Fix processing.py for pipeline integration

* Remove logs

* More fixes on ModelTrainer's compatibility with PipelineVariables

* Fix PipelineVariables used in ModelBuilder

* Fix ModelTrainer request

* Clean up code

* Fix v3 pipeline examples

* Update model registry and examples

* Add transformation job example

* Add feature to ModelBuild for building model with multiple containers (pipeline_models)

* fix code

* Continue fix pipeline model

* Model monitor in pipeline example updated, experiment config example added (#1801)

* model monitoring test and fix

* model monitor changes and notebook (needs final run through)

* remove unused notebook

* add actual notebook

* file dir change

* clarify added and model monitor fixed

* model monitor in pipeline example updated, experiment config example added

* Support TuningStep in Pipeline (#1803)


* Support TuningStep in pipeline

* enables repack_model with js compatability (#1806)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Fix model monitor example notebook, add experiment module example (#1809)

* model monitoring test and fix

* model monitor changes and notebook (needs final run through)

* remove unused notebook

* add actual notebook

* file dir change

* clarify added and model monitor fixed

* model monitor in pipeline example updated, experiment config example added

* fix model monitor example notebook, add experiment module

* Add FrameworkProcessor to support Pytorch preprocessing (#1810)


* Add Framework Processor to support Pytorch models

* step decorator fix (#1812)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* Fix import error and eemove unused examples (#1814)


* Fix import error and eemove unused examples

* Support for FrameWork Models (#1808)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Support for XGBoost, SKLearn, and other framework models

* Update ML Ops examples (#1815)

* Update ML Ops examples

* Update examples

* update transform job pipeline notebook (#1817)

* extra model card changes

* local container fix (#1820)

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

* SA Bugbash fixes (#1822)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Support for XGBoost, SKLearn, and other framework models

* WIP: Updates to model builder and serve components

* WIP: Latest V3 ModelBuilder updates before upstream merge

* Test pipeline v3 (#1824)

* Update ML Ops examples

* Update examples

* Remove unused files

* clean up files

* Move over jumpstart utils

* feat: add jumpstart model_builder telemetry (#1784)

* add jumpstart model_builder telemetry

* chore: use is_jumpstart_model_id to detect JS use case

* feat: add tagging for JS model builder created from trainer

---------

Co-authored-by: aviruthen <91846056+aviruthen@users.noreply.github.com>

* Bug fixes for local mode and tensorflow-serving (#1829)

* Minor bug fixes for ModelBuilder experience

* Cleaned up files and removed unneeded files

* Small fix to pass kwargs through _deploy()

* Bug fixes for xgboost, sklearn, tgi, and tei

* Optimize import fix

* Removing llm_utils dependency

* Bugbash fixes, including region support, role_arn optional, and progress monitoring

* Keep CUDA warning and don't hardcode fallback region

* Support for XGBoost, SKLearn, and other framework models

* WIP: Updates to model builder and serve components

* WIP: Latest V3 ModelBuilder updates before upstream merge

* Support for local mode on all model servers, support for tensorflow-serving

* Updated Autogen SageMaker-Core

* Bug Fixes for JumpStart, ModelTrainer ModelBuilder flow

* Bug fixes, code cleanup, and adding integration tests

* Migrating utils (#1832)

* fixed integ test imports

* fixed pyproj

* toml file fix

* fixed import paths and sm-train

* import update in example notebook

* modelbuilder

* modelbuilder utils reference deleted

* Import bug fixes for jumpstart

* fixes

* pyproj fix

* resource requirement fix

* Bug fixes for jumpstart and training

* model builder fix

* model builder merge

* notebooks fix and modeltrainer

* notebook role fix

---------

Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>
Co-authored-by: aviruthen <91846056+aviruthen@users.noreply.github.com>

* Update MLOps integ tests and ModeBuilder model key(#1839)

* Add v3 inference example notebooks (#1840)

* Cleaned up example notebooks (#1841)

* Removed import warnings and organized example notebooks

* Updated folder name

* Removed legacy folder (#1843)

* Removed import warnings and organized example notebooks

* Updated folder name

* Removed legacy folder

* Remove unused files


* Remove files that depend on predictor classes

* test commit to trigger pr checks

* Fixed tox.ini files and codebuild workflow yaml for PR checks

---------

Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <81834882+mohamedzeidan2021@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>
Co-authored-by: Molly He <mollyhe@amazon.com>
Co-authored-by: Rohan Narayan <narrohag@amazon.com>

* Removing estimator import (#1852)

* Adding back MMS and TEI folders (accidentally deleted) (#1853)

* Removing estimator import

* Adding back MMS and TEI folders

* Recreate PR review for exposing parameters and functions for ModelBuilder (#1854)

* Updated user-exposed functions and parameters

* Updating deploy() header

* Simplifying build() and optimize()

* Update integration tests

* Final update to model_builder.py

* Add unit tests to MLOps, Fix sagemaker-train tests, remove unused imp… (#1856)

* Add unit tests to MLOps, Fix sagemaker-train tests, remove unused imports

* Add unit tests to MLOps, Fix sagemaker-train tests

* Add sagemaker-serve unit tests (#1858)

* Bug fixes for integ tests (#1859)

* Add core unit tests (#1860)

 Fix sagemaker core unit tests

* Add serve unit tests (#1861)

* Add sagemaker-serve unit tests

* Fix sagemaker core unit tests

* More fixes for the unit tests

* ModelBuilder follow up fixes

* ModelBuilder example fixes

* Update integ tests (#1864)

* Fix optimize integ test (#1866)

* Update readme and changelog

* Update readme, changelogs

* Add comprehensive unit tests for sagemaker-core (#1870)

* Test for Model BUilder servers (#1871)

* Add sagemaker-core unit test (#1872)

* Add sagemaker-train and sagemaker-mlops unit tests (#1873)

* Add sagemaker-train and sagemaker-mlops unit tests

* Add sagemaker-serve unit tests

* Fix: Include region_config.json in package data (#1875)

* Add MLOps integ tests (#1877)

* Add additional sagemaker-serve unit tests (#1878)

* removed one integ tests (#1879)

---------

Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <81834882+mohamedzeidan2021@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>
Co-authored-by: Molly He <mollyhe@amazon.com>
Co-authored-by: Rohan Narayan <narrohag@amazon.com>
Co-authored-by: Gokul Anantha Narayanan <166456257+nargokul@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants