Skip to content

Commit

Permalink
Sync branch (#4718)
Browse files Browse the repository at this point in the history
* fix: mainline alt config parsing (#4602)

* fix: parsing

* fix: commit tests

* fix: types

* updated

* fix

* Add Triton v24.03 URI (#4605)

Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com>

* feature: support session tag chaining for training job (#4596)

* feature: support session tag chaining for training job

* fix: resolve typo

* fix: resolve typo and build failure

* fix: resolve typo and unit test failure

---------

Co-authored-by: Jessica Zhu <jessicazhu3@106775307+jessicazhu3@users.noreply.github.com>

* prepare release v2.217.0

* update development version to v2.217.1.dev0

* fix: properly close files in lineage queries and tests (#4587)

Closes #4458

* feature: set default allow_pickle param to False (#4557)

* breaking: set default allow_pickle param to False

* breaking: fix unit tests and linting

NumpyDeserializer will not allow deserialization
unless allow_pickle flag is set to True explicitly

* fix: black-check

---------

Co-authored-by: Ashwin Krishna <ashwikri@amazon.com>

* Fix:invalid component error with new metadata (#4634)

* fix: invalid component name

* tests

* format

* fix vulnerable model integ tests llama 2

* updated

* fix: training dataset location

* prepare release v2.218.0

* update development version to v2.218.1.dev0

* chore: update skipped flaky tests (#4644)

* Update skipped flaky tests

* flake8

* format

* format

* chore: release tgi 2.0.1 (#4642)

* chore: release tgi 2.0.1

* minor fix

---------

Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>

* fix: Fix UserAgent logging in Python SDK (#4647)

* prepare release v2.218.1

* update development version to v2.218.2.dev0

* feature: allow choosing js payload by alias in private method

* Updates for SMP v2.3.1 (#4660)

Co-authored-by: Suhit Kodgule <skodgule@amazon.com>

* chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /doc (#4655)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump tqdm from 4.66.2 to 4.66.3 in /tests/data/serve_resources/mlflow/pytorch (#4650)

Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.2 to 4.66.3.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](tqdm/tqdm@v4.66.2...v4.66.3)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /requirements/extras (#4654)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* prepare release v2.219.0

* update development version to v2.219.1.dev0

* fix: skip flakey tests pending investigation (#4667)

* change: update image_uri_configs  05-09-2024 07:17:41 PST

* Add tensorflow_serving support for mlflow models and enable lineage tracking for mlflow models (#4662)

* Initial commit for tensorflow_serving support of MLflow

* Add integ tests for mlflow tf_serving

* fix style issues

* remove unused attributes from tf builder

* Add deep ping for tf_serving local mode

* Initial commit for lineage impl

* Initial commit for tensorflow_serving support of MLflow

* Add integ tests for mlflow tf_serving

* fix style issues

* remove unused attributes from tf builder

* Add deep ping for tf_serving local mode

* Add integ tests and uts

* fix local mode for tf_serving

* Allow lineage tracking only in sagemaker endpoint mode

* fix regex pattern

* fix style issues

* fix regex pattern and hard coded py version in ut

* fix missing session

* Resolve pr comments and fix regex for mlflow registry and ids

* fix: model builder race condition on sagemaker session (#4673)

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* feat: Add telemetry support for mlflow models (#4674)

* Initial commit for telemetry support

* Fix style issues and add more logger messages

* fix value error messages in ut

* feat: add new images for HF TGI release (#4677)

* chore: add new images for HF TGI release

* test

* feature: AutoGluon 1.1.0 image_uris update (#4679)

Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-154.us-west-2.compute.internal>

* change: add debug logs to workflow container dist creation (#4682)

* prepare release v2.220.0

* update development version to v2.220.1.dev0

* fix: Image URI should take precedence for HF models (#4684)

* Fix: Image URI should take precedence for HF models

* Fix formatting

* Fix formatting

* Fix formatting

* Increase coverage -  UT pass

* feat: onboard tei image config to pysdk (#4681)

* feat: onboard tei image config to pysdk

* fix formatting issue

* minor fix func name

* fix unit tests

---------

Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>

* fix: model builder limited container support for endpoint mode. (#4683)

* Allow ModelBuilder's endpoint mode for Jumpstart models packaged with containers other than TGI and DJL

* increase coverage

* Add JS Support for MMS Serving

* Add JS Support for MMS Serving

* Unit tests

* Refactoring

* Refactoring

* Refactoring

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* change: Add more debuging (#4687)

* change: cover tei with image_uris.retrieve API (#4689)

* fix: JS Model with non-TGI/non-DJL deployment failure (#4688)

* Debug

* Debug

* Debug

* Debug

* Debug

* Debug

* fix docstyle

* Refactoring

* Add Integ tests

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* Feat: Pull latest tei container for sentence similiarity models on HuggingFace hub (#4686)

* Update: Pull latest tei container for sentence similiarity models

* Fix formatting

* Address PR comments

* Fix formatting

* Fix check

* Switch sentence similarity to be deployed on tgi

* Fix formatting

* Fix formatting

* Fix formatting

* Fix formatting

* Introduce TEI builder with TGI server

* Fix formmatting

* Add integ test

* Fix formatting

* Add integ test

* Add integ test

* Add integ test

* Add integ test

* Add integ test

* Fix formatting

* Move to G5 for integ test

* Fix formatting

* Integ test updates

* Integ test updates

* Integ test updates

* Fix formatting

* Integ test updates

* Move back to generate for ping

* Integ test updates

* Integ test updates

* Fix: Add Image URI overrides for transformers models (#4693)

* Fix: Add Image URI overrides for transformers models

* Increase coverage

* Fix formatting

* prepare release v2.221.0

* update development version to v2.221.1.dev0

* Add tei cpu image (#4695)

* Add tei cpu image

* fix format issue

* fix unit tests

* fix typo

* fix typo

* Feat: Add TEI support for ModelBuilder (#4694)

* Add TEI Serving

* Add TEI Serving

* Add TEI Serving

* Add TEI Serving

* Add TEI Serving

* Add TEI Serving

* Notebook testing

* Notebook testing

* Notebook testing

* Refactoring

* Refactoring

* UT

* UT

* Refactoring

* Test coverage

* Refactoring

* Refactoring

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* Convert pytorchddp distribution to smdistributed distribution (#4698)

* rewrite pytorchddp to smdistributed

* remove instance type check

* Update estimator.py

* remove validate_pytorch_distribution

* fix

* fix unit tests

* fix formatting

* check instance type not None

* prepare release v2.221.1

* update development version to v2.221.2.dev0

* Update: SM Endpoint Routing Strategy Support. (#4702)

* RoutingConfig

* Refactoring

* Docstring

* UT

* Refactoring

* Refactoring

---------

Co-authored-by: Jonathan Makunga <makung@amazon.com>

* change: update image_uri_configs  05-29-2024 07:17:35 PST

* Making project name in workflow files dynamic (#4708)

* fix: Fix ci unit-tests (#4713)

* chore(deps): bump requests from 2.31.0 to 2.32.2 in /tests/data/serve_resources/mlflow/pytorch (#4709)

Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.2.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](psf/requests@v2.31.0...v2.32.2)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump apache-airflow from 2.9.0 to 2.9.1 in /requirements/extras (#4703)

* chore(deps): bump apache-airflow in /requirements/extras

Bumps [apache-airflow](https://github.com/apache/airflow) from 2.9.0 to 2.9.1.
- [Release notes](https://github.com/apache/airflow/releases)
- [Changelog](https://github.com/apache/airflow/blob/main/RELEASE_NOTES.rst)
- [Commits](apache/airflow@2.9.0...2.9.1)

---
updated-dependencies:
- dependency-name: apache-airflow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update tox.ini to bump apache-airflow

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kalyani Nikure <110067132+knikure@users.noreply.github.com>

* chore(deps): bump mlflow from 2.10.2 to 2.12.1 in /tests/data/serve_resources/mlflow/pytorch (#4690)

Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.10.2 to 2.12.1.
- [Release notes](https://github.com/mlflow/mlflow/releases)
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md)
- [Commits](mlflow/mlflow@v2.10.2...v2.12.1)

---
updated-dependencies:
- dependency-name: mlflow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump mlflow from 2.11.1 to 2.12.1 in /tests/data/serve_resources/mlflow/xgboost (#4692)

Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.11.1 to 2.12.1.
- [Release notes](https://github.com/mlflow/mlflow/releases)
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md)
- [Commits](mlflow/mlflow@v2.11.1...v2.12.1)

---
updated-dependencies:
- dependency-name: mlflow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump mlflow from 2.11.1 to 2.12.1 in /tests/data/serve_resources/mlflow/tensorflow (#4691)

Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.11.1 to 2.12.1.
- [Release notes](https://github.com/mlflow/mlflow/releases)
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md)
- [Commits](mlflow/mlflow@v2.11.1...v2.12.1)

---
updated-dependencies:
- dependency-name: mlflow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* change: Updates for DJL 0.28.0 release (#4701)

* Sync Branch

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Haotian An <33510317+Captainia@users.noreply.github.com>
Co-authored-by: Nikhil Kulkarni <knikhil29@gmail.com>
Co-authored-by: Nikhil Kulkarni <nikhilsk@amazon.com>
Co-authored-by: jessicazhu3 <106775307+jessicazhu3@users.noreply.github.com>
Co-authored-by: Jessica Zhu <jessicazhu3@106775307+jessicazhu3@users.noreply.github.com>
Co-authored-by: ci <ci>
Co-authored-by: Justin <justinm088@hotmail.com>
Co-authored-by: ASHWIN KRISHNA <38850354+akrishna1995@users.noreply.github.com>
Co-authored-by: Ashwin Krishna <ashwikri@amazon.com>
Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com>
Co-authored-by: Zhaoqi <52220743+zhaoqizqwang@users.noreply.github.com>
Co-authored-by: Kalyani Nikure <110067132+knikure@users.noreply.github.com>
Co-authored-by: Keerthan Vasist <kvasist@amazon.com>
Co-authored-by: SuhitK <kodgule.suhit@gmail.com>
Co-authored-by: Suhit Kodgule <skodgule@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com>
Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com>
Co-authored-by: Jonathan Makunga <makung@amazon.com>
Co-authored-by: Prateek M Desai <prateekmdesai04@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-154.us-west-2.compute.internal>
Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>
Co-authored-by: Samrudhi Sharma <154457034+samruds@users.noreply.github.com>
Co-authored-by: Tom Bousso <tombousso@gmail.com>
Co-authored-by: Zhaoqi <jzhaoqwa@amazon.com>
Co-authored-by: Tyler Osterberg <tyoster@amazon.com>
  • Loading branch information
26 people committed Jun 5, 2024
1 parent 6fb935d commit 1bb9eec
Show file tree
Hide file tree
Showing 54 changed files with 1,642 additions and 219 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/codebuild-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
- name: Run Codestyle & Doc Tests
uses: aws-actions/aws-codebuild-run-build@v1
with:
project-name: sagemaker-python-sdk-ci-codestyle-doc-tests
project-name: ${{ github.event.repository.name }}-ci-codestyle-doc-tests
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
unit-tests:
runs-on: ubuntu-latest
Expand All @@ -74,7 +74,7 @@ jobs:
- name: Run Unit Tests
uses: aws-actions/aws-codebuild-run-build@v1
with:
project-name: sagemaker-python-sdk-ci-unit-tests
project-name: ${{ github.event.repository.name }}-ci-unit-tests
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
env-vars-for-codebuild: |
PY_VERSION
Expand All @@ -93,5 +93,5 @@ jobs:
- name: Run Integ Tests
uses: aws-actions/aws-codebuild-run-build@v1
with:
project-name: sagemaker-python-sdk-ci-integ-tests
project-name: ${{ github.event.repository.name }}-ci-integ-tests
source-version-override: 'refs/pull/${{ github.event.pull_request.number }}/head^{${{ github.event.pull_request.head.sha }}}'
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,26 @@
# Changelog

## v2.221.1 (2024-05-22)

### Bug Fixes and Other Changes

* Convert pytorchddp distribution to smdistributed distribution
* Add tei cpu image

## v2.221.0 (2024-05-20)

### Features

* onboard tei image config to pysdk

### Bug Fixes and Other Changes

* JS Model with non-TGI/non-DJL deployment failure
* cover tei with image_uris.retrieve API
* Add more debuging
* model builder limited container support for endpoint mode.
* Image URI should take precedence for HF models

## v2.220.0 (2024-05-15)

### Features
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.220.1.dev0
2.221.2.dev0
2 changes: 1 addition & 1 deletion requirements/extras/test_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ awslogs==0.14.0
black==24.3.0
stopit==1.1.2
# Update tox.ini to have correct version of airflow constraints file
apache-airflow==2.9.0
apache-airflow==2.9.1
apache-airflow-providers-amazon==7.2.1
attrs>=23.1.0,<24
fabric==2.6.0
Expand Down
12 changes: 12 additions & 0 deletions src/sagemaker/enums.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,15 @@ class EndpointType(Enum):
INFERENCE_COMPONENT_BASED = (
"InferenceComponentBased" # Amazon SageMaker Inference Component Based Endpoint
)


class RoutingStrategy(Enum):
"""Strategy for routing https traffics."""

RANDOM = "RANDOM"
"""The endpoint routes each request to a randomly chosen instance.
"""
LEAST_OUTSTANDING_REQUESTS = "LEAST_OUTSTANDING_REQUESTS"
"""The endpoint routes requests to the specific instances that have
more capacity to process them.
"""
99 changes: 2 additions & 97 deletions src/sagemaker/fw_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,22 +145,6 @@
],
}

PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS = [
"1.10",
"1.10.0",
"1.10.2",
"1.11",
"1.11.0",
"1.12",
"1.12.0",
"1.12.1",
"1.13.1",
"2.0.0",
"2.0.1",
"2.1.0",
"2.2.0",
]

TORCH_DISTRIBUTED_GPU_SUPPORTED_FRAMEWORK_VERSIONS = [
"1.13.1",
"2.0.0",
Expand Down Expand Up @@ -795,7 +779,6 @@ def _validate_smdataparallel_args(
Raises:
ValueError: if
(`instance_type` is not in SM_DATAPARALLEL_SUPPORTED_INSTANCE_TYPES or
`py_version` is not python3 or
`framework_version` is not in SM_DATAPARALLEL_SUPPORTED_FRAMEWORK_VERSION
"""
Expand All @@ -806,17 +789,10 @@ def _validate_smdataparallel_args(
if not smdataparallel_enabled:
return

is_instance_type_supported = instance_type in SM_DATAPARALLEL_SUPPORTED_INSTANCE_TYPES

err_msg = ""

if not is_instance_type_supported:
# instance_type is required
err_msg += (
f"Provided instance_type {instance_type} is not supported by smdataparallel.\n"
"Please specify one of the supported instance types:"
f"{SM_DATAPARALLEL_SUPPORTED_INSTANCE_TYPES}\n"
)
if not instance_type:
err_msg += "Please specify an instance_type for smdataparallel.\n"

if not image_uri:
# ignore framework_version & py_version if image_uri is set
Expand Down Expand Up @@ -928,13 +904,6 @@ def validate_distribution(
)
if framework_name and framework_name == "pytorch":
# We need to validate only for PyTorch framework
validate_pytorch_distribution(
distribution=validated_distribution,
framework_name=framework_name,
framework_version=framework_version,
py_version=py_version,
image_uri=image_uri,
)
validate_torch_distributed_distribution(
instance_type=instance_type,
distribution=validated_distribution,
Expand Down Expand Up @@ -968,13 +937,6 @@ def validate_distribution(
)
if framework_name and framework_name == "pytorch":
# We need to validate only for PyTorch framework
validate_pytorch_distribution(
distribution=validated_distribution,
framework_name=framework_name,
framework_version=framework_version,
py_version=py_version,
image_uri=image_uri,
)
validate_torch_distributed_distribution(
instance_type=instance_type,
distribution=validated_distribution,
Expand Down Expand Up @@ -1023,63 +985,6 @@ def validate_distribution_for_instance_type(instance_type, distribution):
raise ValueError(err_msg)


def validate_pytorch_distribution(
distribution, framework_name, framework_version, py_version, image_uri
):
"""Check if pytorch distribution strategy is correctly invoked by the user.
Args:
distribution (dict): A dictionary with information to enable distributed training.
(Defaults to None if distributed training is not enabled.) For example:
.. code:: python
{
"pytorchddp": {
"enabled": True
}
}
framework_name (str): A string representing the name of framework selected.
framework_version (str): A string representing the framework version selected.
py_version (str): A string representing the python version selected.
image_uri (str): A string representing a Docker image URI.
Raises:
ValueError: if
`py_version` is not python3 or
`framework_version` is not in PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS
"""
if framework_name and framework_name != "pytorch":
# We need to validate only for PyTorch framework
return

pytorch_ddp_enabled = False
if "pytorchddp" in distribution:
pytorch_ddp_enabled = distribution.get("pytorchddp").get("enabled", False)
if not pytorch_ddp_enabled:
# Distribution strategy other than pytorchddp is selected
return

err_msg = ""
if not image_uri:
# ignore framework_version and py_version if image_uri is set
# in case image_uri is not set, then both are mandatory
if framework_version not in PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS:
err_msg += (
f"Provided framework_version {framework_version} is not supported by"
" pytorchddp.\n"
"Please specify one of the supported framework versions:"
f" {PYTORCHDDP_SUPPORTED_FRAMEWORK_VERSIONS} \n"
)
if "py3" not in py_version:
err_msg += (
f"Provided py_version {py_version} is not supported by pytorchddp.\n"
"Please specify py_version>=py3"
)
if err_msg:
raise ValueError(err_msg)


def validate_torch_distributed_distribution(
instance_type,
distribution,
Expand Down
14 changes: 14 additions & 0 deletions src/sagemaker/huggingface/llm_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,20 @@ def get_huggingface_llm_image_uri(
image_scope="inference",
inference_tool="neuronx",
)
if backend == "huggingface-tei":
return image_uris.retrieve(
"huggingface-tei",
region=region,
version=version,
image_scope="inference",
)
if backend == "huggingface-tei-cpu":
return image_uris.retrieve(
"huggingface-tei-cpu",
region=region,
version=version,
image_scope="inference",
)
if backend == "lmi":
version = version or "0.24.0"
return image_uris.retrieve(framework="djl-deepspeed", region=region, version=version)
Expand Down
1 change: 1 addition & 0 deletions src/sagemaker/huggingface/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,7 @@ def deploy(
endpoint_type=kwargs.get("endpoint_type", None),
resources=kwargs.get("resources", None),
managed_instance_scaling=kwargs.get("managed_instance_scaling", None),
routing_config=kwargs.get("routing_config", None),
)

def register(
Expand Down
39 changes: 39 additions & 0 deletions src/sagemaker/image_uri_config/djl-lmi.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"scope": [
"inference"
],
"versions": {
"0.28.0": {
"registries": {
"af-south-1": "626614931356",
"il-central-1": "780543022126",
"ap-east-1": "871362719292",
"ap-northeast-1": "763104351884",
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
"us-west-1": "763104351884",
"us-west-2": "763104351884",
"ca-west-1": "204538143572"
},
"repository": "djl-inference",
"tag_prefix": "0.28.0-lmi10.0.0-cu124"
}
}
}
18 changes: 18 additions & 0 deletions src/sagemaker/image_uri_config/djl-neuronx.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,24 @@
"inference"
],
"versions": {
"0.28.0": {
"registries": {
"ap-northeast-1": "763104351884",
"ap-south-1": "763104351884",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"eu-central-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-3": "763104351884",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
"us-west-2": "763104351884",
"ca-west-1": "204538143572"
},
"repository": "djl-inference",
"tag_prefix": "0.28.0-neuronx-sdk2.18.2"
},
"0.27.0": {
"registries": {
"ap-northeast-1": "763104351884",
Expand Down
32 changes: 32 additions & 0 deletions src/sagemaker/image_uri_config/djl-tensorrtllm.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,38 @@
"inference"
],
"versions": {
"0.28.0": {
"registries": {
"af-south-1": "626614931356",
"il-central-1": "780543022126",
"ap-east-1": "871362719292",
"ap-northeast-1": "763104351884",
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
"us-west-1": "763104351884",
"us-west-2": "763104351884",
"ca-west-1": "204538143572"
},
"repository": "djl-inference",
"tag_prefix": "0.28.0-tensorrtllm0.9.0-cu122"
},
"0.27.0": {
"registries": {
"af-south-1": "626614931356",
Expand Down
Loading

0 comments on commit 1bb9eec

Please sign in to comment.