Skip to content

Fix: Update Airflow prepare_framework Function to not use os.path.join #3393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 18 commits into from
Closed

Fix: Update Airflow prepare_framework Function to not use os.path.join #3393

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
3a8e471
fix: do not use in airflow prepare_framework function
GroovyDan Oct 5, 2022
e6aab74
Merge branch 'master' into fix_airflow_prepare_framework_windows_bug
GroovyDan Oct 25, 2022
d0d0bb6
Merge branch 'master' into fix_airflow_prepare_framework_windows_bug
GroovyDan Nov 7, 2022
c97c467
fix: type hint of PySparkProcessor __init__ (#3297)
NivekNey Dec 2, 2022
de58941
fix: fix PySparkProcessor __init__ params type (#3354)
andre-marcos-perez Dec 2, 2022
41dd330
fix: Allow Py 3.7 for MMS Test Docker env (#3080)
shreyapandit Dec 2, 2022
1e23a3f
refactoring : using with statement (#3286)
maldil Dec 2, 2022
19efadf
Update local_requirements.txt PyYAML version (#3095)
shreyapandit Dec 2, 2022
76f7782
feature: Update TF 2.9 and TF 2.10 inference DLCs (#3465)
arjkesh Dec 2, 2022
fde0738
feature: Added transform with monitoring pipeline step in transformer…
keshav-chandak Dec 2, 2022
7f9f3b0
fix: Fix bug forcing uploaded tar to be named sourcedir (#3412)
claytonparnell Dec 2, 2022
5d59767
feature: Add Code Owners file (#3503)
navinsoni Dec 2, 2022
0f5cf18
prepare release v2.119.0
Dec 3, 2022
f1f0013
update development version to v2.119.1.dev0
Dec 3, 2022
bb4b689
feature: Add DXB region to frameworks by DLC (#3387)
RadhikaB-97 Dec 5, 2022
b68bcd9
fix: support idempotency for framework and spark processors (#3460)
brockwade633 Dec 5, 2022
32969da
feature: Update registries with new region account number mappings. (…
kenny-ezirim Dec 6, 2022
32bca2a
Merge branch 'master' into fix_airflow_prepare_framework_windows_bug
GroovyDan Dec 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,33 @@
# Changelog

## v2.119.0 (2022-12-03)

### Features

* Add Code Owners file
* Added transform with monitoring pipeline step in transformer
* Update TF 2.9 and TF 2.10 inference DLCs
* make estimator accept json file as modelparallel config
* SageMaker Training Compiler does not support p4de instances
* Add support for SparkML v3.3

### Bug Fixes and Other Changes

* Fix bug forcing uploaded tar to be named sourcedir
* Update local_requirements.txt PyYAML version
* refactoring : using with statement
* Allow Py 3.7 for MMS Test Docker env
* fix PySparkProcessor __init__ params type
* type hint of PySparkProcessor __init__
* Return ARM XGB/SKLearn tags if `image_scope` is `inference_graviton`
* Update scipy to 1.7.3 to support M1 development envs
* Fixing type hints for Spark processor that has instance type/count params in reverse order
* Add DeepAR ap-northeast-3 repository.
* Fix AsyncInferenceConfig documentation typo
* fix ml_inf to ml_inf1 in Neo multi-version support
* Fix type annotations
* add neo mvp region accounts

## v2.118.0 (2022-12-01)

### Features
Expand Down
1 change: 1 addition & 0 deletions CODEOWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @aws/sagemaker-ml-frameworks
2 changes: 1 addition & 1 deletion VERSION
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.118.1.dev0
2.119.1.dev0
5 changes: 2 additions & 3 deletions src/sagemaker/git_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,9 +279,8 @@ def _run_clone_command(repo_url, dest_dir):
subprocess.check_call(["git", "clone", repo_url, dest_dir], env=my_env)
elif repo_url.startswith("git@"):
with tempfile.NamedTemporaryFile() as sshnoprompt:
write_pipe = open(sshnoprompt.name, "w")
write_pipe.write("ssh -oBatchMode=yes $@")
write_pipe.close()
with open(sshnoprompt.name, "w") as write_pipe:
write_pipe.write("ssh -oBatchMode=yes $@")
os.chmod(sshnoprompt.name, 0o511)
my_env["GIT_SSH"] = sshnoprompt.name
subprocess.check_call(["git", "clone", repo_url, dest_dir], env=my_env)
Expand Down
30 changes: 30 additions & 0 deletions src/sagemaker/image_uri_config/autogluon.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -56,6 +57,7 @@
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -86,6 +88,7 @@
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -116,6 +119,7 @@
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -146,6 +150,7 @@
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -176,6 +181,7 @@
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -204,19 +210,23 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand All @@ -237,19 +247,23 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand All @@ -270,19 +284,23 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand All @@ -303,19 +321,23 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand All @@ -336,19 +358,23 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand All @@ -369,19 +395,23 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ap-southeast-3": "907027046896",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down
3 changes: 3 additions & 0 deletions src/sagemaker/image_uri_config/huggingface-neuron.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,20 @@
"ap-northeast-2": "763104351884",
"ap-northeast-3": "364406365360",
"ap-south-1": "763104351884",
"ap-south-2": "772153158452",
"ap-southeast-1": "763104351884",
"ap-southeast-2": "763104351884",
"ca-central-1": "763104351884",
"cn-north-1": "727897471807",
"cn-northwest-1": "727897471807",
"eu-central-1": "763104351884",
"eu-central-2": "380420809688",
"eu-north-1": "763104351884",
"eu-west-1": "763104351884",
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"eu-south-1": "692866216735",
"eu-south-2": "503227376785",
"me-south-1": "217643126080",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
Expand Down
3 changes: 3 additions & 0 deletions src/sagemaker/image_uri_config/huggingface-training-compiler.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -89,6 +90,7 @@
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down Expand Up @@ -123,6 +125,7 @@
"eu-west-2": "763104351884",
"eu-west-3": "763104351884",
"me-south-1": "217643126080",
"me-central-1": "914824155844",
"sa-east-1": "763104351884",
"us-east-1": "763104351884",
"us-east-2": "763104351884",
Expand Down
Loading