feature: network isolation mode for xgboost #2626

awsbmillare · 2021-09-10T20:43:59Z

Issue #, if available:

Network isolation mode is not currently working correctly with xgboost as it is with sklearn

Description of changes:

Just like with PR feature: enable sklearn for network isolation mode #1051 but for xgboost (its very similar to sklearn in implementation)

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I used the commit message format described in CONTRIBUTING
[N/A] I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
[N/A] I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have checked that my tests are not configured for a specific region or account (if appropriate)
[N/A] I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2021-09-15T16:07:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 0af675e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-09-15T22:19:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 406785c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-09-17T23:20:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5ad0756
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

More review needed

shreyapandit

Need to verify if this repacking will not break downstream customers who were operating with N/W isolation as True before. Thanks!

awsbmillare · 2021-09-18T00:41:36Z

Need to verify if this repacking will not break downstream customers who were operating with N/W isolation as True before. Thanks!

This shouldn't be an issue because customers wouldn't be able to get xgboost working with network isolation because they would get an error:

FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/code' when creating the endpoint, which means the tarball is not found. This is because in the source code

sagemaker-python-sdk/src/sagemaker/model.py

Line 1103 in c12fd49

dir_name = "/opt/ml/model/code"

a different path is used but the corresponding requirements to change how the data is packed is not also included.

Dismissing review in light of investigation on above possible issue

sagemaker-bot · 2021-09-20T17:20:19Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 14953bd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

shreyapandit

Hi Brent, changes look good, thanks for this fix! Can you add an integration test to make sure this works e2e when running training, a baseline example to follow is the SKLearn test, however please feel free to enhance it.

sagemaker-bot · 2021-10-08T20:37:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1b19f3c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-08T20:39:34Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 77a5a5e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T00:50:48Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 3c2fc2b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T01:29:55Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 3c2fc2b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T01:39:56Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 3c2fc2b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T02:18:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 3c2fc2b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T02:18:41Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d533fa7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T02:51:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d533fa7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T02:59:02Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d533fa7
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T06:35:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 4bb91b5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T07:07:14Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 4bb91b5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T07:16:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 4bb91b5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

tox.ini

tests/data/xgboost_abalone/abalone.py

sagemaker-bot · 2021-10-19T20:41:58Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 97d63cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T21:19:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 97d63cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-19T21:32:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 97d63cd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-21T00:35:08Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9427b8e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-21T01:15:21Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9427b8e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-21T01:17:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9427b8e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-21T01:38:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 1f43a59
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-21T02:16:51Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 1f43a59
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2021-10-21T02:22:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 1f43a59
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

* feature: network isolation mode for xgboost * Add integ tests for xgboost net iso * fix import * fix job_name name * fix black-check * Revert local test setup changes Co-authored-by: Brent Millare <bmillare@amazon.com> Co-authored-by: Ahsan Khan <ahsan.al.zaki@gmail.com> Co-authored-by: Shreya Pandit <shreya.pandit@pillpack.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Shreya Pandit <shreya.pandit25@gmail.com>

Brent Millare and others added 3 commits September 10, 2021 20:40

feature: network isolation mode for xgboost

a58ce71

Merge branch 'master' into xgboost_network_isolation

e91eeb1

Merge branch 'master' into xgboost_network_isolation

0af675e

navinsoni and others added 2 commits September 15, 2021 09:53

Merge branch 'master' into xgboost_network_isolation

144fd07

Merge branch 'master' into xgboost_network_isolation

406785c

shreyapandit requested review from mufaddal-rohawala and navinsoni September 16, 2021 21:31

mufaddal-rohawala previously approved these changes Sep 16, 2021

View reviewed changes

ahsan-z-khan previously approved these changes Sep 17, 2021

View reviewed changes

shreyapandit approved these changes Sep 17, 2021

View reviewed changes

Merge branch 'master' into xgboost_network_isolation

5ad0756

shreyapandit suggested changes Sep 18, 2021

View reviewed changes

navinsoni requested a review from mufaddal-rohawala September 18, 2021 00:21

Merge branch 'master' into xgboost_network_isolation

14953bd

shreyapandit suggested changes Sep 21, 2021

View reviewed changes

ahsan-z-khan previously approved these changes Sep 21, 2021

View reviewed changes

shreyapandit marked this pull request as draft September 21, 2021 19:53

awsbmillare and others added 3 commits October 8, 2021 16:24

Merge branch 'master' into xgboost_network_isolation

1b19f3c

feature: network isolation mode for xgboost

b038943

Add integ tests for xgboost net iso

77a5a5e

awsbmillare dismissed ahsan-z-khan’s stale review via 77a5a5e October 8, 2021 20:30

Merge branch 'master' into xgboost_network_isolation

3c2fc2b

fix black-check

d533fa7

Merge branch 'master' into xgboost_network_isolation

4bb91b5

mufaddal-rohawala requested changes Oct 19, 2021

View reviewed changes

tox.ini Outdated Show resolved Hide resolved

tests/data/xgboost_abalone/abalone.py Show resolved Hide resolved

Revert local test setup changes

97d63cd

shreyapandit approved these changes Oct 20, 2021

View reviewed changes

mufaddal-rohawala approved these changes Oct 20, 2021

View reviewed changes

Merge branch 'master' into xgboost_network_isolation

9427b8e

Merge branch 'master' into xgboost_network_isolation

1f43a59

navinsoni merged commit 25da5cc into aws:master Oct 21, 2021

feature: network isolation mode for xgboost #2626

feature: network isolation mode for xgboost #2626

Uh oh!

Conversation

awsbmillare commented Sep 10, 2021

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Sep 15, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Sep 15, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Sep 17, 2021

AWS CodeBuild CI Report

Uh oh!

shreyapandit left a comment

Choose a reason for hiding this comment

Uh oh!

awsbmillare commented Sep 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sagemaker-bot commented Sep 20, 2021

AWS CodeBuild CI Report

Uh oh!

shreyapandit left a comment

Choose a reason for hiding this comment

Uh oh!

sagemaker-bot commented Oct 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 8, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

Uh oh!

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 19, 2021

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Oct 21, 2021

AWS CodeBuild CI Report

Uh oh!

awsbmillare commented Sep 18, 2021 •

edited

Loading