Skip to content

feature: Add support for Native Pytorch DDP #2705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 4 commits into from
Closed

feature: Add support for Native Pytorch DDP #2705

wants to merge 4 commits into from

Conversation

@satishpasumarthi
Copy link
Contributor

@satishpasumarthi satishpasumarthi commented Oct 14, 2021 •
edited
Loading

Issue #, if available:

Description of changes:

Testing done:

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

  • I have read the CONTRIBUTING doc
  • I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
  • I used the commit message format described in CONTRIBUTING
  • I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
  • I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

  • I have added tests that prove my fix is effective or that my feature works (if appropriate)
  • I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
  • I have checked that my tests are not configured for a specific region or account (if appropriate)
  • I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@satishpasumarthi satishpasumarthi marked this pull request as draft October 14, 2021 23:45
@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Oct 14, 2021

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: ef96783
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Oct 14, 2021

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: 25f9b99
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Oct 15, 2021

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-slow-tests
  • Commit ID: ef96783
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Oct 15, 2021

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-slow-tests
  • Commit ID: 25f9b99
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Oct 15, 2021

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: ef96783
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Oct 15, 2021

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: 25f9b99
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@shreyapandit
Copy link
Contributor

shreyapandit commented Jan 14, 2022

@satishpasumarthi Please fix failing tests and update your branch for us to review this again

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Feb 2, 2022

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: 5dd8b85
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Feb 2, 2022

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: 5dd8b85
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Feb 3, 2022

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-slow-tests
  • Commit ID: 5dd8b85
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

ahsan-z-khan
ahsan-z-khan reviewed Feb 18, 2022
View reviewed changes
Copy link
Contributor

@ahsan-z-khan ahsan-z-khan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this being worked on? or we can close it for now?
@satishpasumarthi

@satishpasumarthi
Copy link
Contributor Author

satishpasumarthi commented Feb 18, 2022

is this being worked on? or we can close it for now? @satishpasumarthi

Hi, Yes it is. There are other connecting parts in sagemaker-training-toolkit and pytorch toolkit which is being worked on. Please leave it in draft for the next few days.

@satishpasumarthi satishpasumarthi changed the title feature: Add support for Pytorch Vanilla DDP feature: Add support for Native Pytorch DDP Feb 21, 2022
This was referenced Feb 21, 2022
Closed
Closed
@satishpasumarthi satishpasumarthi marked this pull request as ready for review February 22, 2022 20:31
@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Feb 22, 2022

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-unit-tests
  • Commit ID: bf74aeb
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Feb 22, 2022

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-slow-tests
  • Commit ID: bf74aeb
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@sagemaker-bot
Copy link
Collaborator

sagemaker-bot commented Feb 22, 2022

AWS CodeBuild CI Report

  • CodeBuild project: sagemaker-python-sdk-local-mode-tests
  • Commit ID: bf74aeb
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@satishpasumarthi satishpasumarthi marked this pull request as draft March 24, 2022 16:59
@vikas0203 vikas0203 requested review from a team, claytonparnell and trajanikant and removed request for a team November 23, 2022 21:37
@mufaddal-rohawala
Copy link
Member

mufaddal-rohawala commented May 9, 2023

Closing as this has been in draft state for too long! Please reopen if the change is still relevant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@trajanikant trajanikant Awaiting requested review from trajanikant trajanikant was automatically assigned from aws/sagemaker-ml-frameworks

@claytonparnell claytonparnell Awaiting requested review from claytonparnell claytonparnell was automatically assigned from aws/sagemaker-ml-frameworks

1 more reviewer

@ahsan-z-khan ahsan-z-khan ahsan-z-khan left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@claytonparnell claytonparnell

Labels

needs contributor changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@satishpasumarthi @sagemaker-bot @shreyapandit @mufaddal-rohawala @ahsan-z-khan @claytonparnell