SageMaker @remote decorator with private package repository

This repository is part of AWS blog post Easily access private repos using the remote decorator mode for SageMaker training workloads which demonstrates how to use @remote decorator from Amazon SageMaker Python SDK while running the SageMaker training job in a VPC with no internet access. To enable download of dependencies for the training job, it uses a privte PyPI AWS CodeArtifact repository instead.

Contents in the repository

• cloudformation folder: This folder contains the cloudformation scripts needed to setup the VPC, CodeArtifact Repository and SageMaker Studio. It contains two files:
  • vpc.yaml: Contains the code for setting up the VPC and VPC endpoints for all supported services.
  • sagemaker_studio_codeartifact.yaml: Contains the code for setting up SageMaker Studio with VPC and Code Artifact Domain, private repository and upstream repository
    
    
    
• config folder: This folder contains:
  • config.yaml: Contains all the configurations needed to run the SageMaker training job while connecting to private PyPI repository.
  • requirements.txt: Contains all the dependencies needed to run the training script.
    
    
• remote-decorator-private-repo.ipynb: Contains the code for running the training job.

Instruction to run the code

• Execute the cloudformtion template vpc.yaml. Provide a name to the stack(example: No-Internet). Wait for it to get completed.
• Execute the cloudformation template sagemaker_studio_codeartifact.yaml. Provide the VPC stack name(No-Internet). You may leave everything else as default or change the value of any other parameter if you like. Wait for the stack to be completed
• Open remote-decorator-private-repo.ipynb.
  • Select Python 3(Data Science) as Kernel and image. Wait for the kernel session to begin.
  • Read through the instructions of the notebook and execute cells.
  • Update the content of config.yaml based on your VPC and CodeArtifact details(detailed instructions in the notebook). Leave everything else in the yaml as-is.
  • Continue the exection of cells after updating config.yaml and run the training job.

Clean up

You will be creating several billable resources in your AWS account as part of this setup. After experimentation, make sure you follow the steps in CLEANUP.md to remove all resources created. If you encounter any issues when deleting the resources, troubleshoot the issues by following the AWS CloudFormation troubleshooting guide

Support

Open a bug in case you find anything not working.

Contributing

See CONTRIBUTING.md for details.

License

See LICENSE