Skip to content
Creates a CloudFormation template that uses AWS StepFunctions to automate the building and training of Sagemaker custom models based on S3 and GitHub events
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Creating initial file from template Feb 26, 2018
assets update README.md Aug 31, 2018
bin
sagebuild removed BYOD example, fixed external repo bug Mar 29, 2019
.gitignore
CHANGELOG.md
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
NOTICE
README.md Update Readme to show images Oct 17, 2018
config.js.example
package.json

README.md

AWS SageMaker Build

Automating the build and deployment of machine learning models is an important step in creating production machine learning services. Models need to be retrained and deployed when code and/or data are updated. This project provides a full implementation of a CI/CD workflow and includes jupyter notebooks showing how to create, launch, stop, and track the progress of builds using python and Amazon Alexa! The goal of aws-sagemaker-build is to provide a repository of common and useful SageMaker/Step Function pipelines, to be shared with the community and grown by the community.

Prerequisites

  • Run Linux. (tested on Amazon Linux)
  • Install npm >5 and node >8. (instructions)
  • Clone this repo.
  • Set up an AWS account. (instructions)
  • Configure AWS CLI and a local credentials file. (instructions)

setup

  1. install all need packages
npm install
  1. copy config.js.example to config.js

  2. create an s3 bucket. instructions. open up config.js and set templateBucket and AssetBucket to the name of your s3 bucket.

  3. launch stack

npm run up

Stack Management

npm run up #launches stack
npm run update #updates the launched stack
npm run down #shuts down stack

template is written to /cloudformation/build/template.json

Architecture

The following diagram describes the flow of the Step Function StateMachine. There are several points where the StateMachine has to poll and wait for a task to complete.

statemachine diagram

The following diagram shows how the services work together

service diagram

Parameters

AWS Systems Manager Parameter Store provides a durable, centralized, and scalable data store. We will store the parameters of our training jobs and deployment here and the Step Function's Lambda functions will query the parameters from this store. To change the parameters you just change the JSON string in the store. The example notebooks included with aws-sagemaker-build show how to do this.

Common

  • hyperparameters: default=HyperParameters,
  • hostinstancecount: default=1,
  • hostinstancetype: default=ml.t2.medium,
  • traininstancecount: default=1,
  • traininstancetype: default=ml.m5.large,
  • trainvolumesize: default=10,
  • trainmaxrun: default=4,
  • inputmode: default=File,
  • modelhostingenvironment: default={}
  • hyperparameters: default={},

BYOD

  • dockerfile_path_Training:
  • dockerfile_path_Inference:
  • train: default=true
  • build: default={Inference:true, Training:true}
  • TrainingImage:

TensorFlow

  • tensorflowversion: default=1.8,
  • trainingsteps: default=1000,
  • evaluationsteps: default=100,
  • requirementsfile: default=none,
  • trainentrypoint: default=none,
  • trainsourcefile: default=none,
  • pyversion: default=py3,
  • hostentrypoint: default=none,
  • hostsourcefile: default=none,
  • enablecloudwatchmetrics: default=false,
  • containerloglevel: default=200,

MXNET

  • mxnetversion: default=1.1,
  • trainentrypoint: default=none,
  • trainsourcefile: default=none,
  • pyversion: default=py3,
  • hostentrypoint: default=none,
  • hostsourcefile: default=none,
  • enablecloudwatchmetrics: default=false,
  • containerloglevel: default=200,

HPO

  • maxtrainingjobs: default=1,
  • maxparalleltrainingjobs: default=1,

Amazon Algorithms

  • algorithm:
You can’t perform that action at this time.