Skip to content

Commit

Permalink
Create setup function, trainer-script updates, lambda-function skeleton.
Browse files Browse the repository at this point in the history
  • Loading branch information
jgreenemi committed Dec 19, 2017
1 parent 773f748 commit fbf63e7
Show file tree
Hide file tree
Showing 6 changed files with 111 additions and 0 deletions.
File renamed without changes.
5 changes: 5 additions & 0 deletions docs/CONFIGURATION.md
Expand Up @@ -28,3 +28,8 @@ Do note that the `cost-limit` parameter is an estimation, and **is not a guarant
}
```

## `training-script-filename` ##

This package comes pre-loaded with a `trainer-script.sh` script to get you started. The Lambda function will find this file in the package and feed it into the CloudFormation template for you when invoked.

* Example: `trainer-script.sh`
7 changes: 7 additions & 0 deletions requirements.txt
@@ -0,0 +1,7 @@
boto3==1.5.2
botocore==1.8.16
docutils==0.14
jmespath==0.9.3
python-dateutil==2.6.1
s3transfer==0.1.12
six==1.11.0
74 changes: 74 additions & 0 deletions setup.py
@@ -0,0 +1,74 @@
# After setting up the config.json, lambda-function.py, and trainer-script.sh files,
# run this script to create the necessary Lambda function. Execution of the Lambda
# function will be triggered manually by you from the AWS CLI or web console, a separate
# program of yours via the AWS SDK, an IoT button, or whatever other trigger you make use of
# for firing off your algorithm's training session.

import boto3
import json
import logging


def lambda_creation(config={}):
"""
Create the lambda function, return the response, give the AWS CLI command to execute it.
:return:
"""
try:
return True

except Exception as e:
return False


def parse_config():
"""
Parse and return the config. This'll be a lot shorter than the _test function.
:return:
"""
try:
config = {}
return config

except Exception as e:
return False


def _test_lambda_creation():
"""
Parse and return the config. This'll be a lot shorter than the _test function.
:return:
"""
try:
return True

except Exception as e:
return False


def _test_parse_config():
"""
Try parsing the JSON config. Check for file existence, valid parsing, and of course
that valid inputs are configured. (Numerical inputs for time and cost, for example.)
:return:
"""
try:
return True

except Exception as e:
return False


if __name__ == '__main__':
"""
Parse the config, craft a CloudFormation template, create the Lambda function.
"""

parse_config_test_result = _test_parse_config()
lambda_creation_test_result = _test_lambda_creation()

# If the test have all passed, go ahead with function creation.

lambda_creation(config=parse_config())

logging.info('setup.py finished.')
Empty file added src/lambda-function.py
Empty file.
25 changes: 25 additions & 0 deletions src/trainer-script.sh
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# This script is what will run on the EC2 instance once it launches.
# You'll want to verify that this functions as expected before launching
# an expensive instance type with it, as simple typos can become costly
# if you're launching and terminating instances every time you have a
# bug.

# You'll want to include some kind of logging facility with this script
# in case things go wrong.

# TODO Does sudo need to be used to run any of the install/poweroff commands?

# Run setup of your training session. Your commands will invariably look different.
cd /tmp
yum install -y git
git clone https://github.com/jgreenemi/MXNet-Familiarity-Project.git
cd MXNet-Familiarity-Project
pip install -r requirements.txt

# Setup done - now run the training job.
python classifier\trainer.py

# If the script has completed, go ahead and turn off the server to eliminate any
# additional costs.
poweroff now

0 comments on commit fbf63e7

Please sign in to comment.