Sample Collection (Lab 1)

Our goal in this section is to gather data from our Fulfillment Center (FC) and store it as a dataset that can later be used to train machine learning models.

Create an S3 bucket to store the samples in

Let's work backwards by first creating a place to store our data. We will use S3 as our datastore because it is well-equipped to handle very large datasets. Later in the workshop we will discover that SageMaker is well integrated with S3, making it easy to train ML models.

Instructions

1. Navigate to S3 from the AWS home page.
2. Select the + Create bucket button in the top-left.
3. Type in aim368-samples-bucket-<FIRSTNAME>-<LASTNAME> (e.g. aim368-samples-bucket-mike-calder)

⚠️ Please follow all naming instructions: Failure to do so could prevent parts of the lab from working!

4. Click the Create button in the bottom-left of the pane.
5. Verify the bucket was created by finding it in the list.

Create a Kinesis Firehose to automatically batch the samples

We need to get our data from the FC to our new S3 Bucket, but the FC is publishing the data on a sample by sample basis. It would not make sense to create a new S3 file (or update a file) for every single sample, so we need an aggregation step. To accomplish this, we can create a Kinesis Firehose that automatically aggregates our data into batches and only writes to S3 after the batch size is reached or after a set amount of time.

Instructions

1. Navigate to Kinesis from the AWS home page.
2. Select the Get started button in the top-middle area.
3. In the Kinesis Firehose box, click Create delivery stream.
4. Enter SampleCollectionFirehose for the delivery stream name.
5. Click the Next button in the bottom-right.
6. Click the Next button in the bottom-right again.
7. Under S3 destination, select your bucket and click Next again.
8. Under Permissions click Create new or choose.
9. Select SampleCollectionFirehoseRole for the IAM Role.
10. Select SampleCollectionFirehosePolicy for the Policy Name.
11. Click the Allow button in the bottom-right.
12. Click the Next button in the bottom-right again.
13. Select Create delivery stream in the bottom-right.
14. Verify the Kinesis Firehose has started creating.
15. Verify the Kinesis Firehose was created.

Create a Lambda function to connect the SNS topic to the Firehose

Finally, we need to add a serverless compute step to read the data notifications from the FC and write the data to the Kinesis Firehose. We will use a Lambda with an SNS trigger to accomplish this.

Instructions

1. Navigate to Lambda from the AWS home page.
2. Select the Create function button in the top-right.
3. Enter SampleCollectionLambdaFunction for the function name.
4. Select Python 3.8 in the dropdown for the runtime.
5. Click Choose or create an execution role.
6. Click the Use an existing role radio button.
7. Select SampleCollectionLambdaRole in the dropdown.
8. Click the Create function button in the bottom-right.
9. Verify the function has been created.
10. Copy-paste the following code into the Function code box: (replacing the sample code)

    import boto3
    
    firehose = boto3.client('firehose')
    
    def lambda_handler(event, context):
        sample = event['Records'][0]['Sns']['Message']
        firehose.put_record(
            DeliveryStreamName = 'SampleCollectionFirehose',
            Record = {'Data': sample.encode('UTF-8')}
        )

11. Click the Save button in the top-right corner.
12. In the top-left, click the + Add trigger button.
13. Select SNS in the trigger configuration dropdown.
14. Click the Add button in the bottom-right.
15. Verify the trigger has been added.

Model Training (Lab 2)

In this module, we will start by loading our data into a SageMaker Notebook for analysis. Then we will train ML models, choose the best model, and finally deploy it, all using SageMaker managed hardware.

Create a SageMaker Notebook

SageMaker Notebooks provides familiar tooling via Jupyter Notebooks running on cloud hardware. Let's first set up a notebook so we can import our data and train some ML models.

Instructions

1. Navigate to Amazon SageMaker from the AWS home page.
2. Select Notebook Instances in the left panel under Notebooks.
3. Click the Create notebook instance button in the top-right.
4. Enter ModelTrainingNotebook for the notebook instance name.
5. Select ml.c5.2xlarge for the notebook instance type.
6. Click the Create notebook instance button in the bottom-right.
7. Verify the Amazon SageMaker notebook has started creating.
8. Wait for the notebook to have the InService status. (3-5 minutes)

Train models in the SageMaker Notebook

Now that the SageMaker Notebook is set up, we can import our data and leverage built-in SageMaker training algorithms to generate ML models. No python or data science experience necessary, just follow the steps to import our notebook files. These files will walk you through some basic data analytics and help you train ML models with three different algorithms.

Instructions

1. Click Open JupyterLab in the right-most column.
2. Select the git clone logo in the top-right of the middle pane.
3. Enter https://github.com/mike-calder/AIM368-Notebooks.git and hit CLONE.

⚠️ There will be no visual indication that the clone is progressing, but it should complete within a minute.

4. Double-click the AIM368-Notebooks folder in the top-left.

5. Double-click the Data-Analysis.ipynb file in the top-left.

6. Walk through the notebook by typing Shift+Enter on each individual cell.

7. Repeat this process for each of the three model training notebooks:

   • K-Nearest-Neighbors.ipynb
   • Linear-Learner.ipynb
   • XGBoost.ipynb

   

Deploy a model from the SageMaker Notebook

Now that we have trained some models, how do we use them to predict the duration of future human-robot interactions? For that, we need to deploy a model. Luckily, all we need is one line of code and we can deploy our model to a SageMaker managed cloud instance that's ready to recieve inference requests.

Instructions

1. Of the three models you trained, choose one to deploy for the accuracy competition.

2. Below that model's training output, create a new notebook cell with the following code:

   model.deploy(initial_instance_count = 1,
                instance_type = 'ml.c5.2xlarge',
                endpoint_name = 'LiveInferenceEndpoint')

3. Deploy the model by pressing Shift+Enter in the new cell.

Live Inference (Lab 3)

In this module we will create a live inference API that routes requests to our SageMaker endpoint. This will allow our FC edge compute resources to call the ML model and receive duration estimates for upcoming human-robot interactions.

Create a Lambda function to handle API Gateway requests

Once again let's work backwards by first creating a Lambda to call our SageMaker endpoint. In a production situation, this would allow us to test our model integration before exposing it to traffic from the FC.

Instructions

1. Navigate to Lambda from the AWS home page.
2. Select the Create function button in the top-right.
3. Enter LiveInferenceLambdaFunction for the function name.
4. Select Python 3.8 in the dropdown for the runtime.
5. Click Choose or create an execution role.
6. Click the Use an existing role radio button.
7. Select LiveInferenceLambdaRole in the dropdown.
8. Click the Create function button in the bottom-right.
9. Verify the function has been created.
10. Copy-paste the following code into the Function code box: (replacing the sample code)

    import boto3
    
    sagemaker = boto3.client('sagemaker-runtime')
    
    def lambda_handler(event, context):
        features = event['queryStringParameters']['features']
        response = sagemaker.invoke_endpoint(EndpointName = 'LiveInferenceEndpoint',
                                             ContentType = 'text/csv',
                                             Body = features)
        
        inference = response['Body'].read().decode()
        
        return {
            'statusCode': 200,
            'body': inference
        }

11. Click the Save button in the top-right corner.
12. Verify that the function updated successfully.

Create an API Gateway endpoint that can receive inference requests

Now that we have a Lambda capable of calling our ML model, we need to provide a way for the FC to trigger the lambda with new data. API Gateway will allow us to set up an external API that recieves data and routes it to the Lambda we created. Then the Lambda can call the ML model and return the estimated duration back to the caller.

Instructions

1. Navigate to API Gateway from the AWS home page.
2. Click Get Started in the top-middle, then Ok to clear the popup.
3. Choose REST for the protocol type and select the New API radio button.
4. Type LiveInferenceAPI for the name, then click Create API in the bottom-right.
5. At the top of the page, click the Actions drop down and select Create Resource.
6. Type liveinference as the resource name, then click Create Resource.
7. At the top of the page, click the Actions drop down and select Create Method.
8. Click the drop down that appears, select GET, then click the check mark to confirm.
9. On the page that appears, check the checkbox to enable Use Lambda Proxy integration.
10. In the Lambda Function field, type LiveInferenceLambdaFunction and click Save.
11. Click Ok to confirm the new permissions for your Lambda function.
12. After a few moments you should see a diagram of your GET - Method Execution, click Method Request.
13. Click on URL Query String Parameters to expand the section and then + Add query string.
14. Type features in the box that says myQueryString, then click the check mark.
15. At the top of the page, click the Actions drop down and select Deploy API.
16. For Deployment Stage select [New Stage] to expand the section.
17. Type in <FIRSTNAME>-<LASTNAME> (e.g. mike-calder) for the stage name and click Deploy.
18. In the middle pane, click the arrow to expand your stage view.
19. Under your liveinference resource, click GET to open the method page.
20. In the right pane, copy the Invoke URL at the top. You will need this for the next section.

⚠️ Make sure this URL ends in liveinference, if it does not please ask for help!

Update the FC edge compute Lambda to call your endpoint

Next, we need to provide your API Gateway endpoint to the FC edge compute so it can begin to send inference requests. In this lab, the FC edge compute is simulated by a Lambda function. Follow the steps below to point that Lambda function at your API Gateway. Once completed, the simulated FC will start requesting inferences from your deployed model about 5 times per second.

Instructions

1. Navigate to Lambda from the AWS home page.
2. Select EdgeComputeLambdaFunction.
3. Scroll down to Environment Variables.
4. For the LIVE_INFERENCE_API_GATEWAY_URL, paste your URL from the previous section.
5. Click the Save button in the top-right.
6. Verify the function updated successfully. You should appear on the leaderboard within a couple of minutes.

Automated Retraining

This section will be used as a starting point for an automated retraining discussion. There are many different strategies in this space, but almost all of them begin with some form of performance monitoring.

Monitor performance with CloudWatch dashboards

When facing real world problems, it's important that the resulting solutions include a feedback loop via metrics and alarms. This will allow you to monitor your solution's performance and react to any errors or regressions. In this lab, we have set up a very simple dashboard to illustrate how CloudWatch can help fill this role.

Instructions

1. Navigate to CloudWatch from the AWS home page.
2. Select Dashboards in the top of the left pane.
3. Click on LiveInferenceDashboard in the list.
4. Monitor the performance of your ML application.