Lesson1_AWSML_Overview.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lesson1-AWSML-Overview.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "1Plkp_JsOq9Q",
        "bq4VmHjPpMOR",
        "Kj977UW3rph_",
        "a_Hn6rgoO_z6",
        "32z26FBzP7Yx",
        "BEnCuHS9dwrz",
        "VZDWHZqseWWv",
        "25Luji1Geomt",
        "1aXnAOugeo6U",
        "djWXK2seiNDW",
        "M4gg-gZRTEHn",
        "85hU7hzXKSCw",
        "u50fSNs-KaGc",
        "R8J2HtlNkSkw",
        "d3zO9gCx2mw8",
        "Tqf9CASngdxS",
        "5F7JuoWB9YMh",
        "6ANmS4rzgwMt",
        "nUiZr9xpzkqa",
        "ITlrgLKMzqHP",
        "7biG-GVVIGC6",
        "Afia2nad3Dfr",
        "YMSvoYJA6yhy",
        "2OgOsp_KCI9q",
        "5C5vF2VgCGRW",
        "WT2SdNwvCx-s"
      ],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/noahgift/aws-ml-guide/blob/master/Lesson1_AWSML_Overview.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iwV-bBLUtTQu",
        "colab_type": "text"
      },
      "source": [
        "# Lesson 1 AWS Machine Learning-Specialty (ML-S) Certification\n",
        "\n",
        "**Prepare for this material by first taking the [AWS Certified Cloud Practitioner 2020-Real World & Pragmatic Course](https://www.udemy.com/course/aws-certified-cloud-practitioner-2020-real-world-pragmatic/?referralCode=CAC679A7D08212773428)**\n",
        "\n",
        "[Watch Lesson 1:  AWS Machine Learning-Speciality (MLS) Video](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_00)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c_Id55m6Jsbu",
        "colab_type": "text"
      },
      "source": [
        "## Pragmatic AI Labs\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e5p96AqpSDZa",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://paiml.com/images/logo_with_slogan_white_background.png)\n",
        "\n",
        "This notebook was produced by [Pragmatic AI Labs](https://paiml.com/).  You can continue learning about these topics by:\n",
        "\n",
        "*   Buying a copy of [Pragmatic AI: An Introduction to Cloud-Based Machine Learning](http://www.informit.com/store/pragmatic-ai-an-introduction-to-cloud-based-machine-9780134863863) from Informit.\n",
        "*   Buying a copy of  [Pragmatic AI: An Introduction to Cloud-Based Machine Learning](https://www.amazon.com/Pragmatic-AI-Introduction-Cloud-Based-Learning/dp/0134863860) from Amazon\n",
        "*   Reading an online copy of [Pragmatic AI:Pragmatic AI: An Introduction to Cloud-Based Machine Learning](https://www.safaribooksonline.com/library/view/pragmatic-ai-an/9780134863924/)\n",
        "*  Watching video [Essential Machine Learning and AI with Python and Jupyter Notebook-Video-SafariOnline](https://www.safaribooksonline.com/videos/essential-machine-learning/9780135261118) on Safari Books Online.\n",
        "* Watching video [AWS Certified Machine Learning-Speciality](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597)\n",
        "* Purchasing video [Essential Machine Learning and AI with Python and Jupyter Notebook- Purchase Video](http://www.informit.com/store/essential-machine-learning-and-ai-with-python-and-jupyter-9780135261095)\n",
        "*   Viewing more content at [noahgift.com](https://noahgift.com/)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Plkp_JsOq9Q",
        "colab_type": "text"
      },
      "source": [
        "## 1.1 Certification Overview\n",
        "\n",
        "[Watch 1.1 Certification Overview Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_01)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bq4VmHjPpMOR",
        "colab_type": "text"
      },
      "source": [
        "### Load AWS API Keys"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aWrzIk7WpRoh",
        "colab_type": "text"
      },
      "source": [
        "Put keys in local or remote GDrive:  \n",
        "\n",
        "`cp ~/.aws/credentials /Users/myname/Google\\ Drive/awsml/`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hPWO_zyRopXN",
        "colab_type": "text"
      },
      "source": [
        "#### Mount GDrive\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XI73HZNLobp4",
        "colab_type": "code",
        "outputId": "024a06d8-2af1-401a-f488-83d0dc028baa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UNyzZwgmoxwm",
        "colab_type": "code",
        "outputId": "13a1dbee-7a6a-43f1-bd41-53920f78e597",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "import os;os.listdir(\"/content/gdrive/My Drive/awsml\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['kaggle.json', 'credentials', 'config']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fYu0ekUlqPk6",
        "colab_type": "text"
      },
      "source": [
        "#### Install Boto"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dJDDrUkWrYRY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip -q install boto3\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FpJhrpSQsK5E",
        "colab_type": "text"
      },
      "source": [
        "#### Create API Config"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QxRwGOZtsN0-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!mkdir -p ~/.aws &&\\\n",
        "  cp /content/gdrive/My\\ Drive/awsml/credentials ~/.aws/credentials "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kj977UW3rph_",
        "colab_type": "text"
      },
      "source": [
        "#### Test Comprehend API Call"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P-A8Cia-raT0",
        "colab_type": "code",
        "outputId": "46e71d25-6835-4850-b40b-cb0db9b5d50e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "import boto3\n",
        "comprehend = boto3.client(service_name='comprehend', region_name=\"us-east-1\")\n",
        "text = \"There is smoke in San Francisco\"\n",
        "comprehend.detect_sentiment(Text=text, LanguageCode='en')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\n",
              "   'content-length': '160',\n",
              "   'content-type': 'application/x-amz-json-1.1',\n",
              "   'date': 'Tue, 11 Dec 2018 01:07:46 GMT',\n",
              "   'x-amzn-requestid': '2b726af9-fce1-11e8-a4e1-2d227516436e'},\n",
              "  'HTTPStatusCode': 200,\n",
              "  'RequestId': '2b726af9-fce1-11e8-a4e1-2d227516436e',\n",
              "  'RetryAttempts': 0},\n",
              " 'Sentiment': 'NEUTRAL',\n",
              " 'SentimentScore': {'Mixed': 0.008628507144749165,\n",
              "  'Negative': 0.1037612184882164,\n",
              "  'Neutral': 0.8582549691200256,\n",
              "  'Positive': 0.0293553676456213}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_Hn6rgoO_z6",
        "colab_type": "text"
      },
      "source": [
        "### Audience\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p6gltgZ4aCXi",
        "colab_type": "text"
      },
      "source": [
        "Intended for individuals who perform Development or Data Science role"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1-ss6GUEQr2G",
        "colab_type": "text"
      },
      "source": [
        "### Validates ability to:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5nMOxGJ6QyD3",
        "colab_type": "text"
      },
      "source": [
        "* \"design, implement, deploy and maintain ML solutions\"\n",
        "* \"select and justify appropriate ML approaches\"\n",
        "* \"indentify appropriate AWS services to implement ML solutions\"\n",
        "* \"Design and implement scalable, cost-optimized, reliable, and secure ML solutions\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fNvM_qt9Tf_9",
        "colab_type": "text"
      },
      "source": [
        "### Recommended AWS Knowledge"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u6OQZt9yYjQQ",
        "colab_type": "text"
      },
      "source": [
        "* Hands-on experience:\n",
        "\n",
        " - developing\n",
        " - architecting\n",
        " - running ML/deep learning workloads on AWS Cloud\n",
        " \n",
        "* Ability to express intuition behind basic ML algorithms\n",
        "* Experience performance basic hyperparameter optimization\n",
        "* Experience with ML and deep learning frameworks\n",
        "* Ability to follow model training best practices\n",
        "* Ability to follow deployment and operational best practices"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zi90QSTFP09x",
        "colab_type": "text"
      },
      "source": [
        "## 1.2 Exam Study Resources\n",
        "\n",
        "[Watch 1.2 Exam Study Resources Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_02)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "32z26FBzP7Yx",
        "colab_type": "text"
      },
      "source": [
        "### Official AWS Material"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ELmnS7-X9OUg",
        "colab_type": "text"
      },
      "source": [
        "![Sequential Split](https://user-images.githubusercontent.com/58792/49745472-de7cb500-fc53-11e8-8dd8-1fb2075d1373.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CVrF3Jt5auYw",
        "colab_type": "text"
      },
      "source": [
        "* [AWS ML University](https://aws.amazon.com/training/learning-paths/machine-learning/)\n",
        "* [Machine Learning on AWS](https://aws.amazon.com/machine-learning/)\n",
        "* [Amazon Machine Learning Concepts](https://docs.aws.amazon.com/machine-learning/latest/dg/amazon-machine-learning-key-concepts.html)\n",
        "* [Splitting Your Data](https://docs.aws.amazon.com/machine-learning/latest/dg/splitting-types.html)\n",
        "* [Types of Machine Learning Models](https://docs.aws.amazon.com/machine-learning/latest/dg/types-of-ml-models.html)\n",
        "* [Data Transformations](https://docs.aws.amazon.com/machine-learning/latest/dg/data-transformations-reference.html)\n",
        "* [Containerized Machine Learning on AWS Video](https://www.youtube.com/watch?v=Jw9CpQkCvpM)\n",
        "* [AWS re:Invent Machine Learning Talk 2017](https://www.youtube.com/watch?v=Q7N2iVfgA0U&list=PLOS8MNIsAkpJCaOSQP6t5uoVtwUr0SFvM)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "BEnCuHS9dwrz"
      },
      "source": [
        "## 1.3 Review Exam Guide\n",
        "\n",
        "[Watch 1.3 Review Exam Guide Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_03)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VAbktr8VEnOa",
        "colab_type": "text"
      },
      "source": [
        "[AWS Certified Machine Learning–Specialty (MLS-C01) Exam Guide ](https://d1.awsstatic.com/training-and-certification/machinelearning/AWS%20Certified%20Machine%20Learning%20-%20Specialty_Exam%20Guide_v1.1_FINAL.pdf)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_MaUFpeYfi7N",
        "colab_type": "text"
      },
      "source": [
        "### Domains Covered"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZDWHZqseWWv",
        "colab_type": "text"
      },
      "source": [
        "#### Data Engineering"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FRiNV06zem2l",
        "colab_type": "text"
      },
      "source": [
        "* Create data repositories for Machine Learning\n",
        "* Identify and implement a data-ingestion solution\n",
        "* Identify and implement a data-transformation solution\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "25Luji1Geomt"
      },
      "source": [
        "#### Exploratory Data Analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uyJ1W9rFerwC",
        "colab_type": "text"
      },
      "source": [
        "* Sanitize and prepare data modeling\n",
        "* Perform feature engineering\n",
        "* Analyze and visualize data for machine learning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "1aXnAOugeo6U"
      },
      "source": [
        "#### Modeling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "vGxJjZRFeomv"
      },
      "source": [
        "\n",
        "\n",
        "*   Frame business problems as machine learning problems\n",
        "*   Select the appropriate model(s) for a given machine learning problem\n",
        "* Train machine learning models\n",
        "* Perform hyperparameter optimization\n",
        "* Evaluate machine learning models\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "djWXK2seiNDW",
        "colab_type": "text"
      },
      "source": [
        "#### Machine Learning Implementation and Operations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GCzIR17CiNWT",
        "colab_type": "text"
      },
      "source": [
        "*   Build machine learning solutions for performance, availability, scalability, resiliency, and fault tolerance.\n",
        "*  Recommend and implement the appropriate machine learning services and features for a given problem.\n",
        "* Apply basic AWS security practices to machine learning solutions.\n",
        "* Deploy and operationalize machine learning solutions.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wHzy9ZUMgW75",
        "colab_type": "text"
      },
      "source": [
        "## 1.4 Exam Strategy\n",
        "\n",
        "[Watch 1.4 Exam Strategy Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_04)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M4gg-gZRTEHn",
        "colab_type": "text"
      },
      "source": [
        "### Official Exam Study Guide"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YuWXy-1wT2Hu",
        "colab_type": "text"
      },
      "source": [
        "![Study Path](https://user-images.githubusercontent.com/58792/49443332-06b27280-f781-11e8-8ec2-88af4d47724e.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5F6qIpnTgdDb",
        "colab_type": "text"
      },
      "source": [
        "## 1.5 Best Practices of ML on AWS\n",
        "\n",
        "[Watch 1.5 Best Practices of ML on AWS Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_05)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hat17x80OVRW",
        "colab_type": "text"
      },
      "source": [
        "### Build vs Buy\n",
        "\n",
        "![hero](https://user-images.githubusercontent.com/58792/45260894-08e67a00-b3a8-11e8-941f-e539cb10f8e1.jpg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "clBbP1yD182S",
        "colab_type": "text"
      },
      "source": [
        "### Overview\n",
        "\n",
        "Auto and Managed ML Services on AWS \n",
        "\n",
        "*   AWS Sagemaker\n",
        "*   AWS Machine Learning\n",
        "\n",
        "Human in the Loop\n",
        "\n",
        "* AWS Sagemaker Ground Truth\n",
        "* AWS Mechanical Turk\n",
        "\n",
        "Supporting Technologies\n",
        "\n",
        "* MXNet\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "85hU7hzXKSCw",
        "colab_type": "text"
      },
      "source": [
        "### AWS Sagemaker"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ptFIFj5KZpz",
        "colab_type": "text"
      },
      "source": [
        "[Sagemaker Features](https://aws.amazon.com/sagemaker/features/)\n",
        "\n",
        "![AWS Sagemaker](https://user-images.githubusercontent.com/58792/45426785-09a23900-b652-11e8-9ba6-4ff58a8103d5.png)\n",
        "\n",
        "\n",
        "*   Build\n",
        "\n",
        " - Preconfigured Jupyter Notebooks\n",
        " - Built in, High Performance Algorithms (Optimized petabyte-scale peformance)\n",
        " - Supports many frameworks:  MXnet, Tensorflow, sklearn\n",
        "\n",
        "\n",
        "*   Train\n",
        "\n",
        " - One click train with S3 target\n",
        " - Automatic hyperparameter tuning\n",
        " - Can scale training to clusters of machines with single line of code\n",
        "\n",
        "*   Deploy\n",
        "\n",
        "  - One click Deploy (creates HTTPS endpoint)\n",
        "  - Automatic A/B Testing\n",
        "  - Fully managed auto-scaling of inference\n",
        "  \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u50fSNs-KaGc",
        "colab_type": "text"
      },
      "source": [
        "### AWS Machine Learning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rhoSapkikDyh",
        "colab_type": "text"
      },
      "source": [
        "[Amazon Machine Learning Service](https://aws.amazon.com/aml/)\n",
        "\n",
        "![alt text](https://user-images.githubusercontent.com/58792/45425471-53892000-b64e-11e8-8895-794014ce3702.png)\n",
        "\n",
        "**Key Features**\n",
        "\n",
        "\n",
        "*   \" makes it easy for developers of all skill levels to use machine learning technology\"\n",
        "*   Managaged ML Service\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "**Resources**\n",
        "*   [Introduction to AWS Machine Learning Video](https://youtu.be/PAHU8tPA7xs)\n",
        "*   [Kaggle + Amazon ML Service](https://aws.amazon.com/blogs/big-data/building-a-numeric-regression-model-with-amazon-machine-learning/)\n",
        "* [amazon ml dev guide](https://docs.aws.amazon.com/machine-learning/latest/dg/what-is-amazon-machine-learning.html)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R8J2HtlNkSkw",
        "colab_type": "text"
      },
      "source": [
        "### MXNet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EyYskTvYkSkx",
        "colab_type": "text"
      },
      "source": [
        "[Open Source Docs](https://gluon.mxnet.io/)\n",
        "\n",
        "![mxnet logo](https://user-images.githubusercontent.com/58792/45424774-5e42b580-b64c-11e8-90f3-1f022fbc35e7.png)\n",
        "\n",
        "\n",
        "*  Alternative to TensorFlow\n",
        "*  Amazon Backed\n",
        "*  Integrated into Sagemaker and DeepLense\n",
        "*  Has production-ready orchestration tools:  ECS, Docker, Kubernetes \n",
        "*  Can do[ inference via AWS Lambda](https://github.com/awslabs/mxnet-lambda) \n",
        "\n",
        "**Resources**\n",
        "\n",
        "* [Serving Machine Learning Models with Apache MXNet & AWS Fargate](https://www.youtube.com/watch?v=YlUoa3hLz78)\n",
        "* [mxnet-lambda](https://github.com/awslabs/mxnet-lambda)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VNOE6ux82c48",
        "colab_type": "text"
      },
      "source": [
        "### Demo AWS Sagemaker"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hksa_zjN2c49",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d3zO9gCx2mw8",
        "colab_type": "text"
      },
      "source": [
        "### Demo AWS ML Service"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zjSJwCj22mw-",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0XPffMjwgdlP",
        "colab_type": "text"
      },
      "source": [
        "#### Build vs Buy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dwhY-5XaJVxt",
        "colab_type": "text"
      },
      "source": [
        "* When to use AI APIs vs ML Platforms"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tqf9CASngdxS",
        "colab_type": "text"
      },
      "source": [
        "## 1.6 Techniques to accelerate hands-on practice\n",
        "\n",
        "[Watch 1.6 Techniques to accelerate hands-on practice Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_06)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ssCPRgECgv9M",
        "colab_type": "text"
      },
      "source": [
        "### Using Interactive QWIKLABS\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K3XgRBEQOr5L",
        "colab_type": "text"
      },
      "source": [
        "A temporary, but real AWS environment that teaches specific labs.\n",
        "\n",
        "![QWiklabs](https://user-images.githubusercontent.com/58792/48742665-b2ea4a00-ec14-11e8-8805-0600f60e637b.png)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ASt4qc7fNsyB",
        "colab_type": "text"
      },
      "source": [
        "#### Big Data on AWS\n",
        "\n",
        "[Big Data on AWS Quest](https://run.qwiklabs.com/quests/5)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gqvsJ3vNWuxk",
        "colab_type": "text"
      },
      "source": [
        "#### Intro to Machine Learning \n",
        "[Introduction to Machine Learning](https://run.qwiklabs.com/focuses/275?parent=catalog)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5F7JuoWB9YMh",
        "colab_type": "text"
      },
      "source": [
        "### Using AWS Console, Sagemaker and APIS\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oVNpPPRg9fVt",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "*   Console to Explore\n",
        "*   Sagemaker\n",
        "*   APIs via Boto\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6ANmS4rzgwMt",
        "colab_type": "text"
      },
      "source": [
        "## 1.7 Understand important ML related services\n",
        "\n",
        "[Watch 1.7 Understand important ML related services Video Lesson](https://learning.oreilly.com/videos/aws-certified-machine/9780135556597/9780135556597-ACML_01_01_07)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fah6T6emIyUe",
        "colab_type": "text"
      },
      "source": [
        "### ML Platform Services"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nUiZr9xpzkqa",
        "colab_type": "text"
      },
      "source": [
        "#### Amazon Sagemaker"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FZYG0GkdzlEJ",
        "colab_type": "text"
      },
      "source": [
        "**[Demo] Overview Console**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ITlrgLKMzqHP",
        "colab_type": "text"
      },
      "source": [
        "#### AWS DeepLens"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q7m3P7VtTIwp",
        "colab_type": "text"
      },
      "source": [
        "![alt text](https://user-images.githubusercontent.com/58792/45308029-688e7380-b4d5-11e8-8ffb-9422184c274c.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kNlxPx5tzqPv",
        "colab_type": "text"
      },
      "source": [
        "**[Demo] Overview Console**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7biG-GVVIGC6",
        "colab_type": "text"
      },
      "source": [
        "#### EMR"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UnQk77SXIUC8",
        "colab_type": "text"
      },
      "source": [
        "* [Amazon EMR](https://aws.amazon.com/emr/?nc2=type_a)\n",
        "* Managed Hadoop/Spark\n",
        "* Use cases:\n",
        "\n",
        "  - clickstream analysis\n",
        "  - real-time analytics\n",
        "  - log analysis\n",
        "  - ETL\n",
        "  - predictive analytics\n",
        "  - genomics\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cmc_bK0O-GLY",
        "colab_type": "text"
      },
      "source": [
        "**[Demo] Overview Console**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YVeraFFMIk2X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "feIMA0rZI0Ln",
        "colab_type": "text"
      },
      "source": [
        "#### Amazon Machine Learning Service"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lTkcPdGRI3mn",
        "colab_type": "text"
      },
      "source": [
        "* [Amazon Machine Learning](https://aws.amazon.com/aml/?nc2=type_a)\n",
        "* Provides visualization and wizards to help create ML models\n",
        "* AutoML technology"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2zexMLPD-HwQ",
        "colab_type": "text"
      },
      "source": [
        "**[Demo] Overview Console**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vuo7p0MaI2yn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JwuIMAsl3Dfo"
      },
      "source": [
        "### Deep Learning on AWS"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "a781nJ5a3Dfq"
      },
      "source": [
        "#### AWS Deep Learning AMIs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GmUsOy9f3Dfr"
      },
      "source": [
        "* [AWS Deep Learning AMIs](https://aws.amazon.com/machine-learning/amis/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Afia2nad3Dfr"
      },
      "source": [
        "#### Apache MXNet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-Jqn-Scd3Dfs"
      },
      "source": [
        "\n",
        "\n",
        "*   [Apache MXNet on AWS](https://aws.amazon.com/machine-learning/amis/)\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YMSvoYJA6yhy"
      },
      "source": [
        "#### TensorFlow on AWS"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Ssjheq6T6yhz"
      },
      "source": [
        "* [TensorFlow on AWS](https://aws.amazon.com/tensorflow/)\n",
        "* [88% of TensorFlow workloads are on AWS](https://d1.awsstatic.com/whitepapers/nucleus-tensorflow.pdf)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cZivE3wYAi8W",
        "colab_type": "text"
      },
      "source": [
        "### AI APIs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2OgOsp_KCI9q",
        "colab_type": "text"
      },
      "source": [
        "#### Vision Services"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "edSjR5ZWAr8B",
        "colab_type": "text"
      },
      "source": [
        "##### Rekognition Image"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U0I36-hkBC_o",
        "colab_type": "text"
      },
      "source": [
        "* Deep learning-based image analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "unO3Xs6mBGVM",
        "colab_type": "text"
      },
      "source": [
        "##### Rekognition Video"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qWEufCj5BG02",
        "colab_type": "text"
      },
      "source": [
        "* Deep learning-based video analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5C5vF2VgCGRW",
        "colab_type": "text"
      },
      "source": [
        "#### Conversational chatbots"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kEtDUKkECGbF",
        "colab_type": "text"
      },
      "source": [
        "##### Amazon Lex"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TewlJ3c_Cuun",
        "colab_type": "text"
      },
      "source": [
        "* [Amazon Lex](https://aws.amazon.com/lex/)\n",
        "* Service for building conversational interfaces"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WT2SdNwvCx-s",
        "colab_type": "text"
      },
      "source": [
        "#### Language Services"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R7ZM8x0YC20l",
        "colab_type": "text"
      },
      "source": [
        "##### Comprehend"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SzXeFzbaC8Mw",
        "colab_type": "text"
      },
      "source": [
        "* [AWS Comprehend](https://aws.amazon.com/comprehend/)\n",
        "* NLP service\n",
        "  - Sentiment Analysis\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ohQ8D7gw6VSf",
        "colab_type": "text"
      },
      "source": [
        "**Neutral Sentiment**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NoVwmOrg4N-4",
        "colab_type": "code",
        "outputId": "0fbefb18-2f3b-49f2-e6a4-f6c34253e034",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "import boto3\n",
        "comprehend = boto3.client(service_name='comprehend', region_name=\"us-east-1\")\n",
        "text = \"There is smoke in San Francisco\"\n",
        "comprehend.detect_sentiment(Text=text, LanguageCode='en')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\n",
              "   'content-length': '160',\n",
              "   'content-type': 'application/x-amz-json-1.1',\n",
              "   'date': 'Tue, 11 Dec 2018 01:07:59 GMT',\n",
              "   'x-amzn-requestid': '3368fdfa-fce1-11e8-8656-5735cedac40a'},\n",
              "  'HTTPStatusCode': 200,\n",
              "  'RequestId': '3368fdfa-fce1-11e8-8656-5735cedac40a',\n",
              "  'RetryAttempts': 0},\n",
              " 'Sentiment': 'NEUTRAL',\n",
              " 'SentimentScore': {'Mixed': 0.008628507144749165,\n",
              "  'Negative': 0.1037612184882164,\n",
              "  'Neutral': 0.8582549691200256,\n",
              "  'Positive': 0.0293553676456213}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R4LrAZbC6ZT2",
        "colab_type": "text"
      },
      "source": [
        "**Negative Sentiment**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7yhXA8g36S-j",
        "colab_type": "code",
        "outputId": "5b578e62-ce5c-448e-919c-da0d01649981",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "import boto3\n",
        "comprehend = boto3.client(service_name='comprehend', region_name=\"us-east-1\")\n",
        "text = \"There is smoke in San Francisco and it makes me very angry\"\n",
        "comprehend.detect_sentiment(Text=text, LanguageCode='en')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\n",
              "   'content-length': '164',\n",
              "   'content-type': 'application/x-amz-json-1.1',\n",
              "   'date': 'Tue, 11 Dec 2018 01:08:17 GMT',\n",
              "   'x-amzn-requestid': '3e127c6f-fce1-11e8-b936-d1be599a3739'},\n",
              "  'HTTPStatusCode': 200,\n",
              "  'RequestId': '3e127c6f-fce1-11e8-b936-d1be599a3739',\n",
              "  'RetryAttempts': 0},\n",
              " 'Sentiment': 'NEGATIVE',\n",
              " 'SentimentScore': {'Mixed': 0.00937745813280344,\n",
              "  'Negative': 0.9539545774459839,\n",
              "  'Neutral': 0.03619137033820152,\n",
              "  'Positive': 0.00047663392615504563}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KiZJ2byCzxyk",
        "colab_type": "code",
        "outputId": "442c88c2-b364-40df-c27a-715eff5badac",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "trump_text = \"\"\"\n",
        "which it was not (but even if it was, it is only a CIVIL CASE, like Obama’s - but it was done correctly by a lawyer and there would not even be a fine. Lawyer’s liability if he made a mistake, not me). Cohen just trying to get his sentence reduced. WITCH HUNT!\n",
        "\n",
        "\"\"\"\n",
        "comprehend = boto3.client(service_name='comprehend', region_name=\"us-east-1\")\n",
        "comprehend.detect_sentiment(Text=trump_text,LanguageCode='en')\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\n",
              "   'content-length': '160',\n",
              "   'content-type': 'application/x-amz-json-1.1',\n",
              "   'date': 'Tue, 11 Dec 2018 01:10:42 GMT',\n",
              "   'x-amzn-requestid': '94b5be74-fce1-11e8-92e1-212d40a34fc0'},\n",
              "  'HTTPStatusCode': 200,\n",
              "  'RequestId': '94b5be74-fce1-11e8-92e1-212d40a34fc0',\n",
              "  'RetryAttempts': 0},\n",
              " 'Sentiment': 'NEGATIVE',\n",
              " 'SentimentScore': {'Mixed': 0.0772324725985527,\n",
              "  'Negative': 0.776195228099823,\n",
              "  'Neutral': 0.10541720688343048,\n",
              "  'Positive': 0.04115508496761322}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l5t2q8wj6q_Q",
        "colab_type": "text"
      },
      "source": [
        "**Positive Sentiment**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6hcW6Dcy6uSm",
        "colab_type": "code",
        "outputId": "7fb8c3b4-10d9-4298-d91f-9c821dddb783",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "import boto3\n",
        "comprehend = boto3.client(service_name='comprehend', region_name=\"us-east-1\")\n",
        "text = \"There is no more smoke in San Francisco and it makes me very happy\"\n",
        "comprehend.detect_sentiment(Text=text, LanguageCode='en')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\n",
              "   'content-length': '163',\n",
              "   'content-type': 'application/x-amz-json-1.1',\n",
              "   'date': 'Mon, 10 Dec 2018 22:41:39 GMT',\n",
              "   'x-amzn-requestid': 'c25e7c7c-fccc-11e8-b966-5f330c378b33'},\n",
              "  'HTTPStatusCode': 200,\n",
              "  'RequestId': 'c25e7c7c-fccc-11e8-b966-5f330c378b33',\n",
              "  'RetryAttempts': 0},\n",
              " 'Sentiment': 'POSITIVE',\n",
              " 'SentimentScore': {'Mixed': 0.01339163538068533,\n",
              "  'Negative': 0.007987627759575844,\n",
              "  'Neutral': 0.04016350954771042,\n",
              "  'Positive': 0.9384573101997375}}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PSyy8s7rC9dt",
        "colab_type": "text"
      },
      "source": [
        "##### Translate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PZsH7AahFDcQ",
        "colab_type": "text"
      },
      "source": [
        "* [Amazon Translate](https://aws.amazon.com/translate/?nc2=type_a)\n",
        "* Natural and accurate language translation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rSv8zQ5Q69RW",
        "colab_type": "code",
        "outputId": "a54ec069-ee0b-45a1-ddbe-36be027ca7d1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import boto3\n",
        "client = boto3.client('translate', region_name=\"us-east-1\")\n",
        "text = \"Hola, mi nombre es Noah y me encanta el aprendizaje automático.\"\n",
        "client.translate_text(Text=text,SourceLanguageCode=\"auto\", TargetLanguageCode=\"en\")\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'ResponseMetadata': {'HTTPHeaders': {'connection': 'keep-alive',\n",
              "   'content-length': '124',\n",
              "   'content-type': 'application/x-amz-json-1.1',\n",
              "   'date': 'Tue, 11 Dec 2018 01:11:05 GMT',\n",
              "   'x-amzn-requestid': 'a1dd350d-fce1-11e8-88f8-15e276ec8e51'},\n",
              "  'HTTPStatusCode': 200,\n",
              "  'RequestId': 'a1dd350d-fce1-11e8-88f8-15e276ec8e51',\n",
              "  'RetryAttempts': 0},\n",
              " 'SourceLanguageCode': 'es',\n",
              " 'TargetLanguageCode': 'en',\n",
              " 'TranslatedText': 'Hello, my name is Noah and I love machine learning.'}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JO9KhxkxF0Xq",
        "colab_type": "text"
      },
      "source": [
        "##### Transcribe"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eDo_M92dF7ml",
        "colab_type": "text"
      },
      "source": [
        "* [Amazon Transcribe](https://aws.amazon.com/transcribe/)\n",
        "* Automatic speech recognition\n",
        "* [Translate Tutorial](https://aws.amazon.com/getting-started/tutorials/create-audio-transcript-transcribe/)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7gpcmdPg_PFt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import boto3\n",
        "s3_path = \"https://s3.amazonaws.com/pai-transcribe/transcribe-sample.dac1d22492611d998262c8c856b98a74180a1a8f.mp3\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xHN6TRiwF3yf",
        "colab_type": "text"
      },
      "source": [
        "##### Polly\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WFmk44OLF8N8",
        "colab_type": "text"
      },
      "source": [
        "* [Amazon Polly](https://aws.amazon.com/polly/)\n",
        "* Turns text into lifelike speech"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vSztPsqL1yv3",
        "colab_type": "text"
      },
      "source": [
        ""
      ]
    }
  ]
}