AIML merge content from public repo (#24)

* Scripts for demo videos

* Shorten AIML20 speaker notes for time

* AIML20 full-length recording

* Custom Vision demo script

* Delete settings.json

* Merge files from public repo

* Merge AIML20 READMEs

.gitignore

@@ -3,6 +3,9 @@
 ##
 ## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore
 
+# VS Code
+.vscode/
+
 # User-specific files
 *.suo
 *.user

aiml20/CV training images/ATTRIBUTIONS.md

@@ -0,0 +1,33 @@
+# Attributions for images in this folder
+
+All images are sourced from [Wikimedia Commons](https://commons.wikimedia.org/) and are used under their respective
+Creative Commons licenses.
+
+Images were selected manually for suitability as part of a custom vision model. Images not eligible for selection include: 
+- Line art
+- SVG files, video files
+- Images with multiple different objects or busy scenes
+- Images including people
+- Images with text or other annotations
+- File size exceeding 6Mb
+
+## Hammers
+
+https://commons.wikimedia.org/wiki/Category:Claw_hammers  
+https://commons.wikimedia.org/wiki/Category:Ball-peen_hammers
+
+## Pliers
+https://commons.wikimedia.org/wiki/Category:Long-nose_pliers 
+https://commons.wikimedia.org/wiki/Category:Needle-nose_pliers  
+https://commons.wikimedia.org/wiki/Category:Linesman%27s_pliers  
+
+## Screwdrivers
+
+https://commons.wikimedia.org/wiki/Category:Screwdrivers
+
+## Drills
+https://commons.wikimedia.org/wiki/Category:Cordless_drills
+
+## Hard Hats:
+https://commons.wikimedia.org/wiki/Category:Hard_hats
+

aiml20/DEMO Computer Vision.md

@@ -0,0 +1,137 @@
+# Using pre-built AI to understand images
+
+In this demonstration, we will use Azure Computer Vision to detect the type of
+object an image represents. 
+
+First, we will use the Computer Vision online web-form to upload an image and
+observe the results.
+
+Then, we will use the Computer Vision API to collect the same information
+programatically, using curl.
+
+## Defining the problem: Shop by Photo doesn't work right
+
+The problem that motivates this talk is that the Shop by Photo tool in the
+Tailwind Traders website isn't correctly identifying products. It's useful to
+run this section in [ONNX Deployment](DEMO%20ONNX%20deployment.md#defining-the-problem-shop-by-photo-doesnt-work-right) at this point
+to set the scene.
+
+## Using Computer Vision via the Web interface
+
+Let's try using computer vision on a picture of a hardware product. If we can
+identify a product that Tailwind Traders sells by name, we can search for that
+name in the catalog for the "Shop by Photo" app.
+
+1. Visit the Computer Vision webpage at
+   [https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/](https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/?WT.mc_id=msignitethetour2019-github-aiml20)
+
+2. Scroll down to the "Analyze an Image" section. It looks like this:
+
+!["Computer Vision: Analyze an Image"](img/Computer%20Vision%20Analyze%20an%20Image.png)
+
+3. Click the "Browse" button, and choose "man in hardhat.jpg" from the "test
+   images" folder in "CV Training Images".
+
+4. After a moment, the analysis of your image will appear in the right pane. It
+   looks like this:
+
+```
+FEATURE NAME:	VALUE
+
+Objects	[ { "rectangle": { "x": 138, "y": 27, "w": 746, "h": 471 }, "object": "headwear", "confidence": 0.616 }, { "rectangle": { "x": 52, "y": 33, "w": 910, "h": 951 }, "object": "person", "confidence": 0.802 } ]
+
+Tags	[ { "name": "man", "confidence": 0.999212 }, { "name": "headdress", "confidence": 0.99731946 }, { "name": "person", "confidence": 0.995057464 }, { "name": "clothing", "confidence": 0.991814733 }, { "name": "wearing", "confidence": 0.9827137 }, { "name": "hat", "confidence": 0.9691986 }, { "name": "helmet", "confidence": 0.9227209 }, { "name": "headgear", "confidence": 0.840476155 }, { "name": "personal protective equipment", "confidence": 0.8358513 }, { "name": "looking", "confidence": 0.832229853 }, { "name": "hard hat", "confidence": 0.8004248 }, { "name": "human face", "confidence": 0.785058737 }, { "name": "green", "confidence": 0.774940848 }, { "name": "fashion accessory", "confidence": 0.706475437 } ]
+
+Description	{ "tags": [ "man", "headdress", "person", "clothing", "wearing", "hat", "helmet", "looking", "green", "jacket", "shirt", "standing", "head", "suit", "glasses", "yellow", "white", "large", "phone", "holding" ], "captions": [ { "text": "a man wearing a helmet", "confidence": 0.8976638 } ] }
+
+Image format	"Jpeg"
+
+Image dimensions	1000 x 1000
+
+Clip art type	0
+
+Line drawing type	0
+
+Black and white	false
+
+Adult content	false
+
+Adult score	0.0126242451
+
+Racy	false
+
+Racy score	0.0156497136
+
+Categories	[ { "name": "people_", "score": 0.69140625 } ]
+
+Faces	[ { "age": 37, "gender": "Male", "faceRectangle": { "top": 419, "left": 363, "width": 398, "height": 398 } } ]
+
+Dominant color background	"White"
+
+Dominant color foreground	"White"
+
+Accent Color	#90A526
+```
+
+(Note, the above analysis may change in the future: the Computer Vision model is
+updated regularly.)
+
+Note that in the first "Objects" result, two objects "headwear" and "person" are
+detected, and their locations in the image is given. The object we want to
+detect is classified "headwear", but for our application we need a more specific
+classification: "hard hat". However "hard hat" is not one of the object types
+that Computer Vision currently detects. (We'll address this problem with Custom
+Vision, later.) Also note that a confidence score is given for each object
+classification.
+
+The second "Tags" result gives a list of labels associated with the entire
+image. The tag with the highest confidence (listed first) is "man", which
+doesn't help us much. The second tag, "headdress", is not exactly what we are
+looking for either.
+
+The other responses are also interesting, but we won't focus on them for our
+demo. But take a look at what's included:
+
+* A caption for the photo ("a man wearing a helmet") in the Description field.
+
+* Image features (is it black and white? a line drawing?)
+
+* Details of any faces detected in the image (identified as a 37-year-old male in this case)
+
+* A score for the content of the image: is it "Adult" or "Racy"?
+
+* Color analysis for the image: the dominant foreground, accent, and background colors.
+
+We're really only interested in the "Tags" field for our purposes, so we'll find
+out how to extract that programatically in the next section.
+
+## Using Computer Vision via the API
+
+You can [control Computer Vision programatically using its REST
+API](https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/vision-api-how-to-topics/howtocallvisionapi?WT.mc_id=msignitethetour2019-github-aiml20).
+You can do this from just about any language or application that has access to
+the Web, but we will use [curl](https://curl.haxx.se/), a common command-line
+application for interfacing with URLs and collecting their outputs. The curl
+application comes pre-installed on most Linux distributions and in recent
+versions of Windows 10 (1706 and later). 
+
+Run the commands in the file [`vision_demo.sh`](vision_demo.sh). You can use a local Azure CLI or
+Azure Cloud Shell, but you must use bash as the shell.
+
+The commands in this script will:
+
+1. Log into your Azure subscription (this step is unneccessary if using Cloud Shell)
+2. Create an Azure Resource Group
+3. Create a Cognitive Service key. (Note: this is an omnibus key that we will also use for Custom Vision, later.)
+4. Find the key
+5. Use CURL to analyze two images
+
+## Manually generating Keys for use with Computer Vision
+
+In the script [vision_demo.sh](vision_demo.sh), run the section "Create a Key" to programatically create a Cognitive Sevices key using the Azure Command Line Interface.
+(If you prefer, you can [create keys interactively with the Azure
+Portal](https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-apis-create-account?tabs=multiservice%2Clinux&WT.mc_id=msignitethetour2019-github-aiml20).)
+
+## Next Step
+
+[Custom Vision](DEMO%20Custom%20Vision.md)

aiml20/DEMO Custom Vision.md

@@ -0,0 +1,80 @@
+# DEMO: Custom Vision
+
+In this demo, we create a custom vision model using the service at
+customvision.ai.
+
+## Create a custom model project with Custom Vision
+
+TIP: You can do this section ahead of time, to save time in a live demo.
+
+1. Sign into [Custom Vision](https://customvision.ai) and create a new project.
+   [Follow the instructions
+   here](https://docs.microsoft.com/azure/cognitive-services/custom-vision-service/getting-started-build-a-classifier?WT.mc_id=msignitethetour2019-github-aiml20).
+
+1. Your new project should have the following settings:
+
+    - Name: Tools
+    - Description: Products sold by Tailwind   Traders
+    - Resource: aiml20-cs-resource
+    - Project Type: classification
+    - Classification Types: Multiclass
+    - Domains: Retail (compact)
+    - Export capabilities: Basic Platforms
+
+1. In the "Tags" tool in the left side, use the "+" button to add the following tags:
+
+    - drill
+    - hammer
+    - pliers
+    - screwdriver
+
+1. We will now add images for each tag from the corresponding folder in "CV Training Images". Here's how to do it for "drill"
+
+    - Click "Add Images" in the top row
+    - Browse to CV Training Images / drills
+    - Select all files: click one file, then press Control-A
+    - Click Open
+    - In the "Image Upload" dialog that appears next, select "drill" in "My Tags" and then click "Upload 16 Files"
+    - repeat this process for the tags: hammer, pliers, screwdriver (we will do "hard hat" later)
+
+## Add images and train a model
+
+At Ignite Tour, the prior steps were all done ahead of time in preparation. Only
+the remaining steps were done live, to save time.
+
+1. If you haven't already, sign into customvision.ai and open your "tools" project.
+
+1. This project has already been provided with images of drills, hammers, pliers, and screwdrivers. Let's add some images of hard hats as well.
+
+- Click "Add Images" in the top row
+- Browse to CV Training Images / drills
+- Select all files: click one file, then press Control-A
+- Click Open
+- In the "Image Upload" dialog that appears next, select "hard hat" in "My Tags" and then click "Upload 16 Files"
+
+1. Click "Performance" in the top menu bar, and then click the green "Train" button. Choose "Quick Training" and then click "Train"
+
+1. Click on the most recent iteration in the left pane to see the results. You
+   should have 90.9% Precision (how many of our training images does the model
+   predict correctly?), 88.2% Recall (when we generate a prediction, is it
+   correct?), and 98.7% AP (overall measure). Note that these depend on the
+   probability threshold -- make this low if you want to avoid false positives.
+   For our retail app, it's safe to make it high.
+
+1. Test out the model with a new picture. Click "Quick Test" and "Browse Local
+   Files", and then choose "test images / man in hardhat.jpg". See that it
+   identifies as a hard hat with 99.9% probability. Try the same with "test images / drill.jpg", which is also identified correctly.
+
+Now we will save the model in the ONNX format for integration into the Tailwind
+Traders app.
+
+1. Click Export, and choose "ONNX". Note the other platform-specific formats
+   available as well.
+
+1. Select the version ONNX 1.2, and then click Export. The TWT app expects a
+   file called products.onnx, so we'll use the one that's already saved in the
+   repo.
+
+## Next Step
+
+[ONNX Deployment](DEMO%20ONNX%20deployment.md)

aiml20/DEMO ONNX deployment.md

@@ -0,0 +1,93 @@
+# DEMO: ONNX Deployment
+
+In this demo, we take the ONNX file we exported in the [Custom
+Vision](DEMO%20Custom%20Vision.md) demo, and deploy it to the Tailwind Traders website.
+
+The website uses the model in `products.onnx` for the Shop by Photo app. The
+uploaded image is processed by the model, which generates one of five labels:
+"hammer", "drill", "pliers", "screwdriver" or "hard hat". The website searches
+the product list for the generated label, and returns the results of the search.
+
+## Load the simple ONNX model
+
+(TIP: You can do this step ahead of time. This step is necessary if you have run
+this demo before on the same deployment.)
+
+We will replace the products.onnx file in the Web app with a version that only recognizes two object categories: "hammer" and "drill".
+
+1. In the Azure Portal, visit your aiml20-demo resource group
+
+1. Click the "aiml20" App Service resource
+
+1. In the left menu under Development Tools, Click Advanced tools, then click "Go" in right pane to launch Kudu.
+
+1. In the main menu bar, Click Debug Console > PowerShell
+
+1. Browse to: site / wwwroot / Standalone / Onnxmodels
+
+1. With Explorer, open the `ONNX / simple model` folder from your AIML20 repo
+
+1. Drag products.onnx into the LEFT HALF of the Kudu window. (IMPORTANT: Do NOT drag into the box that says "drag here to upload and unzip".) This model only knows how to identify drills and hammers.
+
+1. Restart the web server. Return to the "aiml20" App Service resource and click "Restart" in the top menu bar. Wait two minutes for the website to restart completely.
+
+## Defining the problem: Shop by Photo doesn't work right
+
+(Note: This section was done at the beginning of the AIML20 presentation.)
+
+1. Visit the Tailwind Traders website you deployed earlier. 
+
+1. Scroll down to the "Shop by Photo" section of the website
+
+1. Click "Shop by Photo"
+
+1. In your AIML20 repo, select: test images > drill.jpg
+
+1. It correctly identifies it as a drill. Yay!
+
+1. Return to home page and click "Shop by Photo" again
+
+1. In your AIML20 repo, select: test images > pliers.jpg
+
+1. Oh no! It identifies it as a hammer. We'll fix that later, but first, let's understand why it failed.
+
+## Update the ONNX model in the Tailwind Traders website
+
+First, view the exported model in Netron:
+
+1. Browse to https://lutzroeder.github.io/netron/, Click Open Model
+
+2. Open ONNX / Custom Model / products.onnx
+
+3. Scroll through the neural network and note:
+
+ - it's large
+ - at the top, is a 224x224 image as input (dirty secret: computer vision models have pretty poor vision)
+ - add the bottom, it outputs 5 values, these are the confidence scores for our class labels
+
+Next, drop the ONNX file we exported into TWT filesystem
+
+1. In the Azure Portal, visit your aiml20-demo resource group
+
+1. Click the "aiml20" Web App resource
+
+1. Under Development Tools, Click Advanced tools, then click "Go" in right pane to launch Kudu.
+
+1. In the main menu bar, Click Debug Console > PowerShell
+
+1. Browse to: site / wwwroot / Standalone / Onnxmodels
+
+1. With Explorer, open the `ONNX / custom model` folder from your AIML20 repo
+
+1. Drag products.onnx into the LEFT HALF of the Kudu window. (IMPORTANT: Do NOT
+   drag into the box that says "drag here to upload and unzip".)
+
+1. Restart the web server. Return to the "onnx" Web App resource and click "Restart".
+
+Rerun Shop by Photo, upload `test images / pliers.jpg`. Now it works!
+
+## Next Step
+
+[Personalizer](DEMO%20Personalizer.md)
+
+

aiml20/DEMO Personalizer.md

@@ -0,0 +1,24 @@
+# Personalizer Demo
+
+In this demo, observe how the layout of a website adapts to visitor actions using reinforcement learning and [Cognitive Services Personalizer](https://docs.microsoft.com/en-us/azure/cognitive-services/personalizer/?WT.mc_id=msignitethetour2019-github-aiml20).
+
+Personalizer will dynamically reconfigure the interface to optimize the
+likelihood of an anonymous visitor clicking on the featured category in the
+Recommended section.
+
+1. Deploy the Tailwind Traders Website app (if you didn't already do so with `DEMO Setup.md`)
+
+2. Visit the Tailwind Traders homepage
+
+3. Observe the "Recommended" section and the order of the featured categories
+
+4. Refresh the page (you may need to do this a couple of times). Observe that
+   the layout changes.
+
+The Personalizer service is tracking the anonymous visitors and recording the
+time of day, day of week, and browser OS used when clicking on categories. 
+The "reward" is whether or not the large, featured section was clicked. 
+
+Over time, Personalizer will determine the best category to feature based on
+time of day, day of week, and OS. It will also "explore" 20% of the time, to
+surface categories that would otherwise not be presented.