AI Video Intelligence Solution Accelerator

The AI Video Intelligence Solution Accelerator is a variant of the Azure IoT .NET solution for Remote Monitoring which uses specialized Azure IoT Edge devices to capture and process video input from cameras.

The IoT Edge devices used by this solution are capable of simulating camera input, so physical cameras are not required to build and demonstrate the solution. To get started, deploy the IoT Edge device to an Azure Cloud VM. This will leverage an IoT Edge based Azure Cloud VM and deploy the trained Azure ML CPU-based model to that VM. Once you subscribe to Azure Data Box Edge, you can deploy the IoT Edge modules to the Data Box Edge and benefit from the included Azure ML FPGA acceleration.

There are two parts of the solution accelerator: the cloud solution and the edge device that runs on-premises. The cloud solution leverages Azure IoT Hub to manage the deployment to the device and maintain device health. An instance of Azure App Service is used to deploy a web-site that enables an operator portal experience. A number of other Azure services are used to enable storage, analytics via Azure Time Series Insights, stream processing via Azure Stream Analytics, integration via Azure Logic Apps, maps via Azure Maps, database via Azure CosmosDB, etc...

Prerequisites

• Microsoft Azure subscription
• npm version 6.4.1 or greater
• nodejs version 10.14.2 or greater

Deploy the AI Video Intelligence Solution Accelerator

1. Clone the AI Video Intelligence Solution Accelerator (this repo).
2. From a command prompt:
   • navigate to AI-Video-Intelligence-Solution-Accelerator/pcs/cli
   • deploy-pcs.cmd (you will be prompted for your subscription credentials)
3. Wait for deployment to complete. It takes about 10 to 20 minutes.
4. Verify that deployment completed successfully. At the end you should have a statement something like:
   Solution: MyDeployment is deployed at https://MyDeployment.azurewebsites.net
5. While signed into the same account that was used in the pcs login step, launch the Remote Monitoring website in a browser. Verify that the site loads.
6. This sample does not use the standard solution accelerator simulated devices, so to reduce costs, click the settings gear icon in the top right corner, make sure Simulation Data is turned off.

Enable access to BLOB storage for image storage and retrieval

1. Login to https://portal.azure.com with the same account you used to deploy the CLI
2. Select the Resource Group to which the Remote Monitoring Template was deployed
3. Select the Storage Account
4. Under Blob Service, select Blobs
5. Create a new Container called still-images, set Public access level to Private
6. After still-images is created, click the ... at the end of its row, select Access policy
7. Add a policy with the identifier allowWebUiDirectAccess and Read permissions, click OK and be sure to click Save

Give other users access to the solution website

1. On https://portal.azure.com go to the Azure Active Directory resource and select Enterprise applications.
2. Change the selection in the Application Type dropdown to All Applications and click Apply.
3. Find the deployment you are attempting to administer and select it. Searching by deployment name (e.g. MyDeployment) using the text box right below the Application Type dropdown might be helpful.
4. Click Users and groups from the Manage section of the vertical menu choices.
5. Select Add user from the button options above the list of current users.
6. In the Add Assignment panel that appears select User
7. In the subsequent dialog, either select a user who is already a part of your organization or invite a new user with the Search box, and click Select
8. Back in the Add Assignment panel Select Role
9. Choose either Admin or Read Only as appropriate and click Select
10. Finally click Assign

Generate a BLOB Storage SAS URL

1. Open a new portal.azure.com tab in your browser.
2. Navigate to the storage account for your solution's resource group.
3. Under Settings, Select the Shared Access Signature
4. On Allowed services select Blob only
5. On Allowed resource types select Container + Object
6. On Allowed permissions select Write + Create
7. Leave Start time as-is
8. Select some appropriate End time at least a few months into the future.
9. Click the Generate SAS and connection string button and make note of the generated Connection String for later use.

AI Video Edge Device Deployment

In order to see image data in the solution web site you'll need to create at least one AI Video Edge Device. AI Video Devices for the AI Video Intelligence Solution Accelerator are connected to real or simulated cameras which collect the images to be processed.

If you create your AI Video Edge Device locally (rather than as a cloud resource) then you may optionally connect it to a USB camera using the simple Camera Server sample application.

Prerequisites

• VS Code with Azure IoT Hub extensions installed according to this article.

Prepare VS Code for deployment

1. Navigate to the Azure IoT Device source code and open that folder (which contains the deployment.template.json file) in VS Code.
2. Open the .env.template file and save a copy of it as .env.
3. In the .env file, populate the BLOB_STORAGE_SAS_URL value with the BLOB Storage SAS Connection String that you generated earlier. Make sure the value is within quotes. You won't need to add the CONTAINER_REGISTRY_USERNAME and CONTAINER_REGISTRY_PASSWORD values unless you want to try modifying and deploying your own versions of the sample modules.

Deploy to your chosen Azure AI Video Device platform

The primary platform for Azure IoT Devices in the AI Video Intelligence Solution Accelerator is Azure Data Box Edge, but non-DBE computers can also be used. Here are three options for deployment:

• Run an Azure IoT Device on your own computer using VS Code:

  1. Register a new Azure IoT Edge device from Visual Studio Code using these instructions.
  2. Press Ctl-Shift-P and select Azure IoT Edge: Setup IoT Edge Simulator.
  3. Select the name of your newly created device.
  4. Right-click the deployment.template.json file and select Build and Run IoT Edge Solution in Simulator.

  Note that the Edge device will continue to run even if you close VS Code. To stop the Edge device, press Ctl-Shift-P and select Azure IoT Edge: Stop IoT Edge Simulator.

• Deploy to a Data Box Edge:

  1. Follow the Data Box Edge instructions to create a Data Box Edge device that is connected to the IoT Hub that was created when you ran the AI Video Intelligence Solution Accelerator.
  2. In the Azure Portal, navigate to your IoT Hub.
  3. Under Automatic Device Management, select IoT Edge.
  4. In the devices list, find the device that corresponds to the DBE device you just connected. Its name will be the DBE name with "-edge" appended. Make note of the name.
  5. Right-click the deployment.dbe.template.json file and select Generate IoT Edge Deployment Manifest.
  6. In the AZURE IOT HUB section of VS Code, under Devices, right click the name of your DBE device.
  7. Select Create Deployment for Single Device. You will be prompted for a file name.
  8. Select the file named deployment.dbe.amd64.json which can be found in the config directory. Do not select the template file by mistake.

  The deployment to the DBE will take a few minutes, and you can monitor the progress of the deployment via the Azure Portal. If the DBE's FPGA needs to be re-flashed, it may take another 10 or 15 minutes after the deployment succeeds before the FPGA is flashed and operational.

• Deploy to an Azure cloud VM: If you don't want to run the Azure Edge device on your own computer but you don't have a DBE device, you can deploy to a standard Azure VM.

  1. Register a new Azure IoT Edge device from Visual Studio Code using these instructions.
  2. Click on the newly created device in the device list and copy the device connection string (primary key) from the device details page. Save this for later use. (You may have to hit F5 to see your new device in the list.)
  3. Open portal.azure.com in a new tab.
  4. Add a new Azure IoT Edge on Ubuntu resource to the resource group containing your solution.
  5. Enter a name for your VM.
  6. Under Administrator account select "password" as the Authentication Type.
  7. Under Inbound port rules select "SSH".
  8. Click "Review + create".
  9. After validation passes, click "Create".
  10. Wait for the deployment to finish.
  11. Click the "Go to resource" button.
  12. Open the Run command option under Operations.
  13. Select the RunShellScript item.
  14. Enter /etc/iotedge/configedge.sh "{device_connection_string}", where {device_connection_string} is the connection string you saved in Register a new Edge Device with the IoT Edge Hub. This process takes several minutes and gives no feedback until it completes, so be patient.
  15. Right-click the deployment.template.json file and select Generate IoT Edge Deployment Manifest.
  16. In the AZURE IOT HUB section of VS Code, under Devices, right click the name of your device.
  17. Select Create Deployment for Single Device. You will be prompted for a file name.
  18. Select the file named deployment.amd64.json which can be found in the config directory. Do not select the template file by mistake.

  The deployment will take a few minutes, and you can monitor the progress of the deployment via the Azure Portal.

View the results

You can now open the web site for your deployment and select one of the cameras from the dropdown. Images from your new device should appear in the "Insights from video stream" window.

Optional: connect a real camera

If you deployed your AI Video Device locally, you can connect it to a USB camera using the simple Camera Server sample application.