Building Intelligent Apps and Agents with the Computer Vision API, Custom Vision Service, LUIS, Bing Search, and Azure Search
Welcome to the Learn AI Bootcamp for Emerging AI Developers. We will focus on hands-on activities that develop proficiency in Azure Cognitive Services including Computer Vision, Custom Vision, LUIS, and Bing Search. You will also learn to leverage other AI-oriented services such as Azure Search and Azure Bot Services. These labs assume an introductory to intermediate knowledge of these services, and, if this is not the case, then you should spend the time working through the prerequisites.
!!Important note 09/28/2018!!: The Microsoft Bot Builder SDK V4 went GA at Ignite this week. The bot labs have been updated to SDK V4, but the SDK V3 labs are located in the resources folder for the bot labs (not being maintained). Additionally, we have moved the logging and testing labs to supplementary exercises (not being maintained at the moment), and the bootcamp will focus on the addition of the Bing Search APIs to bots and other applications.
If you are an instructor redelivering this course and have questions, please email learnanalytics@microsoft.com.
Most challenges observed by customers in these realms are in stitching multiple services together. As such, where possible, we have tried to place key concepts in the context of a broader example.
At the end of this workshop, you should be able to:
- Understand how to configure your apps to call Cognitive Services
- Build an application that calls various Cognitive Services APIs (specifically Computer Vision)
- Effectively leverage the custom vision service to create image classification and object detection services that can then be leveraged by an application
- Understand how to implement Azure Search features to provide a positive search experience inside applications
- Configure an Azure Search service to extend your data to enable full-text, language-aware search
- Build, train, and publish a LUIS model to help your bot communicate effectively
- Build and publish an intelligent bot using Microsoft Bot Framework that leverages LUIS and Azure Search
- Add Cognitive Services (specifically the Bing Search APIs) to your existing applications
This workshop is meant for an AI Developer on Azure. Since this is only a short workshop, there are certain things you need before you arrive.
Firstly, you should have some previous exposure to Visual Studio. We will be using it for everything we are building in the workshop, so you should be familiar with how to use it to create applications. Additionally, this is not a class where we teach you how to code or develop applications. We assume you have some familiarity with C# (intermediate level - you can learn here and here), but you do not know how to implement solutions with Cognitive Services.
Secondly, you should have some experience developing bots with Microsoft's Bot Framework. We won't spend a lot of time discussing how to design them or how dialogs work. If you are not familiar with the Bot Framework, you should complete this tutorial prior to attending the workshop.
Thirdly, you should have experience with the portal and be able to create resources (and spend money) on Azure. We will not be providing Azure passes for this workshop.
Finally, before arriving at the workshop, we expect you to have completed 1_Setup along with configuring the following for Custom Vision:
- Training API Key: The training API key allows you to create, manage and train Custom Vision project programmatically.
- You can obtain a key by creating a new project at https://customvision.ai and then clicking on the “setting” gear in the top right.
Please note: This is a rough agenda, and the schedule is subject to change pending class activities, breaks, and interactions.
- Day 1: Cognitive Services
- 8-9 (optional): Setup assistance
- 9-10: Introduction and Context for Cognitive Services
- 10-12: Lab 1.1: Simplifying Cognitive Services App Development using Portable Class Libraries
- 12-1: Lunch
- 1-1:30: Introduction to Custom Vision
- 1:30-2: Lab 1.2: Creating an Image Classification Application using the Custom Vision Service
- 2-2:30: Lab 1.3: Creating an Object Detection Application using the Custom Vision Service
- 2:30-2:45: Break
- 2:45-3:45: Lab 1.5: Developing Intelligent Applications with LUIS
- 3:45-5: Business Case I - Cognitive Services
- Day 2: Bots
- 9-10: Lab 2.1: Developing Intelligent Applications with Azure Search
- 10-10:30: Introduction and Context for Bots
- 10:30-12:30: Lab 2.2: Building Intelligent Bots
- 12:30-1:30: Lunch
- 1:30-3: Lab 2.3: Enhancing Applications with Bing Search
- 3-3:15: Break
- 3:15-4:15: Business Case II - Bots
- 4:15-5: Q&A and Feedback for Emerging AI Bootcamp
The LearnAI team has provided some extra labs you may be interested in:
Here are some related courses from the LearnAI team:
- LearnAI: Intelligent Agents: Design and Architecture
- LearnAI: Building Enterprise Cognitive Search Solutions
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.