Sample code for an app that uses search-by-image functionality to showcase Bing API Adaptive Cards rendered on HoloLens (or other Windows Mixed Reality devices). The app allows users to get contexual information about their surroundings, in the form of cards, by taking a picture.
After a user takes a picture, (1) the picture is analyzed via a computer vision API, (2) Adaptive Card markup is retrieved from the Bing Web Search API using the image analysis results as the search query, and (3) the card markup is rendered on the device and shown to the user.
Key features of the project include:
- Bing Web Search API
- Microsoft Cognitive Services Vision API
- Adaptive Cards
- UWP MediaCapture and SpeechRecognition APIs
The app is built as a UWP app with UI optimized for HoloLens. While this keeps the code simple, it means that the user's experience on a Mixed Reality headset is limited to 2D, and is not as immersive as a full 3D app. You're more than welcome to explore further and develop a 3D experience based off this sample, though. Refer to the contribution guidelines below if you'd like to commit your changes to this repo.
To get set up for development, follow the instructions in the Windows Dev Center here. The instructions regarding Unity can be ignored unless you wish to extend upon the sample code using 3D graphics.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.