An iOS app for empowering visually impaired to help finding their belongings and overcome day-to-day struggles through Mobile camera as "Third Eye" using Object Detection Algorithm and speech recognition.
Our project takes input from User to find a specific item using speech-to-text and detects the requested object in the room in "REAL-TIME" and using beep sensing and vibration in the mobile phone it will guide the user to the item.
@authors:
Prashuk Ajmera
Hitesh Verma
Parag Bhingarkar
Genglin Liu
- Detecting Object in "real-time"
- In order to decrease the latency of response from the deep learning model, we had to integrate the neural network with the iOS platform and deploy the model on the mobile phone itself, instead of relying on a cloud service and getting detection results from the cloud
- Choosing the most-fit Object Detection Algorithm
- Integrating the neural network on an iOS platform instead of on a cloud service
- Swift/Python
- AI/Machine Learning
- iOS Frameworks (AVKit, Speech, CoreML)
The app was developed on Xcode 11.2 with Swift 5.1 programming language and it is available for iOS 11.0 and above.
Extending our app using Drone which can help to find items without any human support and can freely rotate in 360 angles. It would be interesting to use a combination of MobileNet and SSD to detect an object in real-time video streaming.