Prototype for SmartCity Project about detecting anomalies in people's commute to reduce uncertainty. We used computer vision and other APIs to notify users at the right time. For example, when there are too many people at the subway station and trains are too crowded, we may alert a user who might be thinking about taking the subway. This way users can think about alternative ways like taking an Uber or bike.
What is this?
Mobile App (Cordova)
- Notifies the user of anomalies
- Connects to existing APIs:
- Google Maps API for getting the route information
- DarkSky API for predictive weather data
- MBTA API for getting possible alerts for a subway line
- Uses cordova plugins for
- running in the background
- creating notification
- The python script works as the image processing backend that counts the number of objects (person, cars, etc.)
- The script listens to firebase for any new images. When an image is pushed, it downloads the image and runs tensor-flow object_datection inferencing model (pre-trained). The result is then pushed back to firebase.
- The dashboard acts as a part of the backend. It pipes an input stream of photos (e.g. from camera) to the tensorflow input stream.
- It visualizes each of the input stream and also the output stream from Tensorflow.
- It captures photos using phone's camera and pushes them to Firebase.
- It can run on any Android Nougat Chrome browser.
- To manually trigger anomalies on the webapp
- Install Tensorflow (link) and the requirements for object detection (link)
- Setup a new Firebase project. Install Firebase-CLI.
- Install Cordova-CLI version 6.5:
npm install email@example.com -g
- Clone tensorflow models This contains the object recognition
- Clone this repository
- Set firebase database and storage rules to allow all requests. Then allow CORS requests on Firebase
- Edit these files to use your firebase tokens
- Camera app
- Edit the mobile app to use your Google Maps, Dark Sky, and MBTA API tokens
- Copy the
tensorflow/object_detection_with_firebase_input.pyfrom this repository into
Gaurav Paruthi, Carlos Armada, Joanne Cheung, Parker Woodworth, Shuya Gong, Gian Pangaro