WebApp has New layers (AI layer + photo classification)
automated, new layers replace data entry
Click on the “chooser” button, snap photo, and submit **
New issue is created on basis of the photo alone
Pretrained model & AI classification model will:
- Determine location/address
- context - photo classification
- Auto-Selection of “type” based on AI analysis of photo
- No need to tell the app (its graffiti, its garbage, … )
Watch video - prototype app built on a few lines of html featuring API layer that:
- Uploads photo to AWS
- API analysis of photo
- Automatic organization of data
- Automatic creation of new issue
- Database update
- Maps / Issues integration
See the Demo upload sample photos, see the results from "point-and-shoot" issue reporting.
Watch a how to video a walk-thru on issue creation
- choose a photo
- click "upload" and be patient while the AI does its thing
- review categorization done by AI layer (address, issue-type)
- click on "done" posts issue to the DB
** Note - due to ios17 restrictions, 2 steps are needed in order to preserve metadata or GPS info in the photo as it uploads:
IOS17 only details on iphone
step 1 open camera app and take the photo that will be used in #2
step 2 using the chooser, select photoroll
then select the photo just taken, click "done" to upload that photo
It could not be any easier.
From the phone's images ( gallery or snapped photos ) it does not require a native app ( ios or android ) in order to extract the following:
- what is the photo about graffiti , garbage , encampment ...
- what are the gps coordinates
- what is the street address
Lightweight webapps use minimal html plus api layer to point, shoot, and submit - AI and a couple cloud, API calls automatically handle the rest. Roboflow AI classifies photos by type in an app context uploading media containing all the metadata needed to define org311 issues like graffiti or garbage. Organized for easy learn to code experience using an interactive notebook format, this project also presents a lightweight webapp front end, simple html providing photo uploads, parsing metadata out of the photo and classifying photos according to issue-type. When combined, these prototype features allow point-and-shoot pothole reporting - a complete capture of a org311 type issue takes place while reporting the issue is as simple as take a picture and submit it.
Notebooks provide mix of docs and special code-execution cells, allowing great access for total beginners to coding, python, and AI. Links below to a cloud feature called colab, a dashboard for running notebooks ( also see video clips helping understand how the notebooks work).
intro to media uploads run this notebook, observing functions to get Address from metadata and to classify the photo with AI
how to use the notebook above on Colab
python methods and more detail on functions from the intro demo
connect your DB from Mongo DB Atlas, store issues that you create in notebook sample 2