Submission for SCDF-x-IBM-Lifesavers-Innovation 2020 48h Hackathon
By Team Last Place
: comprised of
- Loh Kar Wei: Asienwald: Development of Android App functionality, app design and Watson API Integration
- Tan Jia Wei: Hayashion: Designed optimal course of action for emergencies, Video Demonstration and Watson API Integration
- Chan Zun Mun Terence: Hackin7: IoT functionality, setting up of central IoT server and web layout
- Short description
- Demo video
- The architecture
- Long description
- Getting started
- Built with
- Possible Improvements
- Images
SCDF works closely with Community First Responders (CFRs) to provide timely relief and response to emergency situations.
With the increasingly aging population in Singapore, not everyone of them is able to receive effective early emergency response due to inadequate emergency detection and the increasing numbers of elderly with no next of kin.
With the recent advances in Artificial Intelligence, Natural Language Processing, Speech Recognition, we can create interfaces to allow people to get timely medical help easily
We have designed the app ELCare to help the elderly in emergencies. Through it's various functionalities, such as the Watson Assistant and IoT monitoring, it can help identify when an elderly is in danger, and get the necessary help by leveraging the power of CFRs and early emergency detection to ensure they get the help they need before it is too late.
- The Android app is coded in Java with Android Studio
- It connects to a server (Through REST APIs) to retrieve data from IoT devices
- The server is coded in Flask (Python) and stores the current information about the data
- The IoT Devices can connect to the server and send the current data (similarily through REST APIs)
- The app also connects to IBM's servers to use the Speech to Text, Tone Analyzer and Watson Assistant APIs
- The elderly would open the application
- Open
Chat
orSpeak
and communicate with Jolene - Jolene will be able to identify the help needed
- For related individuals, they can access the
Monitor
section on the app, or use the website to monitor the elderly's situation
You can download the app here. Alternatively, you can compile it in Android studio, but take note to set the appropriate API
To set up the server (acting as a bridge between the IoT device and the app)
- Copy
API_KEYS_sample.py
to API_KEYS.py and fill in your API Keys and URLs for SendGrid, IBM STT and IBM Tone Analyzers - Build the Dockerimage by running
docker build Web\ Server
anddocker run -p 5000:5000 <container-id>
- Alternatively, you can use the currently hosted server at https://scdf-x-ibm-web.herokuapp.com/ (This url is hardcoded in many aspects of this system).
- If you set up a server, make sure you can connect to it through https
- If you are checking the dashboard, login with the default credentials
- Username:
GodMode
- Password:
yeshucpo
- Username:
An external device (eg. old smartphone/ laptop) can act as a second set of microphone and camera to detect any falls and loud sounds.
- Login to the server. Go to
<server-url>/monitoring
, and clickStart all
- Position your device such that the camera can see the scene
- Leave it to log any falls and loud sounds, and you can view them on the dashboard.
- Use an ESP8266.
- Connect a sound sensor to GPIO 5, and a PIR sensor to GPIO 4.
- Provide power to all components
- Fill in the Wifi AP Name and Password and replace the server url with your own. Upload the sketch in
IOT Hardware/SimpleMonitoring/
- To use the temperature sensor, connect a DHT22 sensor to GPIO 0, and uncomment the appropriate lines of code
- Use an ESP8266.
- Connect a sound sensor to GPIO 5, and a PIR sensor to GPIO 4.
- Provide power to all components
- Fill in the Wifi AP Name and Password and replace the server url with your own. Upload the sketch in
IOT Hardware/SimpleMonitoring/
- To use the temperature sensor, connect a DHT22 sensor to GPIO 0, and uncomment the appropriate lines of code
- Android Studio
- Invision Studio
- Docker and Heroku (for Deployment of the Web server)
- Google's Teachable Machine 2.0 (for pose detection for a falling person)
- Includes Tensorflow.js and PoseNet
- ESP8266(NodeMCU) and Arduino Programming Language
- DaVinci Resolve (Video Editing)
- Audacity (Audio Recording)
- We could implement better push notifications.
- Use TTS for better communication
- We could interface with the actual first responder API to better notify first responders to help the elderly
- We could have implemented a database to store information more effectively