Official repo of Roby HEAlth Partner mobile App. Please note Unity version 2020.1.0f1 is needed to build and a ARCore-enabled Android device is needed to run it.
There is a link of the Android APK build in the releases page.
Generally, children do not miss the opportunity to express what they are thinking or feeling, however, the current emergency context resulting from COVID-19 has led them to deal with new problems and situations that their parents have not faced either and many times they are also overwhelmed.
Using the spoken language as our spearhead, we developed Roby Health Partner, a virtual parner with whom children can talk naturally and tell how they feel, finding in it a help and answer to their concerns. Roby identifies the symptoms presented by the child and classifies them into: anxiety, sadness, sleep disturbances, irritability and grief, providing listening, attention, guidelines and relevant activities to help the child to better cope with their feelings.
TBD
Ruby uses an IBM Cloud Function (an openwhisk-based service) as endpoint to process and answer interactions. The source code of that service can be found in Functions/api.js. In order to prevent abuses, all the keys in this file have been deleted, so it is necessary to previously create all the services and obtain the relevant keys.
Roby HEAP communicates, via an API created in IBM Cloud Functions, with IBM's Speech to Text, Watson Assistant, Text to Speech, and IBM Cloud Object Storage services (as a cache for TTS audios), as follows:
Roby HEAP uses the following services and libraries:
- IBM Speech to Text To convert the users' voice to text.
- IBM Watson Assistant to manage the conversational dialogue, identify intentions, entities and recommend activities.
- IBM Text to Speech to convert the text of the answers from Assistant to audio.
- IBM Cloud Object Storage to store and reuse Text to Speech responses.
- AR Fundation as AR Framework.
- ARCore as AR implementation for Android.
- Jammo Character as avatar.
- JSONDotNet For Unity to parse JSON responses.
- NAudio to deal with audio encode/decode.