The smart speaker comprised a speaker, a voice accessory expansion board, a 6-mics circular array board, and a micro-computer. The 6-mics array board allows omnidirec-tional voice capture. The micro-computer was a Raspberry Pi, which has more computing power than the Arduino-based micro-controllers of other sensors to process complicated tasks. A representative smart speaker is illustrated in the figure below.
Self-report exhaustion is one of the essential frailty criteria in clinical frailty assessment. We programmed a smart speaker to ask the questions from the Center for Epidemiologic Studies Depression Scale (CES–D) to collect data about self-report exhaustion defined in Fried’s phenotype model. The speaker asked the following two questions sequentially,
- “Do you feel that everything you did was an effort?”
- “How often in the last week did you feel this way?”
While the first question is a yes or no question, the second question expects users to answer one of the three words, “always”, “sometimes”, or “rare”. Users responding “sometimes” or “always” will be categorized as frail for this criterion. A sample interaction between the smart speaker and a user is as illustrated in the following figure.
The AWS Lex was used to build a chatbot to handle the user conversation. The IBM Watson IoT platform was used to receive user responses from the smart speaker.
The smart speaker was deployed and tested in HomeLab at KITE Research, Toronto Rehabilitation Institute, Toronto, Canada. It could be placed in a convenient location according to the older adults' lifestyles. For example, our focus group study published in the top BMC Geriatrics journal found that some older adults preferred to interact with the speaker in the kitchen while preparing food. Others chose to put the speaker near the bed for a quick conversation before sleeping.
In addition to the flexible placement, the smart speaker would play a soft prompting ringtone before the frailty conversation, resembling the one used in the airport before any announcements. At the end of the conversation, the speaker would confirm all information was received and appreciate users’ responses. Other design considerations that have not been incorporated into the current system but will be in the future iteration include:
- adding physical buttons to the smart speaker for those older adults who prefer the familiar, simple button press to the verbal conversation with the smart speaker. The goal was to reduce the complexity or probability of confusion to enhance usability;
- adding enjoyable functions to the smart speaker, such as playing music and telling jokes.