We developed a realistic Figma-based prototype which enables a person to operate a mobile telemanipulator robot (MTR) in a clinical simulation setting. The simulation is designed so clinicians are interrupted at regular interverals, and must make decisions about how to proceed.
We conducted a study which explores interruption mitigiation stategies for MTR. The prototype and materials used in the study and how to use it are described herein.
If you use this work, please cite our paper:
S. Matsumoto, P. Ghosh, R. Jamshad, and L. D. Riek. Robot, Uninterrupted: Telemedical Robots to Mitigate Care Disruption. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 2023.
@inproceedings{matsumoto2023Robot,
title={Robot, Uninterrupted: Telemedical Robots to Mitigate Care Disruption},
author={Matsumoto, Sachiko and Ghosh, Pratyusha and Jamshad, Rabeya and Riek, Laurel D.},
booktitle={Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction},
year={2023}
}
Emergency departments (EDs) are dynamic, safety-critical environments where healthcare workers (HCWs) are interrupted as often as once every six minutes. These interruptions increase the risk of errors and preventable patient harm. As more robots enter hospitals and EDs, they must support HCWs in handling these interruptions, ideally mitigating the harmful effects of interruptions without disrupting ED communication. However, interruption-mitigation strategies are not well understood, particularly for mobile telemanipulator robots (MTRs).
We created a realistic prototype where people can simulate teleoperating a MTR to conduct patient care in an ED. Using this prototype, we conducted a study in which ED HCWs were occasionally interrupted while using the prototype. Our findings revealed insights on how MTRs might support multitasking in environments with frequent task switching, and the place of autonomy in safety-critical spaces.
Here, we describe the prototype we made and explain how to use it.
You can use the following citation or bibtex if you would like to cite this work:
S. Matsumoto, P. Ghosh, R. Jamshad, and L. D. Riek. Robot, Uninterrupted: Telemedical Robots to Mitigate Care Disruption. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 2023.
Bibtex format:
@inproceedings{matsumoto2023Robot,
title={Robot, Uninterrupted: Telemedical Robots to Mitigate Care Disruption},
author={Matsumoto, Sachiko and Ghosh, Pratyusha and Jamshad, Rabeya and Riek, Laurel D.},
booktitle={Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction},
year={2023}
}
Figma is a design tool that can be used to quickly prototype and test designs. In our case, we designed a control interface and interruption mitigation strategies for a MTR. We then used Figma to prototype and test our designs with ED HCWs.
In Figma, you can make frames and flows. When you run a Figma prototype, a frame is displayed on the screen, and flows transition between the frames. For instance, we made a frame with an image of a medical simulation center room from a camera on a robot. We added a buttons from a control interface we designed on top of this image. Then, we made a flow so that when the forwrad button was pressed, the image changed to simulate the robot moving forward. Fig. 1 shows two consecutive frames in our Figma prototype, and Fig. 3 shows flows between frames.
If you are new to Figma, there are many tutorials you can use to learn how to use it. See some tutorials here.
We collected images for the Figma prototype at a medical simulation center. We reserved a room in the simulation center to record footage of the robot. The room had a patient simulator in a hospital bed and a handheld protable ultrasound, as well as various other medical equipment.
We used the Stretch RE1 robot from Hello Robot in our prototype. We took the Stretch to the simulation center and mounted two GoPro cameras on top of it, near its head camera. We used GoPros because we planned to replace the head camera with a different camera, and the GoPros more closely matched the field of view and resolution we wanted. Because the GoPros were not actuated, one experimenter manually controlled the position of the GoPros during the recording. Additionally, during manipulation tasks, we placed a phone on the robot's wrist to simulate a camera mounted to the robot's wrist, a feature we also planned to add to the robot.
Another experimenter teleoperated the robot in the simulation center and interacted with the patient simulator as if they were providing care for a patient. The experimenter moved the robot around the room, picked up the ultrasound probe, and pretended to administer an ultrasound on the patient simulator. All of these actions were recorded on the GoPros, and some were also recorded on the phone camera.
We used the footage recorded at the simulation center to create our prototype. We extracted frames from the videos and chose frames that were approximately evenly spaced and were not blurry. We added these frames to the Figma. We iteratively designed a control interface for the robot and overlaid the interface on the frames. We then added flows in Figma between the frames so pushing a button on the interface caused the frame to change as if the robot moved. For instance, if a user clicked the button to move the robot's base forward, the frame would change as if the robot moved forward a few feet.
Figure 1: In our Figma prototype, pushing the button to move the base forward caused the frame to change as if the robot had moved forward.
To use the prototype, make a copy of it.
Follow this link and click the "Get a copy" button.
Figure 2: Make a copy of the prototype in Figma.
You can then edit the Figma prototype to fit your needs. You can edit components by clicking on the "Design" tab, and you can edit flows (transitions between frames) by clicking on the "Prototype" tab.
Figure 3: When the "Design" tab is clicked, you can edit components. When the "Prototype" tab is clicked, you can edit flows.
Run the prototype in Figma by selecting the frame you want to start from. Then press the "Present" button.
Figure 4: Run the Figma by pressing the "Present" button in the upper right corner. It will begin on the frame labeled "Start" because that frame is selected.
A new tab will open where you can click through the Figma using the buttons.
Figure 5: When you run the Figma, a new tab opens where you can click through the prototype using the buttons.