-
Notifications
You must be signed in to change notification settings - Fork 5
Phase III
The findings from Phase II highlighted usability issues with the wireframe design, including ambiguity in the alarm creation entry point and unclear error recovery options. The team refined the design into an interactive prototype in response to this feedback. In Phase III, the team's focus shifted to validating these design decisions through real user testing. Taking these findings with the user goal in mind helped assess task success, uncover remaining usability issues, and determine which design choices to carry forward into development
We conducted multiple user testing sessions with a think-aloud protocol to gauge how well the current design supports real users. Participants were asked to think aloud as they worked through each task, narrating their actions, reactions, and any points of confusion. Participants completed three task-based scenarios: creating a sequence of alarms for waking up, setting multiple alarms for cooking tasks with different time intervals, and managing multiple alarms by turning them on or off and deleting them in bulk. During each task, the researcher recorded task completion, noted any errors or confusion, and documented observations about usability. After each task, participants rated the difficulty and satisfaction of the task on a 1–5 scale and provided brief explanations for their responses. At the end of the session, participants answered open-ended questions comparing the prototype to their current alarm app, as well as providing feedback on what they liked, disliked, and what features could be improved. All data collected remained anonymous, and no identifying information was linked to participant responses.
The stimuli used was the AlarmFlow Figma prototype, which simulated the core features of the application. The flows covered alarm set creation and alarm deletion. Task prompts were read by the moderator. Prior to starting, participants were assured that moderators were evaluating the application and not the participant. They were informed note taking would occur and that notes would be deleted afterwards. We had 6 participants, all from the class.
Task 1 was: Set an alarm starting at 6:00 AM, ending at 6:25 AM, with a 5-minute interval
Task 2 was: Delete an existing alarm set from the alarm list
Quantitative Results
All six participants completed both tasks within the intended workflow, achieving a 100% success rate for both Task 1 (creating an alarm) and Task 2 (deleting an alarm). Self-assessments of usability and satisfaction (on a scale of 1 to 5) were high for both tasks, with Task 2 receiving slightly higher ratings than Task 1 on both metrics.
| Measure | Task 1 (Create) | Task 2 (Delete) |
|---|---|---|
| Task success | 6/6（100%) | 6/6 (100%) |
| Ease of use | M = 4.33, SD = 1.03 | M = 4.67, SD = 0.82 |
| Satisfaction | M = 4.00, SD = 0.89 | M = 4.17, SD = 0.75 |
The variability in the ratings for Task 1 (SD = 1.03) was driven by two participants who rated usability as 3 out of 5. One of these ratings reflected a bug in the prototype rather than a design flaw; this participant reported that "the drop-down menu does not work." The other reflected a design preference; this participant stated that they "preferred to enter the time manually" rather than using a clock-style interface. Manual entry is in fact supported in the intended design: a keyboard icon on the time-setting screen toggles the input to numeric mode, consistent with the standard Android time picker pattern. However, the Figma prototype did not include the corresponding screen for this flow, so the manual entry mode was not reachable during the session. It is unclear from our notes whether the participant noticed the keyboard icon itself.
- low participate count
- limited by having a non random sample of usability class that is not representative of general population
- (maybe) limitations in figma