-
Notifications
You must be signed in to change notification settings - Fork 5
Phase III
The findings from Phase II highlighted usability issues with the wireframe design, including ambiguity in the alarm creation entry point and unclear error recovery options. The team refined the design into an interactive prototype in response to this feedback. In Phase III, the team's focus shifted to validating these design decisions through real user testing. Taking these findings with the user goal in mind helped assess task success, uncover remaining usability issues, and determine which design choices to carry forward into development
We conducted multiple user testing sessions with a think-aloud protocol to gauge how well the current design supports real users. Participants were asked to think aloud as they worked through each task, narrating their actions, reactions, and any points of confusion. Participants completed three task-based scenarios: creating a sequence of alarms for waking up, setting multiple alarms for cooking tasks with different time intervals, and managing multiple alarms by turning them on or off and deleting them in bulk. During each task, the researcher recorded task completion, noted any errors or confusion, and documented observations about usability. After each task, participants rated the difficulty and satisfaction of the task on a 1–5 scale and provided brief explanations for their responses. At the end of the session, participants answered open-ended questions comparing the prototype to their current alarm app, as well as providing feedback on what they liked, disliked, and what features could be improved. All data collected remained anonymous, and no identifying information was linked to participant responses.
The stimuli used was the AlarmFlow Figma prototype, which simulated the core features of the application. The flows covered alarm set creation and alarm deletion. Task prompts were read by the moderator. Prior to starting, participants were assured that moderators were evaluating the application and not the participant. They were informed note taking would occur and that notes would be deleted afterwards. We had 6 participants, all from the class.
Task 1 was: Set an alarm starting at 6:00 AM, ending at 6:25 AM, with a 5-minute interval
Task 2 was: Delete an existing alarm set from the alarm list
Quantitative Results
All six participants completed both tasks within the intended workflow, achieving a 100% success rate for both Task 1 (creating an alarm) and Task 2 (deleting an alarm). Self-assessments of usability and satisfaction (on a scale of 1 to 5) were high for both tasks, with Task 2 receiving slightly higher ratings than Task 1 on both metrics.
| Measure | Task 1 (Create) | Task 2 (Delete) |
|---|---|---|
| Task success | 6/6（100%) | 6/6 (100%) |
| Ease of use | M = 4.33, SD = 1.03 | M = 4.67, SD = 0.82 |
| Satisfaction | M = 4.00, SD = 0.89 | M = 4.17, SD = 0.75 |
The variability in the ratings for Task 1 (SD = 1.03) was driven by two participants who rated usability as 3 out of 5. One of these ratings reflected a bug in the prototype rather than a design flaw; this participant reported that "the drop-down menu does not work." The other reflected a design preference; this participant stated that they "preferred to enter the time manually" rather than using a clock-style interface. Manual entry is in fact supported in the intended design: a keyboard icon on the time-setting screen toggles the input to numeric mode, consistent with the standard Android time picker pattern. However, the Figma prototype did not include the corresponding screen for this flow, so the manual entry mode was not reachable during the session. It is unclear from our notes whether the participant noticed the keyboard icon itself.
Qualitative Themes
Three recurring themes emerged from the think-aloud sessions and post-task comments.
Strengths: Participants consistently noted that the core workflow was easy to understand and that, in scenarios involving multiple alarms, it was easier to use than the alarm app they currently use. Comments such as “It’s quite easy to understand” (p01), “It’s easy to use” (p02), and “The prototype made the task much easier” (p04) were observed across both tasks. Several participants clearly rated the prototype more favorably than their current solutions. p02 stated, “It’s actually better because I only need to set one set of alarms instead of multiple,” while p04 commented, “It’s excellent because it has so many features.” The delete function received particularly strong support; p05 pointed out that it was easier than managing a physical alarm clock, and p06 highly praised the “ease of access” for the delete operation.
Interaction for entering the time: The clock-style time picker was one of the few elements that participants expressed dissatisfaction with. One participant (p05) stated a clear preference for manually entering numerical values over an analog-style clock UI. Although the intended design supports switching to manual input mode, the corresponding screen transition was not implemented in the Figma prototype, so it was not possible to experience manual input mode during the session. It is unclear to what extent this limitation influenced the participant’s evaluation.
Gaps in Customization: The most frequently requested feature raised spontaneously by participants was related to customization. Specifically, these included the ability to upload custom alarm sounds or audio files (p01, p03, p04), dark mode and visual theme settings (p02), naming individual alarms (p03), longer interval options such as daily or weekly (p05), and automatic deletion of expired alarms (p05). Regarding visual refinement, there were also comments regarding font selection (p01) and color schemes (p02).
Comparison with Current Alarm Apps
When asked how the prototype compared to the apps they currently use, 4 out of 6 respondents answered that this prototype was equal to or better than existing solutions, citing efficiency in managing multiple alarms and the cleanliness of the UI. The remaining two gave more reserved evaluations. p05 stated, “I’m not sure if I’d switch,” adding that they would likely continue using both their current app and a physical clock. p06 said, “I can’t say either way,” noting that the prototype “lacks the features I want.”
The results of this study indicate that the core design of the prototype effectively supports the efficient management of multiple alarms. A 100% task success rate and consistently high ratings across both tasks confirm that the basic workflow is sound. Below, we first present the design elements supported by the data, followed by specific recommendations for improvement.
Aspects Affirmed to Remain As-Is
The Concept of “Alarm Sets” as a Core Architectural Model: Participants clearly appreciated the ability to configure a series of alarms as a single unit, rather than creating each alarm individually. p02 cited this as a clear improvement over the app they currently use, confirming that the central design decision carried over from Phase II was appropriate.
- low participate count
- limited by having a non random sample of usability class that is not representative of general population
- (maybe) limitations in figma