Repository navigation
Phase II
The findings from Phase I helped us identify core features that our competitors do not have. In Phase II, we used our sketches to build a draft UI, or wireframe, where external evaluators conducted a cognitive walkthrough to provide feedback on our design. We also gathered informal feedback from our demonstration to a classroom of undergraduate software engineering students.
A cognitive walkthrough is a usability evaluation method used to identify usability gaps between a project's wireframe and personas. External evaluators use a defined persona and scenario to interact with the wireframe. Their goal is to complete the scenario's objective while staying "in character" and making decisions based on the persona's characteristics, needs, and behaviors.
Each of the participants (n=3) were told to pick a persona and follow their scenarios. The following personas and scenarios were available for this walkthough:
-
Thomas
Thomas has been practicing with his new telescope on the balcony of their apartment, but wants to go somewhere with less light pollution. His mom is going to start planning a trip.
-
George
George just had a double shift at work right before the week ended and hasn’t had the time to plan and discuss with his family about the imminent stargazing they could do the next day. While eating dinner with the family, he vehemently complains to his wife that he would have preferred to get notifications about future events at a reasonable time before they happen.
-
Sage
Sage is wanting to propose to her boyfriend at their favorite hiking spot. She wants the occasion to be special, so she is hoping that there is a full moon to create natural lighting through the trees and shrubbery for their proposal photos. She also hopes he says yes.
-
Jordan
Jordan is an elementary school teacher in Waynesboro, Virginia. He has been working there for a few months and is looking forward to his upcoming space material because it reminds him of his brother George. He is looking for a website that provides information on astrological events and space news in order to provide a fun learning environment to his students.
Once each participant had their persona and scenario, they were told to go through our wireframe and ask the following questions before each interaction:
- Will the user know what to do at this step?
- If the user does the right thing, will the user know that they did the right thing and is making progress toward the goal?
The participants answered these questions in their feedback, which are discussed in the Findings section below.
Informal feedback is method to gather feedback about desired features, potential improvements to the user experiance, and other qualitative aspects of the application.
The Skytracker Software Engineering team demonstrated what has been accomplished within Phase I to a group of their peers, consisting of undergraduate software engineering students. After their demonstration, they were allowed to ask the audience two questions.
The two questions were:
- How much privacy would you give up for location services within Skytracker?
- What features should added to Skytracker?
One of the Skytracker Software Engineers documented the audience responses, which are discussed in the Findings section below.
The participants (n=3) of the cognitive walkthrough found that the 'Export' feature of the 'Calendar' page was confusing. The wireframe did not have functionality if the user clicked on the 'Export' button, so there was no indication that the export function worked. It was also noted that while .ics files may be a standard convention for calendar applications (ie. Google Calendar), users might not know how to interact with the file after it has been exported from the site to their device.
Listed below is a summary of the feedback from the participants of the cognitive walkthrough for each persona and scenario.
-
George
George wants to get notifications about future astrological events. He followed the summarized steps below.
1. Search website through Google
2. Enter Homepage
3. Got to 'Calendar'
4. Click 'Export'George can easily get to the website and the correct page of the website. He is a little confused by the 'Export' feature and is unsure if there is another way to do this. He can do it, but it may require extra effort (ie. tutorial) to get it fully functioning
-
Sage
Sage wants to go camping and on dates with stargazing in mind, so she wants notifications. She followed the summarized steps below.
1. Search website through Google
2. Enter Homepage
3. Got to 'Calendar'
4. Set location and filters boxes within 'Calendar'
5. Click 'Export'
6. Navigate to 'News'Sage can easily get to the website and the correct page of the website. She is a little confused by the 'Export' feature because she clicked 'Export' and there was no confirmation.
-
Jordan
Jordan wants to find astrological news for his elementary classroom. He followed the summarized steps below.
1. Search website through Google
2. Enter Homepage
3. Scroll down to view more 'Featured Events'
4. Click and view on a 'Featured Event'
5. Go to 'News'
6. Search for desired articlesJordan can easily get to the website and the correct page of the website. He is not confused about any of the interactions he had.
The Skytracker Software Engineering team received informal feedback from their peers, consisting of undergraduate software engineering students. Notable concerns during the demonstration were location privacy, and a lack of cardinal directions provided by the prototype.
The students also suggested an array of possible features, including:
- Red light mode for night vision
- Providing information about visibility for the user such as cloud cover, elevation, and light pollution.
- Links to reference information about constellations
- More types of astronomical events, e.g. meteor showers
- A widget for phone users
From what we have discovered in Phase II, we need to adjust our design to better follow the usability guidelines "Visbility of System Status" and "Match Between the System and the Real World".
It was noted from the cognitive walkthrough participants that the 'Export' feature lacked a system status, so users were left confused about whether the feature functioned properly. We feel that by adding a pop-up to nofity the user the file was successfully downloaded or added to their calendar (if that feature is possible), then we will greatly improve our "Visbility of System Status".
It was also noted from the cognitive walkthrough participants that the 'Export' feature could cause confusion as to whether it is the "correct" method to get notifications. We feel that by adding additional methods of getting notifications (ie. SMS, email) or improving how the export function is received from the user (ie. change from 'Export' to 'Add notifications') would greatly improve our "Match Between the System and the Real World". By changing how users receive their notifications or what the 'Export' function is called, we can improve a user's mental model and reduce possible frustrations.
It was noted from the informal feedback participants that there are concerns regarding user privacy and the use of location data for the calendar feature. We feel that implementing a simple pop-up explaining how Skytracker uses location data could help with "Visbility of System Status" because the user is informed what the data is being used for.
Considering all this, it should be noted that the team size and completed research were limited in size and scope due to expected deadlines. The presented methods also relied on peer review, or some "in character" version of it, which can have inherent biases.
The cognitive walkthrough was limited in scope with only a few external evaluators (n=3). Due to the number of participants, the Thomas persona was never evaluated.
The informal feedback was limited in scope, as each group was allowed only two questions within their allotted demonstration time. The received feedback was from a classroom full of undergraduate software engineering students. As a result, they were not screened according to their interest or expertise, and produced unpredictable suggestions regarding what features should be included in a MVP.