VSA, Facilities Team
Laurel Lawrence, Christian Valla, Shawna Hein 02/23/2021
View the Research Plan.
- Understand how Veterans expectations for Find a VA Location search results differ for mobile vs desktop users
- Understand how industry standards impact Veteran expectations for the Facility Locator search experience
- Evaluate new designs created for geolocation, map view, and clear fields
- Validate if veterans are interested in using the map view of the Facility Locator (validate findings in the mobile month study)
We conducted a remote, moderated, task-oriented usability study, using Zoom
- Participants logged into Zoom with their mobile device, shared their screen and opened up the prototype for us to observe their interaction.
- The participants had to imagine a couple of scenarios and were given tasks to perform within the prototype on their mobile phones.
The structure of the study:
- 2 minutes for introduction and helping participant to share their mobile screen on Zoom
- 2 minutes for warm-up questions
- Approximately 20-30 minutes with usability tasks and questions
- 2-4 min for exit questions and wrap-up
The participants in this study consistend of 10 veterans:
- 5 male and 5 female
- 4 from rural area, 6 from a major metropolitan area
- Between 35 and 83 years old (8 of 10 was 55 years or older)
- A mix of educational level
- 5 used Android and 5 used an Apple iPhone
Search box
- Do participants understand how to enable their location?
- Do participants understand how to clear inputs?
Search results
- What are Veteran expectations of Facility Locator mobile experience?
- Will Veterans notice/understand the Geolocation functionality?
- Do Veterans using a mobile device prefer to view results on the map or in a list?
- Do Veterans expect to find detailed information on the map view?
- Do veterans know how to bring up the detailed information for each location?
- Does the information presented for a given facility match expectations?
- Does the Veteran understand how to find more detailed information about the facilities returned?
- What online maps/searches do these participants find easiest to use/why?
- Participants had a variety of past experiences with the VA facility locator, from “...it was a nightmare” to “...it was pretty easy”.
- Most participants did not utilize the “Use My Location” link. A few did not notice it but said they might use it if they noticed it, a few do not want to use location services at all, and a few participants noticed it and thought it could be useful, but found it irrelevant to the task/scenario given to them.
- Most participants did not use the “x” icon to clear their address and retype a new address.
- Choosing Facility type and Service Type was easy for most participants when given the task to look for optometry.
- Most participants found both the list and map view to be useful depending on their circumstance.
- About half the participants did not notice the map view on their own without prompting.
- Most participants understood that they could tap on a pin to view details in the map, however after that, they didn’t seem to understand overall how to dismiss and view other locations. Overall there was awkwardness in the way participants interacted with the new map view designs and switched between map and list views.
1. Participants had a variety of past experiences with the VA facility locator, from “...it was a nightmare” to “...it was pretty easy”.
All participants had used VA.gov at one point, and 5 of 10 participants had visited a VA facility within the last couple of years.
While a few said that using facility locator in the past was easy, some mentioned difficulties when using facility locator in the past, e.g.:
- "I tried to find a clinic that is near me, but it gave me the main VA hospital and did not show what I was looking for."
- "I had trouble trying to find the appointment phone number and directions. Not being able to see the location on a map was disappointing."
- "I was actually trying to look up locations. It was an absolute nightmare because I tried doing it through my laptop and tried searching for locations but wasn’t able to easily find what I needed. It was difficult even for a VA employee."
There were a variety of reasons folks were not using the "Use my location" link.
- 3 of 10 were hesitant to use the location feature on their phones in general
- "I don’t usually use the, “use my location” option on most sites because I usually have the location of my phone turned off."
- 3 of 10 participants said they saw the Use my location feature and might use it, but they did not think to use it, because of the wording of the task.
- "I would have used the “use my location” option, but I wasn’t sure if it would use my actual location or the location that we are using for the scenario."
- A few others did not see the feature at all, but said they would have used it
- "I would make the font larger. It is good to have that option. It is much easier to just click that link than it is to type in my location."
Most participants did not notice the "x" icon, instead using backspace to clear the field. "I used backspace. I did not immediately see the option to clear the text."
4. Choosing Facility type and Service Type was easy for most participants when given the task to look for optometry.
Overall participants didn't have a problem choosing "VA Health" when looking for optometrists. One participant commented "If I'm going to the VA for support I'm going to look for a VA facility and VA health seemed to register for me for what I was looking for... if I'm looking for an optomestrist, I'm going to go to VA health"
A couple of participants commented that the listed options are not alphabetized. "The options are not in alphabetical order."
Most participantss would start with the list view as they find it quickest and easiest to use, with all the information. They would use the map to get more details about locations and directions.
- "I would probably use the list view more because it just gives me the information I need. If I were new to an area, then I would use the list view first and then the map view."
- "I like the map view. I would go to this view to see which location is the best for me. It is good to compare the location to understand if one of the locations is in an area that I’m already going to be. The map is helpful for people, such as myself, who are more visual."
- "If I needed something really quickly, then I would use the list view. It is the first thing to open and it gives you the closest location and its information." "I like them both. I would use either depending on how I feel."
A little over half of the participants did see the map view, with a few of the participants noting that they decided against tapping it due to the (Washington) task/scenario given to them.
- "I did not think I really needed to use the map – I figured it would show me the map with dots over where the locations are, but I’m not great with maps so I would just use the list and then get directions after I choose one."
- "I noticed the tab (Map View), but since all of these are coming up as being in DC, I didn’t feel I needed to look at it – it isn’t relevant to where I live."
This finding disproves the hypothesis coming out of the Mobile Month study - that most participants would not see the map tab - however we still are seeing a solid chunk of folks missing it.
7. Most participants understood that they could tap on a pin to view details in the map, however after that, they didn’t seem to understand overall how to dismiss and view other locations. Overall there was awkwardness in the way participants interacted with the new map view designs and switched between map and list views.
For participants who went to the map view, We saw them for the most part understand that they could tap on the pins. However, after that, interacting with the cards and using the map view seamlessly with the list view seemed to cause issues for folks.
- "It wasn’t initially obvious that I can tap on the letter, but after a moment I realized after having some experience working with maps like these." "There’s usually an X in the top corner of a box to close it but when I tried it took me to the next location. I just clicked on the View Map tab to close it."
- "The letters are markers for the location, they don’t give any information initially. That is why I would want to look at the list view first, because it gives information that the map view does not."
- "The swiping wasn’t obvious to me, but I probably would’ve figured it out. Maybe adding an arrow to the other box would help."
- Regarding details on the list view and cards
- Three participants wanted to see hours of operation on the card (and "is this open now?")
- One participant mentioned wanting to click on the notification (e.g. "limited service and hours") to learn more about what that meant.
- One participant wanted to see type of services available on the card
- One pariticpant was confused at why the card wasn't customized for his service type, e.g. why was he seeing the mental health line when he searched for optometry
- Some participants were interested in more options as filters before they click "search"
- one wanted to pick radius (in miles)
- one wanted to pick service first e.g. pick Optometry first instead of VA Health - "I don’t know the difference between “VA health” and “community care providers. Community care providers are more like little clinics – they also do optometry, so I was unsure. I probably would’ve done the service type first to see if optometry is available at each facility type. It would be good to reverse the order to eliminate some options."
- One wanted a filter where he could see if there was VA transportation available to bring him to the facility
- Folks didn't care that much about "how many results" they saw, they cared more about seeing relevant results
- If the team is implementing the "Use my location" and "Clear address" features, pay attention to analytics and whether or not they are being used. Consider further testing in staging or production to determine whether the features are prominent enough for those that want to use them
- Do not implement the new "pin and swipable card" design that was tested
- Instead, consider an iteration to the design that does away with the tabs and has the map and list views more integrated with each other, where a veteran could more seamlessly switch between the two views. See Airbnb's responsive mobile experience or Google maps responsive mobile experience for examples.