Office of the CTO - Digital Experience (OCTO-DE), Facility Locator, Facilities Team
Date: 02/21/2022
Dave Sukharan, Dave.Sukharan@adhocteam.us
Research Report Presentation Recording
Jump to:
Every year, around 6 million searches occur on the Facility Locator application. Eleven percent of these searches are specifically filtered for VBA locations. The VBA specific searches have the capability to be filtered by specific categories. In the field, VBA staff tag their locations with these categories to allow Veterans to search for specific services.
These labels exist to ensure that Veterans at the point of seeking in-person or on-location help are able to efficiently find help. We were tasked with evaluating the efficacy of these labels. Veterans can seek access to these services over the course of their user journeys ranging from transitioning out to education to burial plots.
- Determine how the current categories are used by Veterans searching for services
- Evaluate the usage of the service categories in the Filters list and Location Details pages
- Note any services that could benefit from updated plain language titles
- Do the existing categories affect Veterans' ability to search for services?
- Do existing category names cause confusion, or do they need to be described with more clear language?
- How can the benefits services be best presented to Veterans?
- What common knowledge or other strategies do Veterans utilize when searching for services?
Moderated usability testing which allowed Veterans to reflect on a past experience using VBA services before presenting tasks. Veterans were told that they needed help accessing a service they've used in the past and were tthen asked to use the Facility Locator tool to find help.
-
Veterans will be able to find a VBA location that can help them with their desired services. Likely False Most veterans tested (70%) received a No Results page after completing their search. They expressed confusion and had difficuly redoing their search to find a location. When a location was found, it was often too far away for their purposes.
-
Veterans will have trouble or lose confidence with the process Likely True Most veterans tested (70%) received a No Results page after completing their search. They expressed confusion and had difficuly redoing their search to find a location.
-
Veterans will ask for clarification on the details of some labels Maybe True Most veterans tested had little to no trouble selecting the correct service label from the dropdown list. However, when accessing a Facility specific location page, veterans asked for more clarity on each label to ensure that their assumptions about the label were correct. Veterans are entering these interactions with prior knowledge and, usually, much prior research on the task they are trying to complete. They have an idea of what services they are eligible for and what they mean.
-
Veterans will use satisficing (suitable alternative routes) to achieve their desired tasks. Likely True Most veterans (70%) suggested a non-web based action for solving their problem first. Rather than using VA.gov or the Facility Locator to find help, they opted for directly contacting a person in their network who might have knowledge. A few Veterans frequently rely on the Site Search function rather than the tools suggested via information architecture. It is up to a larger discussion to decide whether these routes are satisfactory or ideal from a VA perspective.
-
Veterans are often able to select the best label for their scenario on the Facilities Search page.
-
Veterans experience significant confusion and increased cognitive load when receiving a "No Results" search.
-
Veterans experience significant anger or frustration when presented with location pages which are “dead ends”: have no services, no open hours, or are closed without explanation.
-
Veterans appreciate any knowledge to help speed their VBA location visit or confirm their prior knowledge and research.
-
Veterans who receive the wrong information (or perceive the information as wrong) can be subject to critical errors and loss of trust.
-
Veterans don't often want to seek help at a physical location. When they do, they are looking for targeted, personal, and specific help.
Veterans are often able to select the best label for their scenario on the Facilities Search page.
100% of tested Veterans selected the best label for their scenario from the Service Type dropdown menu. Note, though Veterans may be confident in their selection, the internal meaning of these labels may not be consistent across all locations and therefore may not match the Veterans' expectations.
Veterans experience significant confusion and increased cognitive load when receiving a "No Results" search.
I know there are three major bases around here. There's no way they don't offer help with this service! I've been to this base. They offer everything here, for sure 40% of tested users were confused by the map delivering No Results
Veterans experience significant anger or frustration when presented with location pages which are “dead ends”: have no services, no open hours, or are closed without explanation.
“Like this page (sparse location specific page)... it pisses me off bitter laugh. I would say an expletive and leave. I want it to be more like the Baltimore location where someone actually put in some care.” 221 (49%) Location Specific pages offer less than 3 Services 18 (6%) of Location Specific pages are currently marked Closed M-F in their listed hours.
Veterans appreciate any knowledge to help speed their VBA location visit or confirm their prior knowledge and research.
“I would want a list of FAQ's. There’s only a limited number of unique inquiries that a Veteran can make. The VA knows what the Veterans - what WE need to know and what we want to know. The call centers are completely slammed, they know what Veterans want to hear.” “This part up here (the megamenu) is great. Every question I have is just a click away. How to apply, change my address, I just click on it... it feels like this part is done and this part (the services list) is not.”
Veterans who receive the wrong information (or perceive the information as wrong) can be subject to critical errors and loss of trust.
Two of the tested Veterans described painful experiences where they needed to travel to the same VBA location multiple times to complete the same task because the employees necessary to complete their task were not there. Veterans who experienced confusion on the map would also talk about the unreliability or opaqueness of VA systems as a whole.
Veterans don't often want to seek help at a physical location. When they do, they are looking for targeted, personal, and specific help.
70% of users chose a non-web based solution first Multiple tested Veterans said that seeking a physical location would not be their first choice. Instead, they prefer to complete extended research on their own, then seek help from the VA or VBA systems only if those failed.
Veterans have a high amount of animosity towards phone trees and calling the VA. At the same time, Veterans often name people (VBA employees, PEBLO officers, VA peers) as the best information sources, though they’re difficult to find. Efficient treatment of the most vulnerable Veterans can increase trust in the system for all Veterans.
-
Reduce the impact of dead ends as they cause critical failures for users.
-
Rigorously audit and increase the quality of information on individual location pages especially to avoid inaccurate hours, empty service lists, or locations which are never open.
-
Give fewer and better next actions for users who get “No Results” searches.
-
Increase the amount of information offered alongside each service label.
-
Provide a link, dropdown, or short, standard description for each service label in the Location Specific Pages to allow Veterans to confirm their understanding of the label’s meaning.
-
Increase the amount of information on each specific location.
-
Provide parking information, detailed directions to find the front desk, or even staff names and appointment slots to allow Veterans to more efficiently conduct their business.
Michelle Middaugh and Dave Conlon will be discussing findings with the VBA Services team to determine viable next steps.
If the labels are significantly changed due to this research, the new labels should undergo a card sort to evaluate the improvement from this baseline. Analytics can also be utilized to observe how many clicks these new labels receive and how many Veterans go on to take further action after selecting a label.
Synthesis Spreadsheet including:
- Observation Frequency Tracker (Rainbow Spreadsheet)
- Tagged Quotes List
- Card Sort Synthesis Results
- Audit of VBA Label Meanings
- Audit of VBA Hours currently being displayed.
Example of Location Specific page: Baltimore VA Regional Benefits Office
Synthesis Spreadsheet including:
- Observation Frequency Tracker (Rainbow Spreadsheet)
- Tagged Quotes List
- Card Sort Synthesis Results
- Audit of VBA Label Meanings
- Audit of VBA Hours currently being displayed.
Synthesis Spreadsheet including:
- Observation Frequency Tracker (Rainbow Spreadsheet)
- Tagged Quotes List
- Card Sort Synthesis Results
- Audit of VBA Label Meanings
- Audit of VBA Hours currently being displayed.