Research and Prototyping Findings
Our goal was to test a simple visual to capture health, relationships, and investment in a Medicaid Enterprise System (MES) to supplement existing data sources, based on a framework of depicting different systems and modules as “circles within circles.”
Our hypothesis was that creating a diagram that depicts the relationships, investment, and health of the MES system will help State Officers and DSG leadership quickly identify which components DSG should keep an eye on or do more research into.
We focused on learning: What data to start with / reflect in the diagram If and how State Officers would approach creating the diagram If and how State Officers would find the diagram useful If and how a visual representation would surface new insights for State Officers What State Officers currently base their health assessments on
We tested this in two rounds of one-hour Mural workshops with six CMS State Officers (for a total of twelve 2:1 or 2:2 sessions) between March 10th and April 16th, 2020.
During the first round of workshops, we provided some pre-filled, generic components and little guidance on where to pull additional data from. The workshop was focused on State Officers creating the diagrams.
During the second round, we pulled data from states’ Medicaid Detailed Budget Tables (MDBT) and prepared the diagrams for the State Officers in advance. The workshop was focused on the accuracy of the diagram and the health of the system.
-
The MDBTs are a good place to start grounding the visualization in data about system components & investment sizes, but they don’t tell the whole story.
-
We only tested representing DDI funding, but knowing where operations funding is going is important in understanding the full health of a system.
-
State Officers’ visibility into project health is often limited to what’s in APDs or covered in status update conversations.
-
State Officers were hesitant to designate components as yellow or red.
Based on these insights, we’ll want to make a number of adjustments to the prototype itself before our next round of testing the tool with State Officers. For example, we’ll aim to experiment with new ways to:
- better represent the investments that support multiple components, such as cloud platforms
- represent multiple types of funding (e.g., DDI & M&O) together and/or side by side for each component
Before that next round of testing, however, it feels important to determine how best to frame this tool to State Officers, to ensure it’s clear that this is designed to replace some of the existing reporting they’re required to do and ultimately reduce their workload, rather than adding to it. To do so, we will be mapping out the existing tools and processes that State Officers currently interact with, and identifying areas where this tool could replace existing ones (#4).
It also feels important to begin building out a rubric to back up the red/yellow/green color designations (# 3). We’re aiming for this rubric to help State Officers:
- Know what types of signals indicate that a given effort is red, yellow or green
- Know where to look for for those signals
- Know how to proceed when a given effort is marked yellow or red
In addition to this summary, we also created a more detailed breakdown of our findings 🔒.