Statute research

Research plan

Background

The original vision for eRegulations involved regulations, rules, subregulatory guidance, and other kinds of subregulatory materials such as technical assistance documents, but our users also need to work with policy information in statute.

We’ve heard from SMEs and users in previous conversations that doing policy research in statute is important and tricky work. This research project should help us filter through the information we have, fill in the gaps, and lead us toward prioritized actions we can take to integrate statute into eRegulations.

Research goals and research questions

1. Understand how users need to interact with different sources of statutory information.

• What sources of statutory information could be valuable for addressing needs of our users?
• How do users currently navigate to and search for, with their own tools, to statutory information? How do they feel about their current process - what works, what doesn’t?
• What degree of statute integration as a content type would users expect to see within eRegulations when using statute as a starting point in their work? How does that differ when starting from a regulation?
• How will users understand, distinguish between, and react to the different destinations and content types links could send them from eRegulations?
• What is user awareness of the reliability of statute websites and how do they learn about it?

2. Understand what levels of linking to statute content are necessary and worthwhile for users. 

• What level of linking (paragraph, section, etc.) to statutes is most valuable, somewhat valuable, or essential to our users and does it vary depending on their research task?

3. Understand the risks associated with linking statute content within eRegulations, how they affect our users, and how to manage them.

• What are the risks, and ways people would see them as being reduced, of in-line statute links? Who perceives these risks?
• How do we provide access to up-to-date statute in a way that people can trust and believe in, even if the website is unfamiliar?

4. Identify distinctions among the statute content needs that different user groups have.

• Among our user and stakeholder groups, whose trust do we need the most? Whose trust is hardest to gain?
• How and when will users expect to be able to navigate to and from statute differently based on their distinct needs?
• What are the distinct statute content needs of user groups?

5. Identify the navigation and discovery paths to statute content and how to design them.

• How should users experience statute content as part of search results on eRegulations?
• Where in eRegulations would users find most value in linking to statute content and how would that differ by placement?
• How do we curate and design trustworthy, usable in-text links and supporting design elements to statutes?
• How would a statute index/table of contents page view help users with their statute research?

Expected outcomes

We expect to learn how we should prioritize these features we could build, along with informing how we design and build them:

• Inline links: References to statutes become links (at the section level)
  • Goals 2 and 3, possibly 4
• A statute table of contents: translate US Code section numbering to Social Security Act (or other) numbering
  • Goal 5
• eRegulations search includes a crosswalk of topics to statute and regulation citations
  • Goal 5
• Supplemental content sidebars include links to "related statutes" relevant to each subpart, at the statute section level
  • Goals 1 and 2
• Extract the section and paragraph numbers from the Social Security Act and make them into a new list of locations in eRegulations, which could be connected to regulation locations and supplemental content items
  • Goals 3 and 5
• Read Social Security Act text (or other statutes) within eRegulations
  • Goals 1 and 5

Methods

• Individual informational interviews
• Usability tests with Figma prototypes and existing websites
• Summarizing relevant information from previous research projects

Participants and recruiting

We are targeting different user audiences (as defined at Audiences) for different research goals. The first phase of recruitment will involve scheduling individual informational interviews with internal SMEs to help us better describe what we currently know about our research goals and refine our interview and usability questions appropriately before we speak with CMCS staff.

Following the SME interviews, we will proceed to recruit CMCS staff. Pressing factors to consider in recruiting participants for this study are accessibility and availability. It is important we remain considerate of staff's time and schedules. As such, we will recruit participants by reaching out first to those that have participated in recent outreach activities and expressed interest in talking to us. We'll reach out primarily via email. We will also leverage our Slack help channel for additional recruitment as needed. The table below describes the audiences we aim to learn from for each research goal.

Research Goal	Target Audiences	Description
1. Understand how users need to interact with different sources of statutory information.	* Learners * Researchers	* Conducts policy research frequently with or without eRegs and is able to talk about and describe their process. * Frequents statute sources and content for their work.
2. Understand what levels of linking to statute content are necessary and worthwhile for users.	* Learners * Researchers	* Conducts policy research frequently with or without eRegs and is able to talk about and describe their process. * Frequents statute sources and content for their work.
3. Understand the risks associated with linking statute content within eRegulations, how they affect our users, and how to manage them.	* Decision Makers	* Understands sources of statute, recognize their reliability and can provide recommendations to this end * Has staff that conduct policy research and an understanding of what their staff needs.
4. Identify distinctions among the statute content needs that different user groups have.	* Researchers * Knowledge Keepers	* Be Leads/Managers/Directors with unique teams who can speak to their groups’ needs. * Can recommend eRegs to their groups.
5. Identify the navigation and discovery paths to statute content and how to design them.	* Learners * Researchers	* Conducts policy research frequently with or without eRegs and is able to talk about and describe their process. * Frequents statute sources and content for their work. * Can help us test different prototypes and assess navigation/usability.

Ethics considerations

Transparency and informed consent:

• All participants are employees of our company or our partner agency, and they are familiar with our team's work.
• In our recruiting emails, we explain the subject of the study and the expected time commitment.
• Participation is optional and voluntary.
• We plan carefully and with accurate time estimates so that we use staff time wisely.
• At the beginning of each interview, we describe the purpose of the session and ask for consent to record.

Privacy:

• Interview recordings and transcripts are stored in our partner agency's approved tool (Dovetail) and only used by the team for the intended purpose of improving the product.
• When reporting findings outside the approved tool, such as in presentations or other summaries, participant contributions are kept anonymous.

Timeline

• February 2023: Planning and recruiting for activities
• March 2023: Map insights from past research to current research goals + informational interviews with SMEs
• April 2023: Staff research interviews
• May 2023: Synthesis and findings

Interview protocol

Introduction

Hello! Thank you for taking the time to sit down with me today. I’m [name], a UX Researcher at [company] supporting [product name] team.

Introduce other team members and observers

We often reach out to people like you to learn about your work and get insights about how we can make our tool better fit our users’ needs. We’re continuing to explore what we can do with statute, so we’re happy to talk to you to learn how else we can build this. We’ll be asking you some questions today to this end, but note that this is for our learning and we are in no way testing you or your knowledge. Feel free to decline to answer any questions as well.

Before we begin, are you okay with me recording our conversation today?

Do you have any questions for us?

Sample of questions asked

• How often would you say you find yourself looking up statute citations during policy research?
• How would you say the process of navigating to or accessing statute works?
  • Using an example, could you describe and walk us through your version of this process? It’d be helpful if you can share your screen as you do this.
• We’ve been working to integrate statute into eRegs. What do you think is important for us to know about statute in your work as we continue this?
• For the remaining questions, think about a recent or particular policy research task that you could walk us through.
  • We’re interested in seeing how you look up a statute citation. Do you have a specific example you can use? If nothing comes to mind, consider starting by looking up 1902(e)(16) on postpartum extension. Guide us through how you’ve used this source.
    • If needed: Tell me what you are looking for here.
    • If needed: Why did you use the source that you used?
    • If needed: How well does this site/source work for you?
    • If needed: What does success look like for you here?
    • If needed: Are there other places you’d go to find statute?
  • Find a regulation that cites a statute. If nothing comes to mind, let’s try looking up Simplicity of administration" at 435.902. Guide us through how you get from this regulation to its statutory authority.
  • Similarly to the previous task, think of a piece of statute you’ve worked with recently or are familiar with and navigate to it from here. Preferably a different one if you can, and one that you know is related to specific regulation. If nothing comes to mind, we have an example. Look up 1902(j). Guide us through your process of navigating to it and what you do next. We’re interested in how a statute leads you to a specific regulation, how you make that connection, and what you do after you reach one of these.

Closing

Thank you for sharing your time and expertise with us.

Before we wrap up, I want to pause here to see if my colleagues have any follow-up questions.

Is there anything you want to tell us but didn’t get a chance to earlier?

Do you have any questions for us?

Findings

May 2023 summary

Users are excited about any ways eRegulations can incorporate statute. They spend a lot of time moving between statute, regulations, and resources and they appreciate the potential for eRegs to help them with that. Users will appreciate any ways they can access statute that mirror the ways they can currently access regulations.

Key design problems

• Users across groups do tasks that require them to access statute citations, direct quotations from statute, and/or full statute documents. eRegulations should work to build pathways to each of these things where appropriate.
• Navigating statute text is difficult for users. eRegulations should build features that connect users directly to the statute text they need, that orient them within a statute text, and that allow them to move between statute sections with context.
• When completing a policy research task, users need to know that they’ve been comprehensive in their research. When incorporating statute into eRegulations, we need to be clear about how much comprehensiveness we’re offering in any feature.
• When users need to move between statute, regulation, and resources they may benefit from doing that by a topical structure, as long as it provides enough context about the topics.

Key feature expectations

• Inline statute navigation will help users access statute text as long as it links to the right destinations. This is a partial solution to navigating statute text, as users sometimes only need citations, don’t know where to start, or need context about how to use the statute website.
• A topical navigation is a useful feature for beginning research, but how the topics are grouped and what context we give about the topics is very important to users’ interpretation of how comprehensive, trustworthy, and broadly useful the feature is.
• A statute jump-to could help users quickly get to a reference they know they need in various formats, and could potentially allow for copying the formal citation.
• A statute reference table can help users access statute sections they’re most familiar with in sources they are unfamiliar with, but that are more accurate or more relevant to the task they’re doing.

Impact as of March 2024

We decided to implement inline statute links (within regulation text) and a table of contents page (reference table) for relevant portions of the Social Security Act, completed in July/August 2023.

More recently we enabled individual subregulatory content items to be associated with specific statute citations, not just regulation citations, based on additional discussions with SMEs and staff who rely on statute for their work.

This study also influenced later work on topic-based navigation (lookup by subject), a major new feature.