UX design of results display for legal resources

[nickykrause]

Background

This issue unifies several threads of conversation about the legal resources search results display, which have occurred across several different issues.

We are currently adding advanced filtering mechanisms to MURs, AOs, ADRs, and AFs. As we add new filters to these resources, we need to make changes to the search results display so that users know their filters are working.

By results display, I am referring to the part of the interface that shows the user which resources have been returned via their searching/filtering actions. For example, the results display for AOs is visible in this screenshot, under the heading "Searching advisory opinions for 'embezzle'" :

Clearly, the filters that we include on the left (in the filter panel) will have an impact on what needs to be displayed on the right (in the results display). Although each of the legal resources will have different filters associated with it, we also want to create as much consistency as possible in the interface across resource types.

Finally, we do have an understanding of the different filters that will be used for most of the resource types, but this is not clearly documented in a unified place.

Focus of this issue

This issue focuses specifically on the work of identifying what filters we have planned for the different resource types (MURs, AOs, ADRs, AFs) and outlining the user needs associated with the results display.

Completion criteria

• Documentation of the different filters for each resource type and the design pattern that could be used for those filters
• A documented understanding, informed by user research, of what the users will need from the results display once the filters are added

After this is completed, the visual design of the results display can be documented in a separate issue.

[nickykrause]

(cc @noahmanger, @jenniferthibault, and @vrajmohan)

I have created a document that summarizes the desired filters for each resource type (AOs / MURs / ADRs / AFs). It can be found here 🔒 .

Regarding the first tab of the document:

• There are some outstanding questions in the document, and I have included comments where I plan to follow up with Jennie.

  • If you have thoughts on the outstanding questions, please feel free to weigh-in

• I think this does a nice job of visualizing what we have planned vs. what we have released.

• I have tried to show which pieces of information could be considered "case-level," versus "document-level," in order to begin understanding the implications for the results display.

Regarding the second tab of the document:

• I have included links to what I know to be the most recent version of the designs for these filters, by document type. If the filters have already been released, I have simply linked to the live site. There seem to be some outstanding items for AOs/MURs, but I may be wrong here. (Perhaps @jenniferthibault has information to fill in these gaps?)

Next steps

I welcome input here (especially from @noahmanger), but it seems to me like the next steps are:

• Finish out the design of AO and MUR filters that are still missing (assuming there are any - my document may be incorrectly showing outstanding design needs)
• Design the results display for AOs and MURs, so as to accommodate the full set of advanced filters for each
  • Adapt the results display for ADRs
  • Adapt the results display for AFs
• Design the filters for ADRs and AFs

Note: The results display for ADRs will be virtually the same as that for MURs, since most of the fields are the same. If we can get something workable for MURs, then it seems like ADRs will follow closely. AFs are largely unaddressed at this point, in general, but the amount of information they contain is very small -- the results display for AFs will likely just be a slimmed-down version of MURs/ADRs.

^^ Not sure what order those go in, but I assume we need to do this one first, since it impacts our ability to release the first round of advanced filters for MURs:

• Design the results display for AOs and MURs, so as to accommodate the full set of advanced filters for each

[jenniferthibault]

I'm not quite sure I've entirely wrapped my head around all of this yet, but to get started, I left a few specific questions in the google doc. Here's a few others that were too broad to assign to a cell, or needed some screenshots:

Keyword / case number search:

In each of the universal search instances, we have one field that users can enter either a case number or a keyword in—I've noticed recent mockups and the spreadsheet indicate we're moving toward having two or three separate fields for these (keyword, case name, case number). Is there any testing data pointing toward whether users want the specificity of two/three separate fields, and if not, if one that searches them all and allows multiple entries would be more efficient for space?

Combined

Separated

Would it be helpful to have users select from a type-ahead, rather than pure free-text?

You marked this question in several fields, most often about participants. We have a pattern for this type of search that accepts free text and offers typeahead matching. Remembering from the data side, this type is particularly useful for fields that have human's names in them, which are often hard to spell and subject to error.

(Note: I thought I had made the text highlights match between data & legal, but now see they are still different colors in each section. I'd like to fix this, do folks prefer the aqua highlight, or the grey?)

More to come after I open the design links!

[noahmanger]

This is great @nickykrause . I dropped a few comments in the doc, but don't have much to add.

On the question of whether or not to include typeaheads on those fields, we can look into it, but there might be more or less of a technical lift and I'm not sure the payoff would justify the extra development. I think it's good to prioritize typeaheads where precision is really important (like candidate and committee IDs (which we get by matching against names)) and citations (like we're doing here).

Regarding @jenniferthibault 's question about separate fields: the rationale for that decision is that they're searching different things: the keyword search searches the full text of the documents, while the citation inputs search the metadata associated with a case. It's still totally possible to put in a citation in the text search and it would pull up any matches. Plus, as I understand from some of the usability testing, breaking things out as discreet fields reinforces that the fields are available to search (some people didn't realize you could search citations in the main search box).

[jenniferthibault]

Here's the first set of interaction principles & mockups that have passed an initial gut check by @noahmanger & @nickykrause:

Interaction logic:

• The default state is to present only case-level information
• Show categories/columns only for information/filters that help distinguish one case as a better choice/match than another
• When document-level keyword filters are added, surface text matches with an excerpt that they appear in

Hypothesis to test:
Showing change in results through tag application, appearance, and filter validation in the panel where they are applied will be enough to help users narrow down to relevant cases without surfacing the direct match in the search results, which can easily overwhelm the viewing & comparing experience.

Here are the columns/categories intended to be case-level default at the first pass:

And here's how the results would organize when a keyword filter is applied and matches at the document-level

Next steps:

@nickykrause is going to be checking some of our logic on column choices and unanswered questions in pink above in her interviews with FEC legal users tomorrow.

After that, and a preliminary check with FEC stakeholders, we can then focus on the details of each filter's interaction design, and finesse data category language & consistency with @emileighoutlaw . (Does that sound right, @nickykrause / @noahmanger ? ) Is now the right time to tag in Amy & Jennie?

[nickykrause]

👏 So great, @jenniferthibault! You've taken a very complicated thing and made something elegant 🎉

Your write-up sounds right to me, yes! And as for tagging in the FEC: maybe we wait until tomorrow? I'll have all my interviewees' input by lunch time. If we wait until then, our ping to Amy & Jennie could maybe include some of that user insight, in case it's relevant for them?

[amypike]

This looks AMAZING @jenniferthibault , so thank you very much! I can confirm that the citation for all Admin Fine cases is always 2 U.S.C. § 434(a)/ 52 U.S.C. §30104 so I agree that we probably do not need to surface that information. @AmyKort

[jenniferthibault]

Thanks @amypike ! That's great then, if all the Administrative Fine cases have the same citations, it won't help people pick between their results at the search screen. We can safely locate this info in the case's landing page.

@nickykrause it sounds great to me to revise based on what you learn in interviews then go for deeper feedback on the details. Thanks!

[jenniferthibault]

Based on what @nickykrause learned in interviews today, there is another possibility for information to surface in the search results columns on MURs that would help users choose the cases they want to dig into: showing the latest stage of disposition for any respondent in the case. This, instead of penalty.

Transcript from Nicky:

[Nicky shared the MUR search results draft and asked about the total penalty amount column]

What do you think? is this helpful for determining which cases are relevant?

Yes and no. It is helpful in the sense that, if there is a penalty amount at all, then the case must have gone all the way; it must have reached a later stage, where the penalty was assessed.

But, what is really helpful to me is some indication of how far the case went. There are different stages in the case, like, the RTB finding and conciliation and whatnot, and some cases never get past RTB.

Actually, the majority of cases in the past 10-15 years don’t go past RTB. There is a “No RTB” finding, and then the case stops there. This is because the commission, recently, has been less willing to proceed with cases. So, anything past RTB is a rarity, which means that the majority of recent cases are going to end up with a Penalty Amount of zero.

The ideal situation, if it were somehow possible, would be some kind of column that could indicate the latest stage reached for the case. Like, this case stopped at RTB, but this case went all the way to conciliation. That’d be helpful.

Interesting. One question about that, though: I understand that there are different dispositions for each respondent, and there can be many respondents in a case. So, like, a case could have 10 respondents, and 9 of them have a No RTB finding, but one of those respondents goes all the way to Conciliation Agreement. In that situation, would you want that column in the results display to say ‘Conciliation Agreement,’ even though 9/10 respondents stopped at RTB?

Yes, I see what you mean, but yes — because it matters to me that some part of the case went forward beyond RTB, even if it didn’t go further for every respondent. It is still worthwhile to review the cases where anyone (even if not everyone) went beyond RTB.

Here's a first pass at what that would look like, with the caveat that we have not yet thought about how to really phrase that content category so that it works well as a table header:

[nickykrause]

Okay, so now that we have some user feedback incorporated, we have proposed designs and some associated questions that could use input from both Jennie and @amypike. (Jennie is not on GitHub, so I will ping her via email and point her to this issue.)

As Jen explained above, we have designs that depict how the search results could appear for AOs, MURs, ADRs, and AFs.

Across all of the designs that Jen provided, we have the following general questions:

• Do any of the interactions or results that we're showing in these designs seem inaccurate to you, given your familiarity with these cases? We'd love it if you could flag things that we may have overlooked!
• What do you think of these designs as a first pass at meeting the users' needs, with more testing and iterations to come?

For the MUR designs, we have one additional question, which is as follows:

• For our first iteration of the MUR results, do you have any preference as to whether we show the Total penalty amount column, or the Latest disposition stage reached column?

[AmyKort]

Can you help me understand the difference in the data between "Latest disposition stage for any respondent in the case" and the "Final Commission Disposition" filter currently in EQS?

[amypike]

@nickykrause These all look GREAT to me. The interactions seem accurate and I think the design is solid (and clean and pretty!). I think surfacing information about the latest stage in a MUR is a great idea, I'm so happy that user research revealed that. With respect to total penalties versus highest penalty per MUR, my gut says that highest penalty would be what an attorney would find most interesting ("if my client/a respondent did x, what's the most they would have to pay?") but my gut is frequently wrong. Am flagging this for Jennie and @AmyKort for their thoughts.
Whether we should display the total penalty amount or the latest disposition stage question, I would think disposition would be more important since the majority of cases don't reach the penalty stage, but again, would love other people's thoughts.

[amypike]

@nickykrause Okay, now that I'm thinking about it, if we decide to surface the total penalty amount for all respondents in a MUR, what might also be helpful to surface at the same time is the total number of respondents in that MUR. A case where the penalty is $100,000 with 20 respondents is maybe less serious than a case where the penalty is $100,000 but there are only two respondents. I will shut up now.

[jenniferthibault]

@amypike no 🤐 ! We need your thoughts!

@AmyKort my understanding (which I would love other people to check!) is that when you select something from the "Final Commission Disposition" filter currently in EQS, like this:

Case results are returned to you where at least one respondent received the disposition you selected. So the above filter returns this to me as one case in the results because at least one respondent received "PC/NFA" as a disposition:

I am making an educated guess that the reason that EQS uses "Final" Disposition in the filter label is so that the label could be used for MURs, ADRs and AFs at the same time. Since Administrative Fine cases could be challenged and receive a different final disposition from the Commission than what was originally recommended at the Reason to Believe stage, I'm guessing they added "Final".

.

For the search results displays that we're looking at above, the need is to help users scan the list of cases returned once they've applied filters (which at the moment are not shown in the design to keep it easier for us to focus) and determine which results are most relevant to what they're looking for.

In interviews, a user surfaced that being able to get a sense of "how far a case had gone" for any respondent is a signal for how interesting the case might be to them. Our column "Latest disposition stage for any respondent in the case" is an attempt to meet that need by surfacing the most advanced stage of disposition that any respondent received. This will require us to put the disposition possibilities in order from least/most advanced, but we could probably start with the order set here:

I think the most basic difference between the two is that in EQS, users are currently unable to scan the list of matching cases returned for any hint of dispositions rendered. Our proposal here isn't to then surface all of the dispositions because some MURs have dozens of respondents that would render the interface pretty useless for scanning quickly. Rather, give a hint at how far the case progressed by surfacing one of many dispositions.

....that was long. Does that help? Glad to talk this through in person if I waxed incoherent :)

[AmyKort]

Thank you @jenniferthibault ! That made this so amazingly clear. Thank you for taking the time to walk me through it. I like the display visually. I appreciate the user need to quickly identify cases that are significant or interesting. I'm not sure we're going to be able to get there from here when we put this into practice. Can we carve out some time to talk about this?

[nickykrause]

@AmyKort, regarding your question about the difference between EQS's Final Commission Disposition and our proposed Latest stage reached indicator, I would like to add this:

I believe the search for Final commission disposition in EQS works the way that @jenniferthibault described (but we'd need Jennie to confirm). For example, here is a search I did in EQS for MURs where the Final commission disposition is No RTB:

As this screenshot shows, the EQS search results include cases where some respondents actually went beyond RTB...even though some of the respondents had Dispositions of No RTB, others went all the way to Conciliaton.

Our Latest stage reached column would help users see the furthest a case had reached for any respondent, which is slightly different from showing whether or not any respondent had reached a designated stage.

[nickykrause]

Annnnd it looks like you were already all set with your answer! 😄

[AmyKort]

Thanks, @nickykrause !

[jenniferthibault]

After a really helpful feedback session with @AmyKort & @JennieHBee (welcome to GitHub! 🎉 ) we concluded:

For MUR categories:

• the dispositions are helpful enough to disambiguate cases that they should be surfaced, but our first approach could have both positively and negatively singled out respondents, which we should absolutely avoid doing. We decided to try changing approaches, and instead of showing the "Latest/Furthest disposition for any respondent in the case" which would draw a nonexistent linear progression in case steps, we'll list the different types of Dispositions in the case, and how many respondents they applied to.
• To first test showing just "Election cycle" but consider also showing "Closing date" if we see signs that users are having trouble disambiguating cases

For AOs
No change at the moment

For ADRs
No change at the moment

For Admin Fines
Remove the citations column, per @amypike 's report that all Admin fines cite the same thing, so no need to prioritize this on the search results. Citations will appear just on the case's canonical page.

No changes to the search interaction plan for now.
@nickykrause is going to be showing these to FEC users at the beginning of next week.

I'm going to move this issue to "Done" pending the results of those tests.

[AmyKort]

Thank you!

[jenniferthibault]

@nickykrause 's research in https://github.com/18F/fec-testing/issues/26#issuecomment-273645543 raised one issue that we thought was clear enough to incorporate in document keyword matching across all categories: better expectation setting for how many matches a document contains.

This concern also came up when I showed the work to other 18F UX designers not on this project. @nickykrause and I discussed, and thought that the smallest amount of hinting we could do to see if it met the need is to include the number of additional matches in the doc as a text string. The # would not b clickable, but intends to give the user a sense of how prevalent their term is in the doc.

[nickykrause]

Looks good, @jenniferthibault