Design in-context feedback mechanism for interactive data modules.

[jenniferthibault]

As seen here, we know that we want to offer users the ability to give quick, non-distracting feedback on the new data modules, that helps us learn if we are on track to providing the right information. We are trying to balance a few factors: 1) level of detail in the answers we receive 2) perceived time and effort on behalf of the user to complete the questions.

Here's an idea of how this potential interaction could behave:

Open questions:

• Is this the (type of) question we want to ask users?
• What answers do we want to offer to select from?
• What should the resting interaction be? (I.e., should we put any call to action at the end of the sequence, or just let users navigate onwards on their own?)

[jenniferthibault]

cc @onezerojeremy , @noahmanger

[onezerojeremy]

Whoa nice! I especially love the faces!

I read the confused as "meh" a bit but i know i'm out of the mainstream on that (you're following the standard)

possible interaction tweaks:

• maybe put "same as before" in the middle. or! perhaps replace it with something like "don't much care about this particular data"?
• immediately after selection, consider launching an open text box and active cursor with a prompt just saying something simple like "Why?" or "Tell us a bit more!" and then route it to the feedback issue stream. (this way we don't make them click out of the stream, making it as easy as possible for them to say more) and give them a button(s?) to submit that or nothing.
• after they submit that, tell them thanks for helping us improve, and perhaps have a link to "talk to a human" which links to a dedicated ethnio for this feature that we can keep an eye on.

As to the bigger question of whether or not this is the question we should use, I think this one is straightforward and might get us a first blush take on if people get it (and if we add the "don't really care", if they care). Also it's a bit more compelling because of the faces. I think there might be ways to make a more "poll-like" feature that feels more interactive, but that feels like it might be more trouble than it's worth and I'm content to let it ride for now.

[noahmanger]

1. I think this is a great way to ask the question. It feels nice and open ended, but focused enough to get useful data.
2. What about "Confused" "Informed", "Don't care" and "Other", where selecting "other" really emphasized providing more details in the feedback tool. (I think having an "other" option is good practice). I kind of like "Don't care" as a replacement for "same as before" because I think it basically means the same thing, but is a more specific interpretation of "same as before".
3. I don't think we need any more call to action to the resting state.

Other thought: what if we hid this behind a "how are we doing?" button next to "share?" That way, frequent users of the site can just ignore it. Though I'm sure it means less feedback, so maybe not a tradeoff worth making.

[onezerojeremy]

I agree with all Noah's comments except hiding it, at least at first. I do agree inasmuch as I think it'd be good to design a version without it (though I'm content to leave that for later as it may be that we come up with different interactions or questions).

[emileighoutlaw]

I think there's a higher cognitive burden in asking folks to read an introductory sentence to the feedback options and select an option. I recommend streamlining this for ease of scanning:

How are we doing?

• This chart was informative
• I feel neutral about this chart
• This chart was confusing

(The ordering here also starts at most positive and cascades toward negative)

I also like the idea of giving folks a button option to leave more feedback, rather than leaving a long sentence of instructions on how to leave feedback. I'm imagining something like:

Thanks for helping us improve!
[ Leave more feedback ]

[onezerojeremy]

Oh interesting! I think maybe we can simplify it further!

What if instead of "how are we doing", which is pretty vague, we ask about the chart? what would be a question which could prompt the user to understand the really concise, highly scannable options like

How does this chart make you feel? *

• better informed
• confused
• don't care about it
• none of those

Upon further reflection, I don't think a neutral option is very telling (but could be convinced otherwise!). What do we learn that is actionable if people feel neutral? Will neutral people click?

Ideally each would change the prompt on the feedback box:

• better informed : Great! What did you learn?
• confused : Sorry, our bad!* What didn't make sense?
• don't care about it : Sorry, our bad!* Why isn't it compelling?
• none of these : Ah, ok - what is your reaction?

*definitely not actually suggesting that exact language

[emileighoutlaw]

Oh I love this idea @onezerojeremy!

How does this chart make you feel? *

• Better informed
• Confused
• Don't care
• None of those

Follow-ups (clearly marked as optional, maybe with a skip and submit button?)

• Better informed:

Great! What did you learn?

• Confused:

We're sorry to hear that. What didn't make sense?

• Don't care:

We're sorry to hear that. What would you like to see?

• None of those

How we can make this better?

I loooove the language you suggested, but I think it might be a little too playful for FEC

[noahmanger]

Love this. Y'all are brilliant.

[jenniferthibault]

Seems like the benefit of neutral might be for someone who cares about the data, but feels meh about how it's shown. I don't have strong feelings about keeping it or dropping it in favor of "other" though. Defer to the rest of you. I did switch the "confused" face to what was previously "don't care" in this version, because with both options, they needed to be different.

I'm late to the language game, but I come bearing mockups! The only thing I'm not 100% on, is having a government website ask people how something makes them feel. I love the human language, but this feels ripe for a troll attack, or satirical poke from the press. (i.e., "A government agency wants you to talk about your feelings more. Next week: it's not us it's them." I'm white-knuckling my mouse already.) Language that focuses more on the site's performance or relevance feels more accurate to our intentions. Even if we asked it in a way that didn't use the word "feelings" that would even seem better.

Interaction Hypothesis: We want to make sure that feedback is captured even if someone doesn't engage with the skip/submit option at all, right? So I've suggested that the feedback start being tracked in the system as soon as a user clicks a reaction. No idea of the dev lift on this, but it would enable us to capture the minimum engagement effort.

[emileighoutlaw]

I think your interaction hypothesis is spot on, Jen. And I like these mockups a lot!

My intent with the language was that we could drop the How are we doing? entirely in favor of only having How does this chart make you feel? as the title, with no subtitle below. (Less cognitive burden on visitors because there are fewer words to read).

On the thanks screen, let's say:

Thanks for helping us improve!
Would you like to test more new features with our team?
[ Sign up ]

[onezerojeremy]

Love the interaction hypothesis, love the options as well, and agree that a single pointed question should be the goal. If we want to edge away from feelings, perhaps What do you think of this chart? or What do you think after looking at this chart? or something in that vein?

[emileighoutlaw]

Oooo. I like "What do you think of this chart?"

(how does it make you feel seems a little too woo woo-y to me.)

[jenniferthibault]

Aha! I hadn't 100% grokked dropping the headline out of earlier comments, thanks!

❤️ for "What do you think of this chart?". Thanks for feeling the feels on the feelings. When I dropped that in, it became clear we needed to tweak the tense of the answers to match. I took a stab:

[onezerojeremy]

bellicimo, Jen!

imho

[emileighoutlaw]

Love it!

[AmyKort]

I wonder if "don't care" is asking for too much of an emotional response to a chart, or to the information in the chart, or campaign finance, or the election, or my decision to click on a chart when it turned out that the subject matter didn't interest me, or my decision to take a survey about an experience I didn't care about. Maybe we want to know whether the chart was interesting (or, I guess, uninteresting). I think that is a different category from informative. I once took a whole semester lecture on Hegel that was both informative and confusing, but was not interesting. I would not click on that lecture again.

It's very hard to come up with a short word or two that captures what we're looking for . . .

[onezerojeremy]

we could try and swap out don't care for not interested and see if that prompts folks to click (and what comments they leave)?

[AmyKort]

Sounds good. Would it be "not interesting"? Because it is informative, confusing, or not interesting?

I think you actually just clarified for me the concern I've been been trying to articulate, which is that I'd rather focus on the response to the chart (with a laser-like focus that helps us isolate that response from responses to our other content and activities).

But, I also want to stress that this is totally your wheelhouse, Jeremy, and if I'm pointing us in the wrong direction, just tell me and I will thank you for your insight and your honesty. Which I will do anyway.

[onezerojeremy]

Thanks Amy! Seems like that could work; let's try it! I think we should get it close, launch it and tweak the language to see if different wordings change our results.

My earlier thought process:
Grammatically, not interesting certainly matches the other options better and I'm game to give it a go. I was initially drawn to "not interested" over "not interesting" because the former is a statement the tentative new user (that these charts are targeting) is making about themselves: "I'm not interested." This feels a bit more friendlier (and perhaps therefore clickable) than "That's not interesting." which feels more universal and negative (to me), but it's A. a really minute detail and B. one that I have no proof in support of other than my own gut (not sure Hegel would approve) so C. let's try not interesting.

Also busy @emileighoutlaw is the pro here and I will defer to her judgment.

[emileighoutlaw]

Hello! Sorry for my belated response!

I'm game for trying not interesting because it takes the burden off the user (this thing is not interesting), whereas not interested maybe shifts it a little more onto the user (I'm not interested, and maybe that's my fault). As person who is regularly guilty of self-blame, that's immediately where my mind jumps.

All that aside, it'll be good to just implement one, see what kind of response we get, and adjust from there. 😄