New Features/Libraries Question Format (with Inline Follow-Up)

[SachaG]

Feature

Library

What's New

• Ability to group options together for added clarity.
• Any top-level option has the ability to trigger an inline follow-up question when checked.
• Follow-up questions support an "Other…" free-form field
• Note: avoid triggering follow-ups for too many top level options to avoid increasing survey length too much.

[LeaVerou]

Porting comments from the other thread:

@LeaVerou wrote:

Yup, in fact that's basically what I suggested in the email thread:

What if we have a question that appears dynamically if they select "Know what it is, never used it" or "I have used it":
Would you use it in the future?

• Yes
• Yes, once widely supported
• No

Obviously, as with everything that appears dynamically, it should be done in a way that does not make the UI jump around.

Note we need to collect this info even if they have just heard of it, as these may be reasons they didn't use it in the first place.

I really like the multiple options you're proposing here as opposed to just focusing on browser support, though with that many choices I'd worry about cognitive overhead, and it's hard to make the UI not jump around too much when the question is shown.

@SachaG wrote:

I guess it depends on your definition of "used it"… for me that includes trying it out and then realizing it's not usable in production, for example. So I would probably vote for not showing the follow-up when people say they've just heard of a feature, as to me that means they haven't done any actual coding with that feature. I also want to avoid lengthening the survey too much, which I feel like triggering the follow up for 2 out of 3 potential answers would do.

As for UI jumping that's a good point, maybe we can use transitions to smooth it out or something.

Btw for the 2018 State of JS survey we actually had "what do you like about X" and "what do you dislike about X" questions for each of the libraries: 2018.stateofjs.com/front-end-frameworks/react

I don't remember how those questions were phrased but I could see reintroducing them in some form as a follow-up to the libraries questions to parallel what we're discussing for the features questions.

Oh and we could have an "Other…" freeform field too, I think this would be a pretty awesome way of collecting really granular, qualitative data for a specific feature.

I think it'd be cool to also let people know that what they type in that box has a very high chance of being read by people at Google, Apple, Firefox, etc. who can actually do something about those issues!

[LeaVerou]

I guess it depends on your definition of "used it"… for me that includes trying it out and then realizing it's not usable in production, for example. So I would probably vote for not showing the follow-up when people say they've just heard of a feature, as to me that means they haven't done any actual coding with that feature. I also want to avoid lengthening the survey too much, which I feel like triggering the follow up for 2 out of 3 potential answers would do.

I think it's very common to hear about a feature and think "oh, that's cool! I'll try it out when it has more browser support". Many (most?) developers don't want to spend time investing in features they cannot use.

As for UI jumping that's a good point, maybe we can use transitions to smooth it out or something.

That can help, but it's still annoying when it happens over and over.

[SachaG]

Yeah I agree, I just think we need to find the right balance between covering all bases and keeping the survey (relatively) short.

As for the UI jump, I think we'll just have to see it in action. It's not something that usually bothers me too much personally, but if it's really a problem we can always. Anyway for both issues let's decide once we have a prototype. I think it's worth experimenting with at least.

[SachaG]

Related: for features it could be nice to have a "show code examples" button. I initially thought giving people too much details would slow them down but for CSS especially there are now so many features with similar names that it might be worth it. I think we would probably not internationalize code examples though, because that becomes too much work for translators.

[SachaG]

Thinking about how to solve the UI jump issue, here's another idea where we only have the freeform field, and show it in a little tooltip:

(the tooltip could auto-open the first time someone picks that option, and then only open manually for further questions)

[LeaVerou]

Relating to the previous discussion, since you used :has() as an example: If I were filling in the survey today, :has() is something I've heard of, know what it is (even have been pushing for, for years), but haven't actually used due to browser support. 😄

Related: for features it could be nice to have a "show code examples" button. I initially thought giving people too much details would slow them down but for CSS especially there are now so many features with similar names that it might be worth it. I think we would probably not internationalize code examples though, because that becomes too much work for translators.

That's interesting. Could you think of any examples of features that would benefit from this more than the inline code references questions already have?

Thinking about how to solve the UI jump issue, here's another idea where we only have the freeform field, and show it in a little tooltip:

(the tooltip could auto-open the first time someone picks that option, and then only open manually for further questions)

I really like the overlay idea, but not the freeform field. It makes analysis difficult, and is more tedious to fill in. Also, on "Know what it is, but haven't used it" it would cover the third option, which might be jarring (unless it's to the right, if there's space).
Also, if you don't show it after the first time, I can guarantee you that almost nobody will fill it in. There is not enough motivation to actively seek out hidden content in surveys, like there is in regular UIs (and even there hidden content often gets forgotten — "out of sight, out of mind").

[wkillerud]

Also, if you don't show it after the first time, I can guarantee you that almost nobody will fill it in. There is not enough motivation to actively seek out hidden content in surveys, like there is in regular UIs (and even there hidden content often gets forgotten — "out of sight, out of mind").

Agreed. Depending on screen width, I might not register that there is a tooltip at all unless the popup auto-opens every time.

You might also get scroll-in-scroll if there's a long list of options, depending on the viewport. I think I like the initial suggestion better of the two.

[eric-burel]

I guess this has became the sentiment question? Closing for now