2017 survey results

[tfausak]

The survey (#102) went out last Wednesday. I had originally planned to keep it open for a month (through December 1), but now I think I'll only keep it open for a week (through November 8). New submissions have already slowed to a crawl (less than one an hour), and I've received more responses than I've expected (more than 1,100). I would rather close the form and do something with the results than leave it open for a long tail of submissions to trickle in.

So, on that note, here's what needs to be done with the survey data:

• Collect all the responses and export them to some useful format, probably JSON. This is for private use and will not be shared.
• Strip any potentially personal information from the responses and export them to JSON and CSV. This is for public use and will be shared. It's not yet clear if free-form answers should be included or excluded from this data.
• Graph all the basic multiple-choice answers. The graphs should be self-explanatory so that they can be posted to social media without being misconstrued. For example, they should include the year, original question text, original answer choices, and actual data labels.
• Analyze all the free-form answers and synthesize them into some key points.
• Put all that together into a blog post or infographic or something. This is a little weird because Haskell Weekly hasn't published original content in the past. Should it start now, or should I post the results on my blog and feature them here? I'm not sure yet.

[saurabhnanda]

Analyze all the free-form answers and synthesize them into some key points.

Please release all free-form answers publicly. Email the respondent for permission, if required. I can help you with the emailing and collecting responses, if need be.

[tfausak]

I would personally prefer to release free-form answers. I'm hesitant to for privacy reasons. Emailing to get consent would be good, but many people did not supply an email address. Also that would take a while and might even be construed as spam.

In the future, I might put a checkbox on the form that allows people to say if they want their free-form answers shared or not. You could say that since there wasn't one this year, I can do whatever I want with the responses. That may be true, but I don't want to betray the community's trust. After all, I want them to take the survey again next year.

Also it's entirely possible I'm wringing my hands about this when in fact it's not that big of a deal. Based on the responses that I've read so far, they aren't very private or personal.

[saurabhnanda]

What percentage of people haven't supplied email-ids?

[tfausak]

25% of submissions don't have an email address.

[saurabhnanda]

Also that would take a while and might even be construed as spam.

Will you be emailing the 75% participants once the survey results are out? Two options:

• I don't think one well-intentioned email to take permission, before the results are out, will be construed as spam by the community.
• Alternatively, we can add a big button in the email announcing survey results to give permission to publish that respondent's free-form responses. Clicking that can automatically publish the survey results on a dynamically generated page. I can get this setup for you in a day, if you're fine with this.

[tfausak]

Yeah, it would probably be fine to send an email. I'll think about it.

I don't think we need to set up any system for managing permissions.

[tfausak]

Amazingly, a couple of submissions have trickled in since I closed the form yesterday. People keep browser tabs open for a while.

[tfausak]

Here's what I'm thinking for publishing the free-form answers:

• When I publish the results, I'll send an email announcement to everyone that entered an email. (Next year I will be clearer about what emails will be used for. For this year, a single "transactional" email feels harmless.)
• The announcement will include a notice that I will publish your free-form answers unless you respond telling me that you want to keep them private. (Next year I will have a checkbox in the survey for this approval. For this year, the survey did not specify one way or the other and so far the responses have been harmless.)
• I will give people a week to respond, at which point I will filter the responses and publish them. (Note that none of the published information will include email addresses.)

I think this accomplishes several things:

• It allows people to keep their information private if they want to, in spite of the fact that the survey did not have a way to express this desire.
• It avoids sending unnecessary emails to people and/or setting up a program/service to handle this.
• It (hopefully) keeps me in the good graces of the community.
• It makes it clear what needs to be done for next year's survey.

[saurabhnanda]

Sounds good. Although, be ready to develop a thick skin against people who will inevitably call this a dark UX pattern.

[tfausak]

Results published here: http://taylor.fausak.me/2017/11/15/2017-state-of-haskell-survey-results/

I still need to send out an email to people that responded.

[saurabhnanda]

Just received the email. Thank you for your effort in putting this survey together :)

…

On 16-Nov-2017 12:41 AM, "Taylor Fausak" ***@***.***> wrote: Results published here: http://taylor.fausak.me/2017/ 11/15/2017-state-of-haskell-survey-results/ I still need to send out an email to people that responded. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#125 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABu0ctWYYXbrjj7Ph5AMn0Di1O6fkEJks5s2zd4gaJpZM4QSlpH> .