Survey

[tfausak]

I recently saw the Rust survey results: https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html

I know of a few Haskell surveys from the past:

• http://blog.johantibell.com/2010/08/results-from-state-of-haskell-2010.html
• http://blog.johantibell.com/2011/08/results-from-state-of-haskell-2011.html
• https://www.fpcomplete.com/blog/2015/05/thousand-user-haskell-survey

There may be more, but I don't know about them.

The previous Haskell surveys didn't get as many responses as the latest Rust survey:

Survey	Responses
Rust 2017	5 368
Haskell 2015	1 240
Haskell 2011	798
Haskell 2010	804

Regardless, I think it would be good to survey the Haskell community. Probably on a yearly basis. Seeing as I run Haskell Weekly, it makes sense to me to send the survey out through Haskell Weekly. I don't yet know what I want the survey to look like or when it should go out. That's what this issue is for 😄

[alexeyzab]

Sounds like a good idea!

Using the Rust survey as a base and mixing in some questions from the older Haskell surveys would be nice.

In terms of what it should look like, I recall Rust using Google forms for setting it up, if that's what you meant.

If you are referring to the content itself, I'd personally like the following topics covered:

• Using Haskell (How long have you been using it, whether you use it professionally or not, what do you use it for, which platform and so on)
• Satisfaction with the language. (Pain points, lack of libraries, difficulties with introducing it at work, quality of learning materials available)
• Package management. (Things you'd like to see, difficulties with using the tools available)
• General suggestions on improving the ecosystem and increasing adoption.

In terms of when it should go out, it'd probably require several months of getting a decent amount of data. Maybe releasing it in December, as an outline for what the Haskell community should focus on next year, would work?

This is all IMHO, of course.

[alexeyzab]

Sounds like a good idea!

Using the Rust survey as a base and mixing in some questions from the older Haskell surveys would be nice.

In terms of what it should look like, I recall Rust using Google forms for setting it up, if that's what you meant.

If you are referring to the content itself, I'd personally like the following topics covered:

• Using Haskell (How long have you been using it, whether you use it professionally or not, what do you use it for, which platform and so on)
• Satisfaction with the language. (Pain points, lack of libraries, difficulties with introducing it at work, quality of learning materials available)
• Package management. (Things you'd like to see, difficulties with using the tools available)
• General suggestions on improving the ecosystem and increasing adoption.

In terms of when it should go out, it'd probably require several months of getting a decent amount of data. Maybe releasing it in December, as an outline for what the Haskell community should focus on next year, would work?

This is all IMHO, of course.

[tfausak]

Google forms would probably work fine, as would something like Survey Monkey. Alternatively something like Formspree could collect responses from a static HTML form.

That broad outline of content looks good to me. If this is going to be a yearly thing, there's not too much pressure to get it exactly right on the first try. We can solicit feedback about what should be in future surveys.

I don't care too much when the survey is released. Whenever it's ready seems fine. I don't think much stuff in the Haskell world aligns with the calendar year (other than HSoC).

[tfausak]

Google forms would probably work fine, as would something like Survey Monkey. Alternatively something like Formspree could collect responses from a static HTML form.

That broad outline of content looks good to me. If this is going to be a yearly thing, there's not too much pressure to get it exactly right on the first try. We can solicit feedback about what should be in future surveys.

I don't care too much when the survey is released. Whenever it's ready seems fine. I don't think much stuff in the Haskell world aligns with the calendar year (other than HSoC).

[tfausak]

I started working on the survey. You can check it out here: https://docs.google.com/forms/d/e/1FAIpQLSeh_VaXS9Za7hvmhjR7d1NdMv-7e8aVT7TkhhCN1tYkSpijDg/viewform

⚠️ The Google form is out of date. Use this instead: https://haskellweekly.news/surveys/2017.html ⚠️

Unfortunately it's not very easy to collaborate with non-Googlers on Google forms. It's also not easy to dump a list of questions from it. I think I'll write the form manually in HTML and open a pull request for it.

[tfausak]

I started working on the survey. You can check it out here: https://docs.google.com/forms/d/e/1FAIpQLSeh_VaXS9Za7hvmhjR7d1NdMv-7e8aVT7TkhhCN1tYkSpijDg/viewform

⚠️ The Google form is out of date. Use this instead: https://haskellweekly.news/surveys/2017.html ⚠️

Unfortunately it's not very easy to collaborate with non-Googlers on Google forms. It's also not easy to dump a list of questions from it. I think I'll write the form manually in HTML and open a pull request for it.

[alexeyzab]

@tfausak Looks good! Should the "Where do you get packages from?" be a multiple-choice question maybe?

[alexeyzab]

@tfausak Looks good! Should the "Where do you get packages from?" be a multiple-choice question maybe?

[tfausak]

👍 Yeah it probably should. There should also be an "Other" field.

[tfausak]

👍 Yeah it probably should. There should also be an "Other" field.

[tfausak]

Not launched. Still working on this.

[tfausak]

Not launched. Still working on this.

[tfausak]

Some stuff could be aped from the Stack Overflow Developer Survey. In particular, their most popular editors (Visual Studio, Notepad++, and Sublime) aren't options on this survey.

[tfausak]

Some stuff could be aped from the Stack Overflow Developer Survey. In particular, their most popular editors (Visual Studio, Notepad++, and Sublime) aren't options on this survey.

[tfausak]

I should both add a section at the top describing the form overall and add some notes on individual questions/answers that require you to skip around. For example, if you've never used Haskell, you should skip the majority of the survey.

[tfausak]

I should both add a section at the top describing the form overall and add some notes on individual questions/answers that require you to skip around. For example, if you've never used Haskell, you should skip the majority of the survey.

[tfausak]

From #121 (comment):

• Move email address input to a new line.
• Add community-oriented questions.
• Add proficiency questions.

[tfausak]

From #121 (comment):

• Move email address input to a new line.
• Add community-oriented questions.
• Add proficiency questions.

[tfausak]

• The editor question should be changed from radio buttons to check boxes.
• Should we include deliberately inflammatory questions like: "Should ((,) a) have a Foldable instance?"?

[tfausak]

• The editor question should be changed from radio buttons to check boxes.
• Should we include deliberately inflammatory questions like: "Should ((,) a) have a Foldable instance?"?

[tfausak]

I think we should also include a source control question. I can already get a feel for things by looking at Hackage, but it'd be nice to hear it straight from the horse's mouth.

[tfausak]

I think we should also include a source control question. I can already get a feel for things by looking at Hackage, but it'd be nice to hear it straight from the horse's mouth.

[alexeyzab]

Should we include deliberately inflammatory questions like: "Should ((,) a) have a Foldable instance?"?

I'd say sure, that's interesting data to show later.

[alexeyzab]

Should we include deliberately inflammatory questions like: "Should ((,) a) have a Foldable instance?"?

I'd say sure, that's interesting data to show later.

[tfausak]

I'm more interested in actionable data than interesting data. Even if >90% of the responses said that tuples should not have foldable instances, I wouldn't expect the CLC to actually remove the instance. That ship has sailed.

What's more, even if data isn't actionable, it should at least be interesting to compare year-to-year. Obviously thoughts on foldable tuples could change over time, but it doesn't seem too likely.

That being said, there's definitely room for more controversial questions. Most of the questions that use check boxes for answers could also have a follow-up question that uses radio buttons for exclusive choice. For example, "Which build tools do you use?" could be followed up by "What is your favorite build tool?".

[tfausak]

I'm more interested in actionable data than interesting data. Even if >90% of the responses said that tuples should not have foldable instances, I wouldn't expect the CLC to actually remove the instance. That ship has sailed.

What's more, even if data isn't actionable, it should at least be interesting to compare year-to-year. Obviously thoughts on foldable tuples could change over time, but it doesn't seem too likely.

That being said, there's definitely room for more controversial questions. Most of the questions that use check boxes for answers could also have a follow-up question that uses radio buttons for exclusive choice. For example, "Which build tools do you use?" could be followed up by "What is your favorite build tool?".

[alexeyzab]

I'm more interested in actionable data than interesting data.

Should we include some ghc-proposals then? I don't know which ones would be the most controversial, but at least it's more actionable.

That being said, there's definitely room for more controversial questions. Most of the questions that use check boxes for answers could also have a follow-up question that uses radio buttons for exclusive choice. For example, "Which build tools do you use?" could be followed up by "What is your favorite build tool?".

Sounds reasonable to me.

[alexeyzab]

I'm more interested in actionable data than interesting data.

Should we include some ghc-proposals then? I don't know which ones would be the most controversial, but at least it's more actionable.

That being said, there's definitely room for more controversial questions. Most of the questions that use check boxes for answers could also have a follow-up question that uses radio buttons for exclusive choice. For example, "Which build tools do you use?" could be followed up by "What is your favorite build tool?".

Sounds reasonable to me.

[tfausak]

GHC proposals are a good idea. I don't personally hold out much hope for them (or Haskell Prime). After all, we've got ghc-proposals, ghc/rfcs, and Haskell Prime. As far as I can tell, they're all stalled (haskell/rfcs#15).

Maybe a couple questions about language extensions would be good. Something like: "Which language extensions do you regularly use?" and "Which language extensions would you like to be enabled by default?".

[tfausak]

GHC proposals are a good idea. I don't personally hold out much hope for them (or Haskell Prime). After all, we've got ghc-proposals, ghc/rfcs, and Haskell Prime. As far as I can tell, they're all stalled (haskell/rfcs#15).

Maybe a couple questions about language extensions would be good. Something like: "Which language extensions do you regularly use?" and "Which language extensions would you like to be enabled by default?".

[alexeyzab]

Welp, that's a shame. Not sure what else can be measured by the survey and then put to good use.

Yeah, that sounds like a great addition! How about asking which prelude people use?

[alexeyzab]

Welp, that's a shame. Not sure what else can be measured by the survey and then put to good use.

Yeah, that sounds like a great addition! How about asking which prelude people use?

[eborden]

Is it possible to view the questions in the google form without having to complete it?

[eborden]

Is it possible to view the questions in the google form without having to complete it?

[tfausak]

The Google form is out of date. The current version is available here: https://haskellweekly.news/surveys/2017.html

But to answer your question: no. You have to pick an answer, but you don't have to submit the form. You can go back and change your answer to see the other "paths".

[tfausak]

The Google form is out of date. The current version is available here: https://haskellweekly.news/surveys/2017.html

But to answer your question: no. You have to pick an answer, but you don't have to submit the form. You can go back and change your answer to see the other "paths".

[eborden]

On the subject of inflammatory questions, I'd let old dogs lie. The Foldable/Traversable is just scratching an old wound. If you want something inflammatory I'd look towards the future for content, not the past.

[eborden]

On the subject of inflammatory questions, I'd let old dogs lie. The Foldable/Traversable is just scratching an old wound. If you want something inflammatory I'd look towards the future for content, not the past.

[eborden]

Hmmm, could you delete the google form to prevent confusion?

[eborden]

Hmmm, could you delete the google form to prevent confusion?

[eborden]

I'd also avoid having a lot of text boxes. It is a helpful option to have, but it means someone will have to read all of that data and collate trends. The more text boxes you have the less likely it is that someone is going to do that work.

[eborden]

I'd also avoid having a lot of text boxes. It is a helpful option to have, but it means someone will have to read all of that data and collate trends. The more text boxes you have the less likely it is that someone is going to do that work.

[tfausak]

I agree! I'm trying to avoid text boxes as much as possible. Some of them are unavoidable (like the "Other" option for stuff) and others are intentionally collecting free-form responses.

I'm willing to personally spend the time reading through all the long-form answers. Based on the responses, I will make changes to next year's form. That will mean either removing a bunch of free-form answers or adding more options to the multiple choice answers or adding more questions.

[tfausak]

I agree! I'm trying to avoid text boxes as much as possible. Some of them are unavoidable (like the "Other" option for stuff) and others are intentionally collecting free-form responses.

I'm willing to personally spend the time reading through all the long-form answers. Based on the responses, I will make changes to next year's form. That will mean either removing a bunch of free-form answers or adding more options to the multiple choice answers or adding more questions.

[eborden]

How do you primarily use Haskell?

Answers on this question could be less wordy. It could also be a multi select question.

Which Haskell compilers do you use?

Is this even worth asking? GHC has won, I don't think anyone contests that.

If you use GHC, has upgrading it broken your code in the last year?

This question is vague. Are we talking about type errors, type checker bugs, runtime bugs? Being more specific could be more actionable

Which build tools do you use?

There are a few other build tools. I think one is call mafia?

I am satisfied with Haskell's build tools (such as cabal-install or Stack).

This is likely and opportunity to be controversial. You could break this in to two questions targeting stack or cabal.

[eborden]

How do you primarily use Haskell?

Answers on this question could be less wordy. It could also be a multi select question.

Which Haskell compilers do you use?

Is this even worth asking? GHC has won, I don't think anyone contests that.

If you use GHC, has upgrading it broken your code in the last year?

This question is vague. Are we talking about type errors, type checker bugs, runtime bugs? Being more specific could be more actionable

Which build tools do you use?

There are a few other build tools. I think one is call mafia?

I am satisfied with Haskell's build tools (such as cabal-install or Stack).

This is likely and opportunity to be controversial. You could break this in to two questions targeting stack or cabal.

[tfausak]

Thanks for the feedback!

• I'll change the Haskell usage question to be less wordy and also be a multi-select.
• I expect GHC to easily count for >95% of usage, but I still want to ask. I am of the opinion that Haskell is GHC, for better or worse, and I'd like to have some data to support my worldview :)
• I'll add a free-form follow-up question to GHC breakage for explaining how things broke. It sucks to have another text box to read, but it's good stuff to know.
• I mention Mafia in the place holder for the "other" box. If a lot of people fill it in this year, I'll upgrade it to a "proper" choice. (And maybe also downgrade ghc-pkg.)
  • That reminds me that Nix will probably be a popular choice. I'll add that.
• I'll make the build tools question a little more targeted and controversial.

[tfausak]

Thanks for the feedback!

• I'll change the Haskell usage question to be less wordy and also be a multi-select.
• I expect GHC to easily count for >95% of usage, but I still want to ask. I am of the opinion that Haskell is GHC, for better or worse, and I'd like to have some data to support my worldview :)
• I'll add a free-form follow-up question to GHC breakage for explaining how things broke. It sucks to have another text box to read, but it's good stuff to know.
• I mention Mafia in the place holder for the "other" box. If a lot of people fill it in this year, I'll upgrade it to a "proper" choice. (And maybe also downgrade ghc-pkg.)
  • That reminds me that Nix will probably be a popular choice. I'll add that.
• I'll make the build tools question a little more targeted and controversial.

[tfausak]

I tweaked the survey a little more in 92f5c03. I feel like it's getting close to being ready.

Notes to self for next time:

• Writing a form this size by hand in HTML is tedious.
• Numbering the questions was a bad idea. It makes adding, removing, or rearranging questions too hard.

[tfausak]

I tweaked the survey a little more in 92f5c03. I feel like it's getting close to being ready.

Notes to self for next time:

• Writing a form this size by hand in HTML is tedious.
• Numbering the questions was a bad idea. It makes adding, removing, or rearranging questions too hard.

[tfausak]

I think I will publish the survey tomorrow and let it run through the month of November. So if you've got some opinions about the survey, now's the time to share them 😄

[tfausak]

I think I will publish the survey tomorrow and let it run through the month of November. So if you've got some opinions about the survey, now's the time to share them 😄

[eborden]

Will take a last pass today.

[eborden]

Will take a last pass today.

[alexeyzab]

Looks good to me.

[alexeyzab]

Looks good to me.

[eborden]

Took a look. It is looking solid.

Could this question have its answers in a scroll box? It is rather long.

Which language extensions would you like to be enabled by default?

[eborden]

Took a look. It is looking solid.

Could this question have its answers in a scroll box? It is rather long.

Which language extensions would you like to be enabled by default?

[tfausak]

The language extensions question does take up too much space. I thought about making it a <select multiple>, but then I wouldn't be able to link to the documentation. That might be fine, since you shouldn't be voting for something that you'd need to consult the documentation for.

On the other hand, I could just limit the height of the element and force it to scroll. Or I could put it in a <details> element, but I don't think most people are familiar with those.

[tfausak]

The language extensions question does take up too much space. I thought about making it a <select multiple>, but then I wouldn't be able to link to the documentation. That might be fine, since you shouldn't be voting for something that you'd need to consult the documentation for.

On the other hand, I could just limit the height of the element and force it to scroll. Or I could put it in a <details> element, but I don't think most people are familiar with those.

[eborden]

Yeah, I think you'd be fine with just

<div style="overflow-y: scrolling; height: 200px; box-shadow: inset 0 0 10px #000000;">

[eborden]

Yeah, I think you'd be fine with just

<div style="overflow-y: scrolling; height: 200px; box-shadow: inset 0 0 10px #000000;">

[tfausak]

Already getting some feedback on Reddit: https://www.reddit.com/r/haskell/comments/7a3fad/first_annual_haskell_users_survey/

[tfausak]

Already getting some feedback on Reddit: https://www.reddit.com/r/haskell/comments/7a3fad/first_annual_haskell_users_survey/

[tfausak]

Also some feedback on Lobsters: https://lobste.rs/s/f5vsl9/first_annual_haskell_users_survey

[tfausak]

Also some feedback on Lobsters: https://lobste.rs/s/f5vsl9/first_annual_haskell_users_survey

[tfausak]

I don't know why I didn't think of this before, but of course there's tons of discussion around Rust's two surveys.

https://www.reddit.com/r/rust/comments/4ikawg/launching_the_2016_state_of_rust_survey/
https://www.reddit.com/r/programming/comments/4imzad/launching_the_2016_state_of_rust_survey_xpost/
https://www.reddit.com/r/rust/comments/4qmr29/state_of_rust_survey_2016/
https://www.reddit.com/r/programming/comments/4qmr3t/state_of_rust_survey_2016/
https://www.reddit.com/r/rust/comments/698y4e/2017_state_of_rust_survey/
https://www.reddit.com/r/programming/comments/698yen/2017_state_of_rust_survey/
https://www.reddit.com/r/rust/comments/6y9csi/rust_2017_survey_results/
https://www.reddit.com/r/programming/comments/6y9cv1/rust_2017_survey_results/

In retrospect, "state of Haskell survey" is a much better name for this.

[tfausak]

I don't know why I didn't think of this before, but of course there's tons of discussion around Rust's two surveys.

https://www.reddit.com/r/rust/comments/4ikawg/launching_the_2016_state_of_rust_survey/
https://www.reddit.com/r/programming/comments/4imzad/launching_the_2016_state_of_rust_survey_xpost/
https://www.reddit.com/r/rust/comments/4qmr29/state_of_rust_survey_2016/
https://www.reddit.com/r/programming/comments/4qmr3t/state_of_rust_survey_2016/
https://www.reddit.com/r/rust/comments/698y4e/2017_state_of_rust_survey/
https://www.reddit.com/r/programming/comments/698yen/2017_state_of_rust_survey/
https://www.reddit.com/r/rust/comments/6y9csi/rust_2017_survey_results/
https://www.reddit.com/r/programming/comments/6y9cv1/rust_2017_survey_results/

In retrospect, "state of Haskell survey" is a much better name for this.

[eborden]

There was some discussion around the self rating question at front row:

user1 [11:59 PM] did anyone dare put Expert in the Haskell survey? (if you can't tell i'm catching up on Haskell Weekly at the moment)
user2 [12:00 PM] I put Advanced and I think that was a bold statement :grinning:
user1 [12:00 PM] haha same
user3 [12:27 PM] lol, I put expert
user3 [12:27] I think we live in this bubble of smart peeps
user3 [12:28] It was a bold statement on my part too :stuck_out_tongue:
user4 [12:43 PM] How do I express “I’m not kmett but I’m also not hello world”
user1 [12:43 PM] Advanced, was my guess for that range
user5 [12:44 PM] Words like Advanced and Expert are too overloaded imo. For some, Expert means having a deep understanding of Haskell internals, but for others it could mean just being great at _using_ the language
user3 [12:50 PM] "Proficient"
user2 [1:07 PM] I'm just going to copy paste this stream into the survey's github issue :joy:

[eborden]

There was some discussion around the self rating question at front row:

user1 [11:59 PM] did anyone dare put Expert in the Haskell survey? (if you can't tell i'm catching up on Haskell Weekly at the moment)
user2 [12:00 PM] I put Advanced and I think that was a bold statement :grinning:
user1 [12:00 PM] haha same
user3 [12:27 PM] lol, I put expert
user3 [12:27] I think we live in this bubble of smart peeps
user3 [12:28] It was a bold statement on my part too :stuck_out_tongue:
user4 [12:43 PM] How do I express “I’m not kmett but I’m also not hello world”
user1 [12:43 PM] Advanced, was my guess for that range
user5 [12:44 PM] Words like Advanced and Expert are too overloaded imo. For some, Expert means having a deep understanding of Haskell internals, but for others it could mean just being great at _using_ the language
user3 [12:50 PM] "Proficient"
user2 [1:07 PM] I'm just going to copy paste this stream into the survey's github issue :joy:

[tfausak]

Thanks! (Shouldn't be too hard to figure out who user2 is 😉) I haven't actually tallied anything yet, but I've seen a fair amount of self-reported "experts" come in. Maybe I should've added a level above "expert" called "kmett".

[tfausak]

Thanks! (Shouldn't be too hard to figure out who user2 is 😉) I haven't actually tallied anything yet, but I've seen a fair amount of self-reported "experts" come in. Maybe I should've added a level above "expert" called "kmett".

[eborden]

"kmett" would be a nice pun.

[eborden]

"kmett" would be a nice pun.