2018 state of Haskell survey

[tfausak]

📢 You can preview the survey here: https://airtable.com/shr8G4RBPD9T6tnDf.
👂 Let me know if there are any questions that you would like to be added, removed, or changed by replying to this GitHub issue. Thanks!

---

I want to get the ball rolling on this year's community survey.

• I published last year's survey on November 1 and I think I'll do that again this year.
• I was not a fan of the static HTML form sending results to Formspree. This year I think I'll use a service like Airtable, Google Forms, or SurveyMonkey.
• In general I was pretty happy with the questions, but there's definitely room for improvement. In particular I would like to collect demographic data.
• This is a relatively minor point, but I want to avoid selection bias. I didn't consider this last year, but people have complained about it (for example). I don't want this to be a "pro Stack" or "pro Cabal" or "pro Nix" survey.
• I need to be clearer about how the data will be shared. It will be nice to be able to link to last year's results as a concrete example.
• I also need to make it clearer that most (if not all) questions are optional.

👀 #102 #121 #125

[tfausak]

Now that November 1 is less than a month away, I think it's time to actually start putting this together. Broadly speaking I think the survey's questions will be similar to last year's. In addition to the links above, here are some valuable resources:

• http://blog.johantibell.com/2010/08/results-from-state-of-haskell-2010.html
• http://blog.johantibell.com/2011/08/results-from-state-of-haskell-2011.html
• https://haskellweekly.news/surveys/2017.html
  • https://lobste.rs/s/f5vsl9/first_annual_haskell_users_survey
  • https://www.reddit.com/r/haskell/comments/7a3fad/first_annual_haskell_users_survey/
• https://www.fpcomplete.com/blog/2018-haskell-survey-results

I have been using Airtable for other projects and really enjoy it. I will probably set up this survey to be run through Airtable rather than the static HTML and Formspree setup I used last year.

[tfausak]

I started translating the old survey into Airtable. You can see it here: https://airtable.com/shrigdVbhGA5NH60E https://airtable.com/shr8G4RBPD9T6tnDf

[lettier]

Hello @tfausak,

I may have a useful suggestion.

What are your thoughts on adding a multiple choice question regarding what Haskell related content topics the respondent would enjoy reading? This could help discover a gap in coverage, if one exists, and/or help Haskell content creators align their efforts with the needs of the greater Haskell audience. You could even use the results to help decide what to feature in future Haskell Weekly issues. :)

👍

[tfausak]

That's a great idea! Do you have something in mind for the question itself and the answer choices?

[lettier]

For the choices, we could take a data driven approach. Using topic analysis on the linked to articles, I can try to uncover the hidden themes found in all of the back issues of Haskell Weekly.

Another approach would be to infer some choices from Answer the Public for the query "Haskell".

In it I see:

• Use Cases / Case Studies
• Project Setup
• Project Maintenance
• Beginner Fundamentals (Hello Worlds)
• Tooling Choices with Pros and Cons
• Design Patterns
• Application Architectures
• Algorithm Implementations
• Machine Learning
• GUI Options
• Performance Analysis
• Back/Front End Web Development
• Game Development
• Mobile Development
• Debugging How-tos
• Library Walkthroughs
• The History of Haskell
• The Development of Haskell
• Comparisons to other Languages
• Deploying to Production
• Best Practices
• Industry Economics
• State Management
• Data Management
• Source-to-source Compilation or Transcompilation

In addition to the specific options, it might be best to have a write-in option as well.

For the question, here are some possible choices:

• Which of the following Haskell topics do you read about the most?
• Which of the following Haskell topics do you search for the most?
• Which of the following Haskell topics do you find the most enjoyable?
• Which of the following Haskell topics do you find the most valuable?
• Which of the following Haskell topics do you want to see more written about?

Or, more interestingly, you could flip those:

• Which of the following Haskell topics do you read about the least?
• Which of the following Haskell topics do you search for the least?
• Which of the following Haskell topics do you find the least enjoyable?
• Which of the following Haskell topics do you find the least valuable?
• Which of the following Haskell topics do you want to see less written about?

It may be helpful to include both a positive and negative question using the same categorical options. They may be truly disjoint or have some significant overlap indicating controversial or "mixed bag" topics.

👍

[tfausak]

Thanks! That's a lot to chew on. Of the questions you listed, the ones I'm personally most interested in seeing the response to are "Which of the following Haskell topics do you want to see more/less written about?" Those would be very useful to me as an author/curator.

[tfausak]

Note to self: Areas not covered by last year's survey that I want to cover this year:

• Comparison to other languages. Which other languages do people use? Which languages would people use if Haskell went away?
• Demographics. Who uses Haskell? Where do they live? Which languages do they speak?
• Which industries is Haskell used in? I basically only use it for web programming, for example.
• Testing. Do people write tests? Which tools do they use?
• Bugs. Is the saying "if it compiles, it works" even close to true? Is fixing bugs easy?
• Infrastructure. How do people deploy Haskell applications? Docker, Nix, static binaries, something else...
• Profiling. Do people profile things? Which tools do they use?
• Benchmarks. Do people run benchmarks? Do they use Criterion or something else?
• Hiring. If people are in a position to hire, is finding qualified candidates hard? If people are looking for a job, are they easy to find?
• Libraries versus applications. How many people write applications in Haskell?

Other useful links:

• https://blog.golang.org/survey2017-results
• https://blog.cognitect.com/blog/2017/1/31/clojure-2018-results
• https://blog.rust-lang.org/2017/09/05/Rust-2017-Survey-Results.html

[kitoran]

Hello @tfausak. I'm sorry if this isn't the right place to express my sugestion, but here we go.
In the results of 2017 state of Haskell survey results there are questions concernig satisfaction with the language and tools:
I am satisfied with Haskell as a language.
I am satisfied with Haskell’s build tools (such as cabal-install or Stack).
I am satisfied with Haskell’s package repositories (such as Hackage or Stackage).
I think it's useful to add similar question about compiler - I am satisfied with compiler that i use.

[tfausak]

This is definitely the right place, and thanks for the suggestion! I would be happy to add a question about compiler satisfaction.

[tfausak]

I'm not very happy with how Airtable is working out as a form solution. A few things in particular are annoying me:

1. It's not possible to have a single question with multiple answer choices along with an "other" choice. Since this is something that I do a lot, it's really frustrating.

2. It's also not possible to show/hide a question based on the answer to a previous question. Many of the questions are follow-ups for certain answers, and it would be nice to hide them.

3. It's not possible to break things up into sections, which I think would help make the survey feel less imposing.

I think Airtable could be a good resource for analyzing the responses, but I'm going to investigate other form services.

[tfausak]

Another thing I would like to ask about is Haskell Prime, also known as Haskell2020, or just the next version of the Haskell standard. Do people want it? What do they want included in it or excluded from it?

[tfausak]

Discussion is also happening on the haskell-community mailing list: https://mail.haskell.org/pipermail/haskell-community/2018-October/000323.html

[tfausak]

After briefly looking at SurveyMonkey and Google Forms, I think I'll stick with Airtable. It's pretty bare bones, but it'll get the job done.

I plan on working on the survey this weekend and publishing it on Thursday. If you have any feedback about this year's survey, now's the time!

[ChrisPenner]

I haven't fully thought through this yet, but I thought the way that the state of JS survey displayed some of their results was really cool! They allow you to see connections between tools and find pockets of users; e.g. Maybe most of the users of nix also use cabal instead of stack; etc. Not sure how tough it would be to do something like this, but it's definitely worth a look!

https://2017.stateofjs.com/2017/connections/