Organize a survey of our community

[erickt]

What do we want to know about our users? To get us started:

• What version of Rust are you using?
• What testing tools are you using?
• What market are you targeting with your projects? Gaming, databases, data science, etc?
• What's missing from the Rust ecosystem? GUI support, security features (LDAP libraries, kerberos, etc).

@steveklabnik mentioned that we should make our questions append-only, so if we ask people if they're using testing tool X or Y in one year, the next we ask X, Y, or Z. This would let us see how trends change over time.

[aturon]

cc @brson @wycats @nikomatsakis

[aturon]

cc @brson @wycats @nikomatsakis

[aturon]

cc @jonathandturner

[aturon]

cc @jonathandturner

[erickt]

Here's some other project surveys I've found:

• Go Language Survey
• Ember
• Jenkins
• Ethereum
• Eclipse

@steveklabnik: I couldn't find the ruby one you mentioned, could you post a link to it?

[erickt]

Here's some other project surveys I've found:

• Go Language Survey
• Ember
• Jenkins
• Ethereum
• Eclipse

@steveklabnik: I couldn't find the ruby one you mentioned, could you post a link to it?

[Manishearth]

• Tooling in general is a good question too, not just testing. E.g. rustfmt/clippy.
• What IDE/editor do you use?
• Do you contribute to any rust-lang repos?
• Do you contribute to or own any crates on crates.io?
• How excited are you for the following new features? (this can't be append-only, sadly).
• What languages did you know before learning Rust?
• What languages have you learned after learning Rust? Did knowing Rust help?

[Manishearth]

• Tooling in general is a good question too, not just testing. E.g. rustfmt/clippy.
• What IDE/editor do you use?
• Do you contribute to any rust-lang repos?
• Do you contribute to or own any crates on crates.io?
• How excited are you for the following new features? (this can't be append-only, sadly).
• What languages did you know before learning Rust?
• What languages have you learned after learning Rust? Did knowing Rust help?

[aturon]

@erickt Process suggestion -- I propose iterating on this for a bit with the community team, then presenting a draft to the core team for further refinement.

[aturon]

@erickt Process suggestion -- I propose iterating on this for a bit with the community team, then presenting a draft to the core team for further refinement.

[jonathandturner]

• Do you use Rust for hobby projects? At work?
• Do you use Rust for OSS? For non-open projects?
• How big is your Rust codebase? 1k? 10k? 100k? > 100k?
• Free form answer: what are the biggest challenges you see for Rust?

[jonathandturner]

• Do you use Rust for hobby projects? At work?
• Do you use Rust for OSS? For non-open projects?
• How big is your Rust codebase? 1k? 10k? 100k? > 100k?
• Free form answer: what are the biggest challenges you see for Rust?

[steveklabnik]

I couldn't seem to either, so I asked the person who's done it https://twitter.com/steveklabnik/status/717052982848798720

[steveklabnik]

I couldn't seem to either, so I asked the person who's done it https://twitter.com/steveklabnik/status/717052982848798720

[timonv]

While we're at it:

• Are there meetups near you / have you attended any Rust meetups?

Opinions are nice too, but maybe that should be up to the meetup organizers.

[timonv]

While we're at it:

• Are there meetups near you / have you attended any Rust meetups?

Opinions are nice too, but maybe that should be up to the meetup organizers.

[brson]

Demographics. Age, race, gender.

[brson]

Demographics. Age, race, gender.

[erickt]

• Did you stop using rust? if so, why?

[erickt]

• Did you stop using rust? if so, why?

[carols10cents]

I couldn't seem to either, so I asked the person who's done it https://twitter.com/steveklabnik/status/717052982848798720

Just checked the responses to steve's tweet and the Ruby survey is now up here: https://askr-prod.firebaseapp.com/ruby

Demographics. Age, race, gender.

Please make these optional, and please make gender a text box.

Demographics could also include your primary human language, which could help guide where we should focus translation efforts.

[carols10cents]

I couldn't seem to either, so I asked the person who's done it https://twitter.com/steveklabnik/status/717052982848798720

Just checked the responses to steve's tweet and the Ruby survey is now up here: https://askr-prod.firebaseapp.com/ruby

Demographics. Age, race, gender.

Please make these optional, and please make gender a text box.

Demographics could also include your primary human language, which could help guide where we should focus translation efforts.

[edunham]

• How did you find out about Rust? (freeform text field)
• If you're writing a Rust replacement for an existing tool, what language was that tool primarily implemented in?

[edunham]

• How did you find out about Rust? (freeform text field)
• If you're writing a Rust replacement for an existing tool, what language was that tool primarily implemented in?

[carols10cents]

Are there meetups near you / have you attended any Rust meetups?

Also we should ask what city/country they're in :)

[carols10cents]

Are there meetups near you / have you attended any Rust meetups?

Also we should ask what city/country they're in :)

[jonathandturner]

+1 @carols10cents re: gender

Maybe a more general question - are there decisions we would make differently knowing our demographics in detail? I think we'd still want to be the same welcoming community to the best of our ability.

Maybe it helps to flesh out the primary purpose of the survey. Are there particular choices we're trying to make and using the information will help us do that?

[jonathandturner]

+1 @carols10cents re: gender

Maybe a more general question - are there decisions we would make differently knowing our demographics in detail? I think we'd still want to be the same welcoming community to the best of our ability.

Maybe it helps to flesh out the primary purpose of the survey. Are there particular choices we're trying to make and using the information will help us do that?

[carols10cents]

Maybe a more general question - are there decisions we would make differently knowing our demographics in detail? I think we'd still want to be the same welcoming community to the best of our ability.

I totally agree that we should have a purpose in mind for asking each question. For the demographics, I'd love to have year-over-year information to know how well (or not) our outreach efforts are working.

[carols10cents]

Maybe a more general question - are there decisions we would make differently knowing our demographics in detail? I think we'd still want to be the same welcoming community to the best of our ability.

I totally agree that we should have a purpose in mind for asking each question. For the demographics, I'd love to have year-over-year information to know how well (or not) our outreach efforts are working.

[brson]

Maybe a more general question - are there decisions we would make differently knowing our demographics in detail? I think we'd still want to be the same welcoming community to the best of our ability.

Knowing demographics over time tells us whether our diversity efforts are working.

[brson]

Maybe a more general question - are there decisions we would make differently knowing our demographics in detail? I think we'd still want to be the same welcoming community to the best of our ability.

Knowing demographics over time tells us whether our diversity efforts are working.

[nikomatsakis]

It might be nice to get a sense of what people would most want to improve upon in Rust, to help guide overall priorities. But I'm not sure how just what questions to ask.

[nikomatsakis]

It might be nice to get a sense of what people would most want to improve upon in Rust, to help guide overall priorities. But I'm not sure how just what questions to ask.

[erickt]

Talked about it in the community team meeting

• Length was fine
• Email request should be made more prominent. First question could be "Your email (optional, but we want to get to know you)" or “if you are using Rust at work (or thinking about it) and would like to talk with a Rust community or core team member, put your email here:"
• If ..." questions are phrased as yes/no when a N/A might be sufficient. People will probably just choose "No" in the N/A case.

[erickt]

Talked about it in the community team meeting

• Length was fine
• Email request should be made more prominent. First question could be "Your email (optional, but we want to get to know you)" or “if you are using Rust at work (or thinking about it) and would like to talk with a Rust community or core team member, put your email here:"
• If ..." questions are phrased as yes/no when a N/A might be sufficient. People will probably just choose "No" in the N/A case.

[erickt]

• How many times has a Rust compiler upgrade broken your code since version 1.0?
• How many times has an upgrade broken you in an annoying way / required a large amount of work to rewrite your code to work?
• How many times has a change in a dependency broken your code?
• "Do you need production support?" but phrased better
• Remove the "have you read X" section
• Need input from @steveklabnik on if there are doc-specific questions he wants to ask

[erickt]

• How many times has a Rust compiler upgrade broken your code since version 1.0?
• How many times has an upgrade broken you in an annoying way / required a large amount of work to rewrite your code to work?
• How many times has a change in a dependency broken your code?
• "Do you need production support?" but phrased better
• Remove the "have you read X" section
• Need input from @steveklabnik on if there are doc-specific questions he wants to ask

[erickt]

• Are you worried about compilation time?
• Any cargo-related questions?

[erickt]

• Are you worried about compilation time?
• Any cargo-related questions?

[jdm]

"complication time"?

[jdm]

"complication time"?

[erickt]

@jdm: hehe, "compilation" time

[erickt]

@jdm: hehe, "compilation" time

[erickt]

• Are you happy with the quality of error messages?
• Do the error messages have enough information in them to help you find the problem?
• Are the error messages too noisy?

[erickt]

• Are you happy with the quality of error messages?
• Do the error messages have enough information in them to help you find the problem?
• Are the error messages too noisy?

[erickt]

• Do you expect to be using stable / nightly / etc.
• Do you contribute or maintain a crate on crates.io
• What other languages do you use together with Rust?
• Do you plan to try to use Rust at work?
  • If not, why not?
• Demographics need word smithing
• Replace "Other" boxes with "anything else you want to tell us"?
• Do you think we should prioritize
  • New features
  • Tooling
  • Compile times
• What new thing are you most excited about coming in the next year?
• What is the worst thing about Rust?
• What are the biggest problems about Rust
• When you try to evangelize rust, what are the worst and best things for adoption?
• What would you like to see cargo do?
• Do you like cargo?
• Do you use Cargo - yes production, yes in hobby, none of the above
• For production users, are you using Cargo? If not, why?

[erickt]

• Do you expect to be using stable / nightly / etc.
• Do you contribute or maintain a crate on crates.io
• What other languages do you use together with Rust?
• Do you plan to try to use Rust at work?
  • If not, why not?
• Demographics need word smithing
• Replace "Other" boxes with "anything else you want to tell us"?
• Do you think we should prioritize
  • New features
  • Tooling
  • Compile times
• What new thing are you most excited about coming in the next year?
• What is the worst thing about Rust?
• What are the biggest problems about Rust
• When you try to evangelize rust, what are the worst and best things for adoption?
• What would you like to see cargo do?
• Do you like cargo?
• Do you use Cargo - yes production, yes in hobby, none of the above
• For production users, are you using Cargo? If not, why?

[erickt]

• Important to be up front about what we do about the data.
• Strong preference that we only provide anonymized data and aggregate data.

[erickt]

• Important to be up front about what we do about the data.
• Strong preference that we only provide anonymized data and aggregate data.

[aturon]

Possibly relevant: State of Clojure 2015 survey results.

[aturon]

Possibly relevant: State of Clojure 2015 survey results.

[brson]

How do you install Rust today?

• rust-lang tarballs
• rust-lang Windows .msi installers
• rust-lang OS X .pkg installers
• multirust.sh
• rustup.rs
• rustup.sh
• Linux distro package
• Homebrew
• Other

How would you prefer to install Rust?

• rust-lang tarballs
• rust-lang Windows .msi installers
• rust-lang OS X .pkg installers
• multirust.sh
• rustup.rs
• rustup.sh
• Linux distro package
• Homebrew
• Other

Do you work in an environment where it would be undesirable to install Rust via rustup? Explain.

[brson]

How do you install Rust today?

• rust-lang tarballs
• rust-lang Windows .msi installers
• rust-lang OS X .pkg installers
• multirust.sh
• rustup.rs
• rustup.sh
• Linux distro package
• Homebrew
• Other

How would you prefer to install Rust?

• rust-lang tarballs
• rust-lang Windows .msi installers
• rust-lang OS X .pkg installers
• multirust.sh
• rustup.rs
• rustup.sh
• Linux distro package
• Homebrew
• Other

Do you work in an environment where it would be undesirable to install Rust via rustup? Explain.

[erickt]

Prototype survey: http://goo.gl/forms/LTNphA69Xn

[erickt]

Prototype survey: http://goo.gl/forms/LTNphA69Xn

[carols10cents]

More feedback happened in IRC https://github.com/rust-lang/meeting-minutes/blob/master/community-team/2016-04-13.txt#L150

[carols10cents]

More feedback happened in IRC https://github.com/rust-lang/meeting-minutes/blob/master/community-team/2016-04-13.txt#L150

[skade]

Feedback:

• Languages should include Haskell and Clojure (first is often a source of new contributors, clojure is currently a language for the curious)
• "What is your race" sits very bad in cultures where the term race is only used by racists (german language, for example) Also, most people don't know the proper terms to answer that question when outside of the US. (I don't know mine, I think it's "caucasian"
• "preferred gender identity" can someone reread that. Is that the current way to phrase it?
• Many community org questions assume you are not already involved, which makes people that are involved dodging them and us loosing the information about how many people identifying as "already involved" we have

On 27 Apr 2016, at 18:09, Erick Tryzelaar notifications@github.com wrote:

Prototype survey: http://goo.gl/forms/LTNphA69Xn

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

[skade]

Feedback:

• Languages should include Haskell and Clojure (first is often a source of new contributors, clojure is currently a language for the curious)
• "What is your race" sits very bad in cultures where the term race is only used by racists (german language, for example) Also, most people don't know the proper terms to answer that question when outside of the US. (I don't know mine, I think it's "caucasian"
• "preferred gender identity" can someone reread that. Is that the current way to phrase it?
• Many community org questions assume you are not already involved, which makes people that are involved dodging them and us loosing the information about how many people identifying as "already involved" we have

On 27 Apr 2016, at 18:09, Erick Tryzelaar notifications@github.com wrote:

Prototype survey: http://goo.gl/forms/LTNphA69Xn

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

[carols10cents]

I have done the following:

• Moved all email-related requests to the first section
• Added "I don't know" and/or "Not Applicable" to all radio button questions that seemed to need it
• Removed a duplicate question about translating the docs that was in both the demographics and community section; it's now only in the demographics section right after the language you speak
• Added questions about compiler upgrades breaking or being annoying, and dependency upgrades breaking
• Removed the documentation section since it only had the "have you read?" question that was slated for removal

Things I have not done because I have questions

• "Do you need production support?" but phrased better - I'm not sure how to phrase this better. What are we looking to get out of this question? Is this covered by the email request with the reason being that you use rust at work?
• Are you worried about compilation time? is this about compiling rust itself or compiling one's own projects? What are we looking for here? Scale of 1-5 from not at all worried to very worried?

[carols10cents]

I have done the following:

• Moved all email-related requests to the first section
• Added "I don't know" and/or "Not Applicable" to all radio button questions that seemed to need it
• Removed a duplicate question about translating the docs that was in both the demographics and community section; it's now only in the demographics section right after the language you speak
• Added questions about compiler upgrades breaking or being annoying, and dependency upgrades breaking
• Removed the documentation section since it only had the "have you read?" question that was slated for removal

Things I have not done because I have questions

• "Do you need production support?" but phrased better - I'm not sure how to phrase this better. What are we looking to get out of this question? Is this covered by the email request with the reason being that you use rust at work?
• Are you worried about compilation time? is this about compiling rust itself or compiling one's own projects? What are we looking for here? Scale of 1-5 from not at all worried to very worried?

[carols10cents]

Taking a break for a min but will continue to address comments starting from #15 (comment) after that :)

[carols10cents]

Taking a break for a min but will continue to address comments starting from #15 (comment) after that :)

[carols10cents]

More stuff I've done:

• Added Portland to the rust conference locations
• Added to the Versions section:
  • channel question
  • rust installation questions
• Added to the Rust Language Projects section the questions about cargo-- I made the usage question look like this to more accurately capture all the variations:
  
• Added to the Rust Projects section:
  • What programming languages, if any, do you use together with Rust?
  • Do you contribute to or maintain a crate on crates.io?
• Added to the Rust at work section:
  • Added "how many total developers are at your company" after how many developers use rust
  • If you don't currently use Rust at work, do you plan to try to use Rust at work?
  • If you are not currently or planning to use Rust at work, why not?
• Added to the Challenges and Feedback section:
  • the questions about error messages
  • Anything else you would like to tell us?
  • Do you think we should prioritize New features/Tooling/Compile times
  • What new thing are you most excited about coming in the next year?
  • What is the worst thing about Rust?
  • What are the biggest problems about Rust
  • When you try to evangelize rust, what are the worst and best things for adoption?
  • How do you feel about the borrow checker?
  • What learning resources have been most helpful?
  • What learning resources do you wish were available?

I have questions about

• I added some questions about operating systems and cross compiling to the Rust Projects section-- someone on the core team should see if I've captured everything that makes sense and would be interesting, or delete those questions if you don't care, but I think they would be interesting.
• I don't feel qualified to address @skade's concerns about the wording of the race/ethnicity/gender questions

I think I'm done

I believe I've addressed all the feedback aside from the issues I created or punted on as noted :)

[carols10cents]

More stuff I've done:

• Added Portland to the rust conference locations
• Added to the Versions section:
  • channel question
  • rust installation questions
• Added to the Rust Language Projects section the questions about cargo-- I made the usage question look like this to more accurately capture all the variations:
  
• Added to the Rust Projects section:
  • What programming languages, if any, do you use together with Rust?
  • Do you contribute to or maintain a crate on crates.io?
• Added to the Rust at work section:
  • Added "how many total developers are at your company" after how many developers use rust
  • If you don't currently use Rust at work, do you plan to try to use Rust at work?
  • If you are not currently or planning to use Rust at work, why not?
• Added to the Challenges and Feedback section:
  • the questions about error messages
  • Anything else you would like to tell us?
  • Do you think we should prioritize New features/Tooling/Compile times
  • What new thing are you most excited about coming in the next year?
  • What is the worst thing about Rust?
  • What are the biggest problems about Rust
  • When you try to evangelize rust, what are the worst and best things for adoption?
  • How do you feel about the borrow checker?
  • What learning resources have been most helpful?
  • What learning resources do you wish were available?

I have questions about

• I added some questions about operating systems and cross compiling to the Rust Projects section-- someone on the core team should see if I've captured everything that makes sense and would be interesting, or delete those questions if you don't care, but I think they would be interesting.
• I don't feel qualified to address @skade's concerns about the wording of the race/ethnicity/gender questions

I think I'm done

I believe I've addressed all the feedback aside from the issues I created or punted on as noted :)

[carols10cents]

Oh also, @erickt, I'm sure you've thought about this already, but before this gets widely publicized, you should revoke editing privileges from everyone who has the link to the form, since the edit link is public info now :)

[carols10cents]

Oh also, @erickt, I'm sure you've thought about this already, but before this gets widely publicized, you should revoke editing privileges from everyone who has the link to the form, since the edit link is public info now :)

[erickt]

@carols10cents: I just checked in a private browser, and it doesn't appear that the form is editable.

[erickt]

@carols10cents: I just checked in a private browser, and it doesn't appear that the form is editable.

[carols10cents]

Oh! ok cool, I thought you had done the "anyone with the link can edit" thing, I see now that you've shared it with specific people. Sorry about the alarm!

[carols10cents]

Oh! ok cool, I thought you had done the "anyone with the link can edit" thing, I see now that you've shared it with specific people. Sorry about the alarm!

[skade]

/cc @1000miles, who will have a look over the questions I pointed out.

[skade]

/cc @1000miles, who will have a look over the questions I pointed out.

[skade]

Also: we have diversity-related questions, but no questions about disability/accessibility issues.

[skade]

Also: we have diversity-related questions, but no questions about disability/accessibility issues.

[1000miles]

Hey there, thanks for inviting me to join this discussion, @skade.

I had a look on the prototype survey regarding the two demographic questions:

1. What is your race?
2. What is your preferred gender identity?

To point 1:
I do agree that it would be better to change this question. There has been a long on-going discussion about race as a social construct, pls find some infos here. As @skade already pointed out the german translation of race for example would be problematic because this term has its' roots in slavery & colonialism which was also used by the Nazi regime to legitimize their atrocities.

Asking people with immigration background & non-white people about their ethnicity sometimes can also be a sensitive topic because this can transfer the concept of othering. In the context of improving and evaluating the diversity outreach of the Rust community you might be able to ask: What is your (ethnic) background? I do believe that this alternative question will not work out for everyone though, but it might leave the question more open to reply sth. like Mexican-American-etc., Asian-German or African-American (as political terms)?

To point 2:
I think asking people within the binary system (cis-male/-female) for their gender is mostly okay but for people beyond the binary system (neither nor, both, etc.) it can be too intimate which they might don't want to share. What if you ask for What is or are your pronouns? (he/she/they) - wouldn't it be enough and you get the similar data you basically need?

Does this all make sense to you? Happy to discuss! :)

[1000miles]

Hey there, thanks for inviting me to join this discussion, @skade.

I had a look on the prototype survey regarding the two demographic questions:

1. What is your race?
2. What is your preferred gender identity?

To point 1:
I do agree that it would be better to change this question. There has been a long on-going discussion about race as a social construct, pls find some infos here. As @skade already pointed out the german translation of race for example would be problematic because this term has its' roots in slavery & colonialism which was also used by the Nazi regime to legitimize their atrocities.

Asking people with immigration background & non-white people about their ethnicity sometimes can also be a sensitive topic because this can transfer the concept of othering. In the context of improving and evaluating the diversity outreach of the Rust community you might be able to ask: What is your (ethnic) background? I do believe that this alternative question will not work out for everyone though, but it might leave the question more open to reply sth. like Mexican-American-etc., Asian-German or African-American (as political terms)?

To point 2:
I think asking people within the binary system (cis-male/-female) for their gender is mostly okay but for people beyond the binary system (neither nor, both, etc.) it can be too intimate which they might don't want to share. What if you ask for What is or are your pronouns? (he/she/they) - wouldn't it be enough and you get the similar data you basically need?

Does this all make sense to you? Happy to discuss! :)

[erickt]

Thanks @skade and @1000miles! @jonathandturner rephrased it to be Do you self-identify as an underrepresented demographic in technology? (Optional). How's that sound? And @1000miles, I wanted to extend a very last minute invite to our community meeting in 10 minutes on https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23rust-community. We'll be talking about the survey (among other things) if you want to join in.

[erickt]

Thanks @skade and @1000miles! @jonathandturner rephrased it to be Do you self-identify as an underrepresented demographic in technology? (Optional). How's that sound? And @1000miles, I wanted to extend a very last minute invite to our community meeting in 10 minutes on https://client00.chat.mibbit.com/?server=irc.mozilla.org&channel=%23rust-community. We'll be talking about the survey (among other things) if you want to join in.

[skade]

A list of examples of what could constitute that might be worthwhile.

[skade]

A list of examples of what could constitute that might be worthwhile.

[erickt]

I added "state" to the location question, but it seems to be a little US-centric. Is there a better phrasing to "What city, state, country do you live in? (Optional)" that works internationally?

[erickt]

I added "state" to the location question, but it seems to be a little US-centric. Is there a better phrasing to "What city, state, country do you live in? (Optional)" that works internationally?

[skade]

Closing, as successful. \o/

[skade]

Closing, as successful. \o/