New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2017 Community survey planning #28

Closed
skade opened this Issue May 11, 2016 · 50 comments

Comments

Projects
None yet
@skade
Contributor

skade commented May 11, 2016

This is a tracking issue for everything that we failed at during the 2016 essay and should make better in 2017. Automatically self-destructs on May 15th, 2017.

@aturon

This comment has been minimized.

Show comment
Hide comment
@vladimir-lu

This comment has been minimized.

Show comment
Hide comment
@vladimir-lu

vladimir-lu May 18, 2016

I had a comment from someone that the survey came across as too professional-programmer focused and that it shouldn't assume you do programming for your day job.

vladimir-lu commented May 18, 2016

I had a comment from someone that the survey came across as too professional-programmer focused and that it shouldn't assume you do programming for your day job.

@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents May 18, 2016

Contributor
  • we forgot to ask if we could contact people with offers in the survey (e.g. send them emails about a conference location if they selected a conference place where we actually run a conference, or if multiple people selected the same location and "interested in a meetup" if we could get them in contact with each other)
Contributor

carols10cents commented May 18, 2016

  • we forgot to ask if we could contact people with offers in the survey (e.g. send them emails about a conference location if they selected a conference place where we actually run a conference, or if multiple people selected the same location and "interested in a meetup" if we could get them in contact with each other)
@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Jun 16, 2016

Contributor
  • How many miles would you be willing to travel for a conference?
Contributor

erickt commented Jun 16, 2016

  • How many miles would you be willing to travel for a conference?
@skade

This comment has been minimized.

Show comment
Hide comment
@skade

skade Jun 23, 2016

Contributor

Next survey should include a direct "want to sign up for our newsletter?".

Contributor

skade commented Jun 23, 2016

Next survey should include a direct "want to sign up for our newsletter?".

@erickt erickt changed the title from Community survey 2017 to 2017 Community survey planning Jun 30, 2016

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Jun 30, 2016

Contributor
  • I think the 'crash report' question was too easy to misunderstand.
  • We should break up "No" and "Stopped using Rust" into two separate answers.
  • We should go through the demographics and add any missing demographics.
  • We need to another pass around describing the demographics. Here's a relevant quote:

I wonder, are black people living and working in Africa under-represented? What should they answer in your survey? This question in survey looks somewhat US-centric. Even the term "person of color" is a US-term and can be hard for people in other countries to understand. I don't completely understand it, e.g. does it include Asians or not.

Contributor

erickt commented Jun 30, 2016

  • I think the 'crash report' question was too easy to misunderstand.
  • We should break up "No" and "Stopped using Rust" into two separate answers.
  • We should go through the demographics and add any missing demographics.
  • We need to another pass around describing the demographics. Here's a relevant quote:

I wonder, are black people living and working in Africa under-represented? What should they answer in your survey? This question in survey looks somewhat US-centric. Even the term "person of color" is a US-term and can be hard for people in other countries to understand. I don't completely understand it, e.g. does it include Asians or not.

@jonathandturner

This comment has been minimized.

Show comment
Hide comment
@jonathandturner

jonathandturner Jun 30, 2016

On a similar note from another comment:

"'Talking with my Korean friends here in Korea, I found that they didn't
check "person of color" checkbox because they had no idea what that is.
Typical response: "does it mean person with colorful personality?" Just for
your information.'"

On Thu, Jun 30, 2016 at 4:11 PM, Erick Tryzelaar notifications@github.com
wrote:

I wonder, are black people living and working in Africa under-represented?
What should they answer in your survey? This question in survey looks
somewhat US-centric. Even the term "person of color" is a US-term and can
be hard for people in other countries to understand. I don't completely
understand it, e.g. does it include Asians or not.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#28 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAhZVrudoNSs2l4anuZqYaL8JSSmWV3nks5qRCLbgaJpZM4IcPSV
.

Jonathan

jonathandturner commented Jun 30, 2016

On a similar note from another comment:

"'Talking with my Korean friends here in Korea, I found that they didn't
check "person of color" checkbox because they had no idea what that is.
Typical response: "does it mean person with colorful personality?" Just for
your information.'"

On Thu, Jun 30, 2016 at 4:11 PM, Erick Tryzelaar notifications@github.com
wrote:

I wonder, are black people living and working in Africa under-represented?
What should they answer in your survey? This question in survey looks
somewhat US-centric. Even the term "person of color" is a US-term and can
be hard for people in other countries to understand. I don't completely
understand it, e.g. does it include Asians or not.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#28 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAhZVrudoNSs2l4anuZqYaL8JSSmWV3nks5qRCLbgaJpZM4IcPSV
.

Jonathan

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Jun 30, 2016

Contributor

From HN:

Why not simply ask the question directly? "Do you feel welcome in the Rust community?"
For those that respond "no, I don't feel welcome", it might be interesting to also ask why, and to see if that correlates with demographics.

Contributor

erickt commented Jun 30, 2016

From HN:

Why not simply ask the question directly? "Do you feel welcome in the Rust community?"
For those that respond "no, I don't feel welcome", it might be interesting to also ask why, and to see if that correlates with demographics.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik commented Jun 30, 2016

Some lobste.rs feedback

@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents Jul 27, 2016

Contributor

Ask explicitly:

  • Would you like to be contacted about conferences whose location you have expressed interest in
  • Would you like to be put in touch with others in your area interested in meetups
Contributor

carols10cents commented Jul 27, 2016

Ask explicitly:

  • Would you like to be contacted about conferences whose location you have expressed interest in
  • Would you like to be put in touch with others in your area interested in meetups
@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Sep 1, 2016

Would be good to circulate the survey for feedback (particularly to the sub-teams) before sending it out - I would have loved to tweak some of the questions but didn't see the survey was out.

nrc commented Sep 1, 2016

Would be good to circulate the survey for feedback (particularly to the sub-teams) before sending it out - I would have loved to tweak some of the questions but didn't see the survey was out.

@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents
Contributor

carols10cents commented Sep 20, 2016

#79

@skade

This comment has been minimized.

Show comment
Hide comment
@skade

skade Sep 20, 2016

Contributor

I would prefer if we used another service next year, http://typeform.com was really great and produces better output AFAIK. Also, it handles notifying respondents, etc.

If we want to go into the habit of having more surveys, we could also take one of the paid plans.

Contributor

skade commented Sep 20, 2016

I would prefer if we used another service next year, http://typeform.com was really great and produces better output AFAIK. Also, it handles notifying respondents, etc.

If we want to go into the habit of having more surveys, we could also take one of the paid plans.

@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents Feb 17, 2017

Contributor

Assuming rust-lang/rfcs#1824 is accepted and implemented by then, I would like to add a question or questions about whether the information crates.io provides is useful for evaluating crates.

Contributor

carols10cents commented Feb 17, 2017

Assuming rust-lang/rfcs#1824 is accepted and implemented by then, I would like to add a question or questions about whether the information crates.io provides is useful for evaluating crates.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik commented Mar 6, 2017

Here's the results of the Go survey https://blog.golang.org/survey2016-results

@Dowwie

This comment has been minimized.

Show comment
Hide comment
@Dowwie

Dowwie Mar 6, 2017

Would you consider adding "What do you like most about Rust?" to the next survey, using numerical ranking so as to avoid having to choose only 1 value

Dowwie commented Mar 6, 2017

Would you consider adding "What do you like most about Rust?" to the next survey, using numerical ranking so as to avoid having to choose only 1 value

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 19, 2017

Contributor

We should consider putting out the survey in multiple languages, in case there are communities that don't speak english.

Contributor

erickt commented Apr 19, 2017

We should consider putting out the survey in multiple languages, in case there are communities that don't speak english.

@jonathandturner

This comment has been minimized.

Show comment
Hide comment
@jonathandturner

jonathandturner Apr 19, 2017

From the IRC conversation - we should have the data gathering script ready before we go live.

Also, we should separate out the free form, so they don't get mixed in with the drop-down results.

jonathandturner commented Apr 19, 2017

From the IRC conversation - we should have the data gathering script ready before we go live.

Also, we should separate out the free form, so they don't get mixed in with the drop-down results.

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor
  • "How would you prefer to install Rust?" is a little odd. We should change this to ask "Would you prefer to install Rust through a different mechanism?". Maybe make it a long form questions.
  • How about we rephrase "After 1.0, has upgrading to a new version of the Stable Rust compiler broken your code?" into "has upgrading to a new stable version of Rust in the last year broken your code?"
  • We should ask how people like rustup.rs
  • We should ask about rls
  • Are there any other editors we should add to the editor list?
  • Should we ask about OS versions? cc @brson / @alexcrichton
  • Is there more that we could ask to people who don't use rust at work. How about asking:
    • What could we do to make rust more acceptable at your company?
    • How many in your company would you say would be interested in using rust?
  • How can we make this less biased towards programmers? Are there non-programmer roles was could add to the job descriptions?
Contributor

erickt commented Apr 26, 2017

  • "How would you prefer to install Rust?" is a little odd. We should change this to ask "Would you prefer to install Rust through a different mechanism?". Maybe make it a long form questions.
  • How about we rephrase "After 1.0, has upgrading to a new version of the Stable Rust compiler broken your code?" into "has upgrading to a new stable version of Rust in the last year broken your code?"
  • We should ask how people like rustup.rs
  • We should ask about rls
  • Are there any other editors we should add to the editor list?
  • Should we ask about OS versions? cc @brson / @alexcrichton
  • Is there more that we could ask to people who don't use rust at work. How about asking:
    • What could we do to make rust more acceptable at your company?
    • How many in your company would you say would be interested in using rust?
  • How can we make this less biased towards programmers? Are there non-programmer roles was could add to the job descriptions?
@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor
  • on "What domain(s) do you work in currently?"
    • change title to "What industries(s) do you work in currently?"?
    • we should change "servers" to something else?
  • Add "what programming languages does your company use"?
  • We should add more programming languages
  • "How would you rate your Rust expertise?" may be redundant with the "How long have you been working with Rust?" question
  • For "Are you part of an underrepresented demographic in technology?" we should add a section about economic challenges, which came up in 2016.
  • We should ask permission if we can refer them to a local meetup. We could have a separate survey that collects emails for particular regions so we can decouple the answers to the survey to just being contacted for meetup/conference.
  • I saw other surveys ask a CAPTCHA. should we do that?
  • "To which of these cities would you travel to attend a Rust conference?" needs some work now that we got I think 5 large events happening this year.
Contributor

erickt commented Apr 26, 2017

  • on "What domain(s) do you work in currently?"
    • change title to "What industries(s) do you work in currently?"?
    • we should change "servers" to something else?
  • Add "what programming languages does your company use"?
  • We should add more programming languages
  • "How would you rate your Rust expertise?" may be redundant with the "How long have you been working with Rust?" question
  • For "Are you part of an underrepresented demographic in technology?" we should add a section about economic challenges, which came up in 2016.
  • We should ask permission if we can refer them to a local meetup. We could have a separate survey that collects emails for particular regions so we can decouple the answers to the survey to just being contacted for meetup/conference.
  • I saw other surveys ask a CAPTCHA. should we do that?
  • "To which of these cities would you travel to attend a Rust conference?" needs some work now that we got I think 5 large events happening this year.
@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents Apr 26, 2017

Contributor
  • "How long did you stop using Rust before you stopped?" doesn't make sense to me, I think instead this should say "How long did you [try using|use] Rust before you stopped?"
  • "Approximately how large are all the Rust projects you work on?" Are we looking for the sum total or the average of all the rust projects someone works on? It's not clear to me, we should either say "Approximately how many lines would the sum of all the Rust projects you work on be?" or "Approximately, what is the average number of lines of all the Rust projects you work on?" depending on what we're looking for
Contributor

carols10cents commented Apr 26, 2017

  • "How long did you stop using Rust before you stopped?" doesn't make sense to me, I think instead this should say "How long did you [try using|use] Rust before you stopped?"
  • "Approximately how large are all the Rust projects you work on?" Are we looking for the sum total or the average of all the rust projects someone works on? It's not clear to me, we should either say "Approximately how many lines would the sum of all the Rust projects you work on be?" or "Approximately, what is the average number of lines of all the Rust projects you work on?" depending on what we're looking for
@erickt

This comment has been minimized.

Show comment
Hide comment
@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

This comment has been minimized.

Show comment
Hide comment
@carols10cents

carols10cents Apr 26, 2017

Contributor

bstrie> erickt Manishearth: I'd rather be very explicit and say "has upgrading to a new version of the stable rust compiler (not upgrading library dependencies) broken your code in the past year?"
12:29 PM <~erickt> even better
12:29 PM <@Manishearth> bstrie: yes
12:29 PM and then move the library dependencies question right after that and make dependencies bold
12:30 PM <@Manishearth> erickt: we had issues with folks thinking a broken clippy was breaking changes too
12:30 PM <@Manishearth> yes
12:30 PM <@Manishearth> yall are great

Contributor

carols10cents commented Apr 26, 2017

bstrie> erickt Manishearth: I'd rather be very explicit and say "has upgrading to a new version of the stable rust compiler (not upgrading library dependencies) broken your code in the past year?"
12:29 PM <~erickt> even better
12:29 PM <@Manishearth> bstrie: yes
12:29 PM and then move the library dependencies question right after that and make dependencies bold
12:30 PM <@Manishearth> erickt: we had issues with folks thinking a broken clippy was breaking changes too
12:30 PM <@Manishearth> yes
12:30 PM <@Manishearth> yall are great

@erickt

This comment has been minimized.

Show comment
Hide comment
Contributor

erickt commented Apr 26, 2017

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor

Questions derived from the nim survey:

  • How did you find out about Rust?
  • Most appealing, most displeasing aspects of Rust
  • What learning resources did you use to learn Rust
  • If you read the book, how far did you get? did you enjoy it?

From elixir:

  • We should add questions on testing, underhandedness, etc
  • Employment status?
    • Employed full time by a company
    • freelancer
    • run my own business developing software for clients
    • build my own apps exclusively
    • unemployed
    • student

  • Should we split the language question into "what programming languages do you use at work?", "what programming languages do you use in your free time?"

From elixir:

  • How do you deploy your rust code? Kubernetes/AWS/etc
Contributor

erickt commented Apr 26, 2017

Questions derived from the nim survey:

  • How did you find out about Rust?
  • Most appealing, most displeasing aspects of Rust
  • What learning resources did you use to learn Rust
  • If you read the book, how far did you get? did you enjoy it?

From elixir:

  • We should add questions on testing, underhandedness, etc
  • Employment status?
    • Employed full time by a company
    • freelancer
    • run my own business developing software for clients
    • build my own apps exclusively
    • unemployed
    • student

  • Should we split the language question into "what programming languages do you use at work?", "what programming languages do you use in your free time?"

From elixir:

  • How do you deploy your rust code? Kubernetes/AWS/etc
@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor

From clojure:

  • What has most frustrating aspect to using rust?
    • Hiring / staffing
    • Docs
    • Long term viability
    • portability
    • finding libraries
  • Deployment options: AWS lambda, containers, ECS, App store

From pycharm:

  • What frameworks are you using? Diesel, Rocket, Iron, etc.
  • Do you use debugging tools, profiling, code coverage, etc? Often/time to time/never.
Contributor

erickt commented Apr 26, 2017

From clojure:

  • What has most frustrating aspect to using rust?
    • Hiring / staffing
    • Docs
    • Long term viability
    • portability
    • finding libraries
  • Deployment options: AWS lambda, containers, ECS, App store

From pycharm:

  • What frameworks are you using? Diesel, Rocket, Iron, etc.
  • Do you use debugging tools, profiling, code coverage, etc? Often/time to time/never.
@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor

From ember:

  • Is your company planning on hiring Rust developers? If so, what's the timescale?
  • What would be required in order for your company to hire Rust developers?
  • Did you change jobs to join a company to work with Rust?
  • If your company is using Rust, do people feel like they are working faster or slower? Are they happy or not happy?
  • How old are your Rust applications?
  • Which region do you live? North America, Europe, Asia, Oceania, South America, Africa, Central America
Contributor

erickt commented Apr 26, 2017

From ember:

  • Is your company planning on hiring Rust developers? If so, what's the timescale?
  • What would be required in order for your company to hire Rust developers?
  • Did you change jobs to join a company to work with Rust?
  • If your company is using Rust, do people feel like they are working faster or slower? Are they happy or not happy?
  • How old are your Rust applications?
  • Which region do you live? North America, Europe, Asia, Oceania, South America, Africa, Central America
@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Apr 26, 2017

Contributor

Perhaps a question on how hard/easy it is for job-seekers to find rust jobs in their location?

Contributor

Manishearth commented Apr 26, 2017

Perhaps a question on how hard/easy it is for job-seekers to find rust jobs in their location?

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor
  • For underrepresented in tech, how about something like "English is not my primary language"?
  • We should get the survey translated if possible.
Contributor

erickt commented Apr 26, 2017

  • For underrepresented in tech, how about something like "English is not my primary language"?
  • We should get the survey translated if possible.
@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt Apr 26, 2017

Contributor

From go's survey:

  • How satisfied are you with Rust support in your preferred editor?
  • What one addition would make the biggest improvement in your preferred editor?
  • My team deploys Rust/non-Rust programs to: ...
  • Where do you go to get your Rust answers from? ...
  • Where do you get your Rust news from? ...
  • I have attended:
    • a Rust Meetup
    • a Rust conference
    • Rust training
    • technical conference for Rust content
    • RustBridge event
    • Rust remote meetup
  • where could the Rust community be more welcoming?
  • for underrepresented in tech
    • I object to the question
Contributor

erickt commented Apr 26, 2017

From go's survey:

  • How satisfied are you with Rust support in your preferred editor?
  • What one addition would make the biggest improvement in your preferred editor?
  • My team deploys Rust/non-Rust programs to: ...
  • Where do you go to get your Rust answers from? ...
  • Where do you get your Rust news from? ...
  • I have attended:
    • a Rust Meetup
    • a Rust conference
    • Rust training
    • technical conference for Rust content
    • RustBridge event
    • Rust remote meetup
  • where could the Rust community be more welcoming?
  • for underrepresented in tech
    • I object to the question
@sebasmagri

This comment has been minimized.

Show comment
Hide comment
@sebasmagri

sebasmagri Apr 26, 2017

Hi! I'd like to volunteer to translate the survey to Spanish.

A few notes I left on IRC:

  • Geographical division in America will be more meaningful if we use USA/CA and Latin America instead of North, Central and South America.
  • What will be the criteria to define the options available in the frameworks question in case it's not an open field?
  • I believe that non-English speakers is definitely a good option to have in the underrepresented groups question.
  • I'd leave an open field with limited length for the most frustrating part of Rust question.

sebasmagri commented Apr 26, 2017

Hi! I'd like to volunteer to translate the survey to Spanish.

A few notes I left on IRC:

  • Geographical division in America will be more meaningful if we use USA/CA and Latin America instead of North, Central and South America.
  • What will be the criteria to define the options available in the frameworks question in case it's not an open field?
  • I believe that non-English speakers is definitely a good option to have in the underrepresented groups question.
  • I'd leave an open field with limited length for the most frustrating part of Rust question.
@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Apr 26, 2017

Contributor

I believe that non-English speakers is definitely a good option to have in the underrepresented groups question.

What does "non-English speaker" mean in this context? Someone who doesn't speak the language, someone who can speak a bit but not communicate, someone who can communicate but isn't fluent, or someone who is fluent but better with a different language?

I suspect you mean "cannot communicate in English" here; if so, we should be explicit on that

Geographical division in America will be more meaningful if we use USA/CA and Latin America instead of North, Central and South America.

Could you expand on what

Contributor

Manishearth commented Apr 26, 2017

I believe that non-English speakers is definitely a good option to have in the underrepresented groups question.

What does "non-English speaker" mean in this context? Someone who doesn't speak the language, someone who can speak a bit but not communicate, someone who can communicate but isn't fluent, or someone who is fluent but better with a different language?

I suspect you mean "cannot communicate in English" here; if so, we should be explicit on that

Geographical division in America will be more meaningful if we use USA/CA and Latin America instead of North, Central and South America.

Could you expand on what

@sebasmagri

This comment has been minimized.

Show comment
Hide comment
@sebasmagri

sebasmagri Apr 26, 2017

@Manishearth I believe there are two main things why people need English for; documentation and community support (IRC, SO, Reddit, etc). So we could probably be more explicit on this and focus on those two options in two separate questions out of the underrepresented groups section.

Regarding the geographical division in America, there is a huge difference between Mexico and US/CA for example, but there is not much difference between Mexico and Argentina or any other Latin America Country OTOH. So by grouping North America as in Mexico+US+Canada we might be loosing the ability to classify Mexico correctly, and doing US/CA, Mexico, Central America and South America looks like too much divisions across an area that is actually pretty much the same in the industry.

sebasmagri commented Apr 26, 2017

@Manishearth I believe there are two main things why people need English for; documentation and community support (IRC, SO, Reddit, etc). So we could probably be more explicit on this and focus on those two options in two separate questions out of the underrepresented groups section.

Regarding the geographical division in America, there is a huge difference between Mexico and US/CA for example, but there is not much difference between Mexico and Argentina or any other Latin America Country OTOH. So by grouping North America as in Mexico+US+Canada we might be loosing the ability to classify Mexico correctly, and doing US/CA, Mexico, Central America and South America looks like too much divisions across an area that is actually pretty much the same in the industry.

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth Apr 26, 2017

Contributor

Ah, fair.

Contributor

Manishearth commented Apr 26, 2017

Ah, fair.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Apr 27, 2017

Some kneejerk feedback as I read through.

Never used Rust

  • I'm not sure "I'm comfortable with my current language(s)" is so useful. OTOH, we're missing an option for "Rust doesn't solve a problem for me".
  • Here are my suggested options, to try to make this more crisp (with a request that the respondent leave a comment explaining in more detail for any selection):
    • Rust doesn't solve a problem for me
    • Rust is too hard to learn
    • Rust doesn't have the libraries I need
    • Rust doesn't have the tools I need
    • Rust doesn't support the platforms I need
    • Rust seems too risky to use in production
    • Rust isn't permitted by my company

Rust compiler and Tools

  • The "Do you have any other details you want to provide" question got random answers last time; let's make it more specific. "Please provide details about what broke and what was needed to fix it"
  • "If you have used Cargo, do you like it" -- what is this question trying to achieve? We know people like Cargo. Feels like an odd thing to ask.

Rust Ecosystem Projects

  • This section is somewhat oddly named. It seems mostly focused on platform and platform tooling questions, but at the end randomly jumps to talking about dependencies.
  • The "do you have any other details" question has the same problem mentioned above.
  • The "Has a minor dependency upgrade broken your code" should probably be reworded to make clear that you mean a semver minor version bump.

Using Rust at work

  • We probably want more gradations at the small count: 1, 2-5, 6-10 or something, given that higher numbers will be much less common

And that's it! Everything else looks fantastic to me.

aturon commented Apr 27, 2017

Some kneejerk feedback as I read through.

Never used Rust

  • I'm not sure "I'm comfortable with my current language(s)" is so useful. OTOH, we're missing an option for "Rust doesn't solve a problem for me".
  • Here are my suggested options, to try to make this more crisp (with a request that the respondent leave a comment explaining in more detail for any selection):
    • Rust doesn't solve a problem for me
    • Rust is too hard to learn
    • Rust doesn't have the libraries I need
    • Rust doesn't have the tools I need
    • Rust doesn't support the platforms I need
    • Rust seems too risky to use in production
    • Rust isn't permitted by my company

Rust compiler and Tools

  • The "Do you have any other details you want to provide" question got random answers last time; let's make it more specific. "Please provide details about what broke and what was needed to fix it"
  • "If you have used Cargo, do you like it" -- what is this question trying to achieve? We know people like Cargo. Feels like an odd thing to ask.

Rust Ecosystem Projects

  • This section is somewhat oddly named. It seems mostly focused on platform and platform tooling questions, but at the end randomly jumps to talking about dependencies.
  • The "do you have any other details" question has the same problem mentioned above.
  • The "Has a minor dependency upgrade broken your code" should probably be reworded to make clear that you mean a semver minor version bump.

Using Rust at work

  • We probably want more gradations at the small count: 1, 2-5, 6-10 or something, given that higher numbers will be much less common

And that's it! Everything else looks fantastic to me.

@ubsan

This comment has been minimized.

Show comment
Hide comment
@ubsan

ubsan Apr 27, 2017

Kneejerk feedback:

"My company doesn't let me use Rust" - I'd rather have something like "My company doesn't use Rust": it feels less oppositional.

"What operating system do you compile for and run your Rust projects on?" - this list should probably ordered in some way by popularity, alphabetic ordering feels wrong here.

Again for the cross-compile bit.

"Which titles do you believe best match your role?" - this has a weird, kind of alphabetical but kind of not ordering? Also, it'd be nice to have a "hobbyist" option.

"What language are you fluent in?" should be "languages". Also, if this is going to be an exclusively english survey, this seems weird to ask; I'd have to assume they're at least fluent in english, or else they'd not take the survey.

"To which of these regions would you travel to attend a Rust conference?" - This seems weirdly split up, having multiple areas for a single small continent (Europe - East vs West), versus North America, which is one place; might be better to split it up into four corners? Similarly with Africa, Africa is giant and someone who's willing to travel to South Africa may not be willing to travel to Ethiopia, or Morocco. Similarly, Russia is one country, but Eastern Russia is very different from Western Russia (I assume you mean like, Moscow, but you could specify); and by Asia, do you mean East Asia, Southeast Asia, India, the Middle East?

Otherwise, see @aturon's answer for the rest of my feedback :)

ubsan commented Apr 27, 2017

Kneejerk feedback:

"My company doesn't let me use Rust" - I'd rather have something like "My company doesn't use Rust": it feels less oppositional.

"What operating system do you compile for and run your Rust projects on?" - this list should probably ordered in some way by popularity, alphabetic ordering feels wrong here.

Again for the cross-compile bit.

"Which titles do you believe best match your role?" - this has a weird, kind of alphabetical but kind of not ordering? Also, it'd be nice to have a "hobbyist" option.

"What language are you fluent in?" should be "languages". Also, if this is going to be an exclusively english survey, this seems weird to ask; I'd have to assume they're at least fluent in english, or else they'd not take the survey.

"To which of these regions would you travel to attend a Rust conference?" - This seems weirdly split up, having multiple areas for a single small continent (Europe - East vs West), versus North America, which is one place; might be better to split it up into four corners? Similarly with Africa, Africa is giant and someone who's willing to travel to South Africa may not be willing to travel to Ethiopia, or Morocco. Similarly, Russia is one country, but Eastern Russia is very different from Western Russia (I assume you mean like, Moscow, but you could specify); and by Asia, do you mean East Asia, Southeast Asia, India, the Middle East?

Otherwise, see @aturon's answer for the rest of my feedback :)

@nrc

This comment has been minimized.

Show comment
Hide comment
@nrc

nrc Apr 30, 2017

Overall, this looks good, but it feels really long. I worry that it might put off some people. I think it could be less exhaustive and still be just as useful (especially if we get more replies).

How would you rate your Rust expertise? I'm not a fan of 'average' because it suggests that you need to know where you stand relative to others, whereas the other options are kind of absolute. I'd prefer 'intermediate'.

Along with these tools, what other tools do you use? 'these tools' is confusing, does it mean the tools before or after? I would change to 'the compiler and Cargo'. Could you add IntelliJ to this list please? (I guess it is on the next list, so maybe not) I'm not sure it is worth asking about the RLS - it is not user facing, so might be confusing, plus we have not really released it.

Cross compile questions could be simplified, I think - feels like there is redundant information here.

Which titles do you believe best match your role? I feel like this list could be halved - does it matter if someone regards themself as ceo vs founder vs executive? 'Designer' should be more explicit - does this mean design code, graphic design, UI design?

What language are you fluent in? Should be languages, likewise in the 'other' question. This seems a pretty random list of languages, what were the criteria for choosing? 'Chinese' is not a language, it should probably be 'a Chinese language' (likewise with Indian, which is not on the list, but should be). In fact, it might be better to list actual languages - Mandarin/Cantonese/Hindi, say.

'Are you interested...' questions probably don't need a 'I don't know' answer

'Challenges and Feedback' section - we should be clear about whether this means the challenges for the person answering in particular or what that person thinks the challenges for the community as a whole are.

Including the 'contact' section with the survey makes me a bit twitchy. I'd rather the survey were kept anonymous and we collect contact info elsewhere.

nrc commented Apr 30, 2017

Overall, this looks good, but it feels really long. I worry that it might put off some people. I think it could be less exhaustive and still be just as useful (especially if we get more replies).

How would you rate your Rust expertise? I'm not a fan of 'average' because it suggests that you need to know where you stand relative to others, whereas the other options are kind of absolute. I'd prefer 'intermediate'.

Along with these tools, what other tools do you use? 'these tools' is confusing, does it mean the tools before or after? I would change to 'the compiler and Cargo'. Could you add IntelliJ to this list please? (I guess it is on the next list, so maybe not) I'm not sure it is worth asking about the RLS - it is not user facing, so might be confusing, plus we have not really released it.

Cross compile questions could be simplified, I think - feels like there is redundant information here.

Which titles do you believe best match your role? I feel like this list could be halved - does it matter if someone regards themself as ceo vs founder vs executive? 'Designer' should be more explicit - does this mean design code, graphic design, UI design?

What language are you fluent in? Should be languages, likewise in the 'other' question. This seems a pretty random list of languages, what were the criteria for choosing? 'Chinese' is not a language, it should probably be 'a Chinese language' (likewise with Indian, which is not on the list, but should be). In fact, it might be better to list actual languages - Mandarin/Cantonese/Hindi, say.

'Are you interested...' questions probably don't need a 'I don't know' answer

'Challenges and Feedback' section - we should be clear about whether this means the challenges for the person answering in particular or what that person thinks the challenges for the community as a whole are.

Including the 'contact' section with the survey makes me a bit twitchy. I'd rather the survey were kept anonymous and we collect contact info elsewhere.

@erickt

This comment has been minimized.

Show comment
Hide comment
@erickt

erickt May 2, 2017

Contributor

@rust-community/community-team: here is a draft of the survey. Please look it over. We will wipe it before we go live, so please put in arbitrary data. Our biggest concerns are:

  • are we asking actionable questions?
  • are there any questions that are low value and should be cut?
  • are we too long?
  • is our demographics question offensive, or will it miss anything significant?

We also are considering cutting collecting emails in this survey and having a separate one to collect contact information for future events, but we weren't sure how the best way to implement that flow.

Please look it over! It'd be great to get feedback before we launch (today if possible!)

Contributor

erickt commented May 2, 2017

@rust-community/community-team: here is a draft of the survey. Please look it over. We will wipe it before we go live, so please put in arbitrary data. Our biggest concerns are:

  • are we asking actionable questions?
  • are there any questions that are low value and should be cut?
  • are we too long?
  • is our demographics question offensive, or will it miss anything significant?

We also are considering cutting collecting emails in this survey and having a separate one to collect contact information for future events, but we weren't sure how the best way to implement that flow.

Please look it over! It'd be great to get feedback before we launch (today if possible!)

@killercup

This comment has been minimized.

Show comment
Hide comment
@killercup

killercup May 2, 2017

Some feedback, sorry that it's so much 😅

  • Add an estimate on how long it takes to fill out the survey. Seeing just one question on the first of 16 "pages" gives me no way of guessing that. FYI, it took me about 5 minutes to read through all the questions without actually filling out any long form answers, so I'd go with something like

    This survey will take about 10 minutes to fill out.

  • Text answers are single-line fields. That doesn't invite to write much.

  • The "Yearly" option of "How regularly do you work with Rust?" seems weird. It'd rewrite that question to get another metric, e.g.

    Since the beginning of the year (the last four month), how regularly have you worked with Rust?

    • Daily
    • A few times a week
    • A few times a month
    • Once or twice (not regularly)
  • I'm not sure what "If you summed the size of all Rust projects you work on, how big would it be?" can tell you. I'd at least add another question

    What is the size of the biggest Rust project you worked on? (Lines of Rust code)

    but that will still be distorted when someone added a single doc comment to rustc (which has >100k sloc)…

  • I'm not sure if there was a question asking about the kind of Rust projects people were involved in. Something like

    What kind of software did you write in Rust? (multiple choice)

    • Haven't written anything
    • CLI tools
    • Networking application
    • GUI applications
    • Games
    • Tools for working with Rust itself, e.g., working on the compiler or a cargo subcommand
    • Format parsers (like mp4, jpeg, gzip, …)
    • Compilers (that are not rustc)
    • Other
  • Maybe these two questions next to each other

    Have you seen the Rust 2017 roadmap?

    and

    What features that are not (yet) in Rust are you most looking forward to? (either multiple choice or plain text)

  • When asking about travel plans for a Rust conf, maybe also ask

    Did you attend a Rust conference in the last 12 months?

killercup commented May 2, 2017

Some feedback, sorry that it's so much 😅

  • Add an estimate on how long it takes to fill out the survey. Seeing just one question on the first of 16 "pages" gives me no way of guessing that. FYI, it took me about 5 minutes to read through all the questions without actually filling out any long form answers, so I'd go with something like

    This survey will take about 10 minutes to fill out.

  • Text answers are single-line fields. That doesn't invite to write much.

  • The "Yearly" option of "How regularly do you work with Rust?" seems weird. It'd rewrite that question to get another metric, e.g.

    Since the beginning of the year (the last four month), how regularly have you worked with Rust?

    • Daily
    • A few times a week
    • A few times a month
    • Once or twice (not regularly)
  • I'm not sure what "If you summed the size of all Rust projects you work on, how big would it be?" can tell you. I'd at least add another question

    What is the size of the biggest Rust project you worked on? (Lines of Rust code)

    but that will still be distorted when someone added a single doc comment to rustc (which has >100k sloc)…

  • I'm not sure if there was a question asking about the kind of Rust projects people were involved in. Something like

    What kind of software did you write in Rust? (multiple choice)

    • Haven't written anything
    • CLI tools
    • Networking application
    • GUI applications
    • Games
    • Tools for working with Rust itself, e.g., working on the compiler or a cargo subcommand
    • Format parsers (like mp4, jpeg, gzip, …)
    • Compilers (that are not rustc)
    • Other
  • Maybe these two questions next to each other

    Have you seen the Rust 2017 roadmap?

    and

    What features that are not (yet) in Rust are you most looking forward to? (either multiple choice or plain text)

  • When asking about travel plans for a Rust conf, maybe also ask

    Did you attend a Rust conference in the last 12 months?

@sebasmagri

This comment has been minimized.

Show comment
Hide comment
@sebasmagri

sebasmagri May 2, 2017

Some feedback after a run.

  • Took 10 minutes for me
  • In the underrepresented groups question, the Economic, Educational, Political and Language options seems too vague, probably adding a little bit to them will make it work.
  • In how far would you travel, calculating distances is probably not easy or fast for a lot of people. Why don't we use times instead? Like how much time would you travel to attend a Rust event?
  • On the new things this year question, we should ask first if the person have seen the roadmap and put a link to it there.
  • On the Rust conferences section, I believe it's missing a question to check if the responder has attended a Rust event/conference before and which one.
  • Not completely sure on this point, but the adoption challenges question in the final page is quite open for the responder and sounds a bit to me like We have not managed to identify the reasons why Rust is not being adopted, can you tell us why?. Probably it's just a matter of tone of course. Probably this What do you think is necessary to improve knowledge and adoption of Rust?

sebasmagri commented May 2, 2017

Some feedback after a run.

  • Took 10 minutes for me
  • In the underrepresented groups question, the Economic, Educational, Political and Language options seems too vague, probably adding a little bit to them will make it work.
  • In how far would you travel, calculating distances is probably not easy or fast for a lot of people. Why don't we use times instead? Like how much time would you travel to attend a Rust event?
  • On the new things this year question, we should ask first if the person have seen the roadmap and put a link to it there.
  • On the Rust conferences section, I believe it's missing a question to check if the responder has attended a Rust event/conference before and which one.
  • Not completely sure on this point, but the adoption challenges question in the final page is quite open for the responder and sounds a bit to me like We have not managed to identify the reasons why Rust is not being adopted, can you tell us why?. Probably it's just a matter of tone of course. Probably this What do you think is necessary to improve knowledge and adoption of Rust?
@jdm

This comment has been minimized.

Show comment
Hide comment
@jdm

jdm May 2, 2017

The presence of Meh in What other tools do you use felt out of place compared to the other choices.

jdm commented May 2, 2017

The presence of Meh in What other tools do you use felt out of place compared to the other choices.

@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie May 2, 2017

Copypasting my feedback from IRC:

  • re-phrase "If you do not use the latest Stable compiler, why?" to "If you do not use the latest Stable compiler, why? Please name the specific dependency or unstable feature"
  • typo on "What platforms are you targetting?", it's spelled "targeting"
  • if you can easily reorder survey pages, One Weird Trick to making the survey seem shorter is to put the shorter pages first, so that people feel like they've made more progress when they're hitting the long pages
  • re: the underrepresented question, based on feedback from last year I'd change it from "Are you part of an underrepresented demographic in technology?" to "Do you consider yourself a member of an underrepresented demographic in technology?"
  • I would also recommend making the "personal information" page the last or second-to-last one in the survey, that tends to be the norm for all surveys
  • the question "To which of these regions would you travel to attend a Rust conference?" is very strange, some of those regions are huge

bstrie commented May 2, 2017

Copypasting my feedback from IRC:

  • re-phrase "If you do not use the latest Stable compiler, why?" to "If you do not use the latest Stable compiler, why? Please name the specific dependency or unstable feature"
  • typo on "What platforms are you targetting?", it's spelled "targeting"
  • if you can easily reorder survey pages, One Weird Trick to making the survey seem shorter is to put the shorter pages first, so that people feel like they've made more progress when they're hitting the long pages
  • re: the underrepresented question, based on feedback from last year I'd change it from "Are you part of an underrepresented demographic in technology?" to "Do you consider yourself a member of an underrepresented demographic in technology?"
  • I would also recommend making the "personal information" page the last or second-to-last one in the survey, that tends to be the norm for all surveys
  • the question "To which of these regions would you travel to attend a Rust conference?" is very strange, some of those regions are huge
@bstrie

This comment has been minimized.

Show comment
Hide comment
@bstrie

bstrie May 2, 2017

Also, given how much feedback we're getting today, I wouldn't be opposed to launching the survey tomorrow instead of today. Our (Boston's) birthday meetup is tomorrow, and I think we're the first, no?

bstrie commented May 2, 2017

Also, given how much feedback we're getting today, I wouldn't be opposed to launching the survey tomorrow instead of today. Our (Boston's) birthday meetup is tomorrow, and I think we're the first, no?

@mbrubeck

This comment has been minimized.

Show comment
Hide comment
@mbrubeck

mbrubeck May 2, 2017

Some minor feedback from a survey-taker's point of view:

Pedantry: "How long have you been working in Rust" does not have an option between "Less than a year" and "Multiple years" (e.g. for someone with 1.2 years of experience). Maybe change "Multiple years" to "More than a year"?

On the "Has upgrading broken your code", the "How much work did it take to fix?" answers are selectable even if you choose "No", and can't be un-selected if accidentally chosen. (Not sure whether this is fixable or how it'll affect the final data processing.)

In the "preferred way of installing" question, "official tarballs" or "rust-lang.org tarballs" might be clearer than "rust-lang tarballs", etc.

Typo: "If you do not to use rustup.rs, why not?"

mbrubeck commented May 2, 2017

Some minor feedback from a survey-taker's point of view:

Pedantry: "How long have you been working in Rust" does not have an option between "Less than a year" and "Multiple years" (e.g. for someone with 1.2 years of experience). Maybe change "Multiple years" to "More than a year"?

On the "Has upgrading broken your code", the "How much work did it take to fix?" answers are selectable even if you choose "No", and can't be un-selected if accidentally chosen. (Not sure whether this is fixable or how it'll affect the final data processing.)

In the "preferred way of installing" question, "official tarballs" or "rust-lang.org tarballs" might be clearer than "rust-lang tarballs", etc.

Typo: "If you do not to use rustup.rs, why not?"

@jdm

This comment has been minimized.

Show comment
Hide comment
@jdm

jdm May 2, 2017

The questions that start with If other (like the job title, comfortable programming languages, demographics, and conference locations) follow questions that do not have an "other" option. Maybe there's a way of rephrasing them?

"If you don't feel welcome, please let us know how we can be more welcoming" could be rephrased as "What are actions we could take that would make you feel more welcome?"

"This will help us understand how well our outreach efforts are going." might be better phrased as "This will help us measure the effectiveness of our outreach efforts."

Rather than "Woman" in the demographics question, what about "Non-male" to encompass non-binary people?

"What new things are you most excited about coming in the next year?" is awkwardly phrased. What about "What new things related to the Rust project are you most excited about in 2017?"

jdm commented May 2, 2017

The questions that start with If other (like the job title, comfortable programming languages, demographics, and conference locations) follow questions that do not have an "other" option. Maybe there's a way of rephrasing them?

"If you don't feel welcome, please let us know how we can be more welcoming" could be rephrased as "What are actions we could take that would make you feel more welcome?"

"This will help us understand how well our outreach efforts are going." might be better phrased as "This will help us measure the effectiveness of our outreach efforts."

Rather than "Woman" in the demographics question, what about "Non-male" to encompass non-binary people?

"What new things are you most excited about coming in the next year?" is awkwardly phrased. What about "What new things related to the Rust project are you most excited about in 2017?"

@Manishearth

This comment has been minimized.

Show comment
Hide comment
@Manishearth

Manishearth May 2, 2017

Contributor

Which version(s) of Rust do you use for your applications?

Perhaps we should have version numbers against the previous/current stable release options?

How many developers at your company work with Rust?

"On company time?". Perhaps clarify distinction between Rustacean coworkers and coworkers who do Rust for their job.

Is your company planning on hiring Rust developers in the next year?

Might want to pitch the friends page somewhere in small text on this page

If other, what underrepresented demographic are you a part of?

There is no "other" option

Contributor

Manishearth commented May 2, 2017

Which version(s) of Rust do you use for your applications?

Perhaps we should have version numbers against the previous/current stable release options?

How many developers at your company work with Rust?

"On company time?". Perhaps clarify distinction between Rustacean coworkers and coworkers who do Rust for their job.

Is your company planning on hiring Rust developers in the next year?

Might want to pitch the friends page somewhere in small text on this page

If other, what underrepresented demographic are you a part of?

There is no "other" option

@ubsan

This comment has been minimized.

Show comment
Hide comment
@ubsan

ubsan May 3, 2017

@jdm I like the current split of NB and woman. although it depresses me that woman is an option there :/

ubsan commented May 3, 2017

@jdm I like the current split of NB and woman. although it depresses me that woman is an option there :/

@sebasmagri

This comment has been minimized.

Show comment
Hide comment
@sebasmagri

sebasmagri May 3, 2017

@ubsan we're working to fix that, don't feel bad.

sebasmagri commented May 3, 2017

@ubsan we're working to fix that, don't feel bad.

@jplatte

This comment has been minimized.

Show comment
Hide comment
@jplatte

jplatte May 3, 2017

Under

What is your experience with other tools you use?

There is an option rustdoc. I've never used it directly, but I'm assuming that's what I'm using when I run cargo doc? Maybe that could be made clearer.

jplatte commented May 3, 2017

Under

What is your experience with other tools you use?

There is an option rustdoc. I've never used it directly, but I'm assuming that's what I'm using when I run cargo doc? Maybe that could be made clearer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment