Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Planning a survey for Rust game developers #21

Open
erlend-sh opened this issue Jul 6, 2019 · 27 comments
Open

Planning a survey for Rust game developers #21

erlend-sh opened this issue Jul 6, 2019 · 27 comments
Assignees

Comments

@erlend-sh
Copy link
Member

erlend-sh commented Jul 6, 2019

I mentioned several weeks back that I wanted to do a survey. Will try get that drafted here now. I plan to do something similar to what I’ve done before with Discourse. Main difference will be that I plan to source the survey answers through issue/forum replies, so survey answers are public to all (except for private answers sourced through a Google Form).

The survey will consist of a few simple questions, with mostly (or completely) freeform answers. If you have some pressing questions for the Rust gamedev community, please bring them up here for consideration. If we find ourselves with an abundance of good questions we’ll just have to do multiple surveys over time.

Questions I’d like to ask the Rust game dev community:

  • Do you consider yourself a hobbyist or a professional game developer?

    • hobbyist
    • professional
    • neither
  • Are you currently using, or considering using Rust for game development?

    • currently using
    • considering using
    • have not yet seriously considered using
  • What about Rust as a language and ecosystem presents the biggest downsides for you as a game developer right now?

  • ...biggest upsides?

  • what are some libraries you are thankful for or take special interest in?

  • What do you think should be the GD working group's priorities for the next 3-6 months?

@bitshifter
Copy link

I know of many people working in game development, both AAA and indie who are Rust enthusiasts but aren't using Rust in their professional work. Would this survey cover them too?

@erlend-sh
Copy link
Member Author

Would this survey cover them too?

As long as we're able to reach out to them and get them to answer the survey, absolutely!

I think we should add one more question then, conditional on checking "professional":

  • What would need to be different about the state of game development in Rust in order for you to use it professionally?

@msiglreith
Copy link

Awesome! Sounds like one of the most important tasks to me. Getting a better understanding of the different subareas (audio, graphics, threading, math, physics, etc.) would be ideal as basis for further discussions. Would be interesting to see which libraries are used in the specific areas and if people are rolling their own ones, why so?

@Lokathor
Copy link
Member

No one has more NIH in their blood than a game dev does.

@erlend-sh
Copy link
Member Author

Survey has been published:

https://users.rust-lang.org/t/survey-from-the-rust-game-development-working-group/31270

@msiglreith
Copy link

I would be interested what's the state of this topic?

@ozkriff
Copy link
Member

ozkriff commented Sep 15, 2019

https://twitter.com/erlend_sh/status/1167343250258223105

Yep! The survey is considered soft-closed now and I’ve begun summarizing the answers. We got 360 responses so it’s gonna take me a few days though.

@erlend-sh
Copy link
Member Author

Yup, still on this! Been quite occupied lately but it’s 80% done now.

@Wodann
Copy link
Contributor

Wodann commented Jan 28, 2020

@erlend-sh Were you able to wrap up the results?

@erlend-sh
Copy link
Member Author

Survey results were at long last finished up today. I’ll make a PR tomorrow.

@ozkriff
Copy link
Member

ozkriff commented Mar 11, 2020

This can be closed now, right?

@AlexEne
Copy link
Member

AlexEne commented Mar 11, 2020

Yup. Thanks for all the hard work everyone!

@AlexEne AlexEne closed this as completed Mar 11, 2020
@erlend-sh
Copy link
Member Author

I'm preparing a new survey for 2020. You can test it here:

https://docs.google.com/forms/d/e/1FAIpQLSfu1gAi_Gbp8ODJY3tYkXkYS6nQI0G_qaHvwkIfGNNljuETdg/viewform?usp=sf_link

I changed last year's free-form entries into a ranked-choice vote.

@erlend-sh erlend-sh reopened this Sep 30, 2020
@repi
Copy link

repi commented Sep 30, 2020

@erlend-sh nice! any specific feedback you want on it? When do you plan to send it out?

@erlend-sh
Copy link
Member Author

erlend-sh commented Sep 30, 2020

Sending it out in a week or two.

I’d love to know if there is something more specific we could try dig into in addition to the “most important?” query.

Maybe another ranked choice for “What has been taking up more of your development time than you’d like?”, and we could list all major areas of development like Graphics, Physics, Networking, Input, Audio etc.

What do you think about the free-form question?

What has been the GOOD (worked well), the BAD (difficult) and the UGLY (works, but...) of your Rust gamedev journey so far?

I want to solicit brief stories from people.

@17cupsofcoffee
Copy link
Collaborator

The ranked choices seem like a good idea (nice to have a mix of hard data and free-form feedback) - wonder if it might be worth having an optional field where people can list anything they feel is important that was left off the ranked list? I can't think of anything off the top of my head, but it might be handy for refining the list next time round.

@kvark
Copy link

kvark commented Sep 30, 2020

I feel like the "What is most important for you right now as a Rust game developer?" question is a bomb there. All the other ones are small and quick, but this one is heavy and overloaded. Could it be split into several questions, touching different areas? For example, one question could be about platform support, such as Console/Web/Mobile, integrating with toolchains. Another - about dev experience, including IDE, compile speed, and documentation. Splitting would also help to solicit separate pieces of feedback on those areas in custom "type what you want" fields.

@erlend-sh
Copy link
Member Author

Yes! I was thinking exactly the same thing. Respondents shouldn’t need to rank more than 3-5 items at a time.

@erlend-sh
Copy link
Member Author

Here's what I've come up with:

https://www.surveymonkey.com/r/F2JYRFF

@17cupsofcoffee None of the survey systems I've tried have a "custom entry" option to go with a ranking question. I think the workaround will just have to be the last free-form question.

@kvark
Copy link

kvark commented Oct 21, 2020

Most important for ecosystem maturity

It's weird that one can both set the priority numbers and re-order the entries. Just re-ordering would be nicer.

We could also add "polish existing crates better", since this need doesn't get covered by the listed entries.

Most important for a "full-featured" Rust game engine or framework

This could have an additional "Performance" choice.

Most important ecosystem-segment to contribute to right now

This could have an additional "Organization" kind of choice for ECS and things

Good,Bad,Ugly

Can we have those multi-line? If somebody wants to provide more info, we don't want to penalize this.

@17cupsofcoffee
Copy link
Collaborator

The ranked choice questions look way better now than they did with the big 1-10 grid 👍

@erlend-sh
Copy link
Member Author

We could also add "polish existing crates better", since this need doesn't get covered by the listed entries.

Is it meaningfully different from "More v1.0 crates, less churn"? More v1.0 crates is what "more polish" looks like to me in the less abstract. How do you measure "polish"?

This could have an additional "Performance" choice.

This could have an additional "Organization" kind of choice for ECS and things

✅ Added as "ECS and similar"

Can we have those multi-line? If somebody wants to provide more info, we don't want to penalize this.

Unfortunately we cannot. As a workaround I added a notice about it:

"Note: This would be a multi-line field if we could make it so! Feel free to write multiple paragraphs."


I'm gonna go public with this in 24hrs if there aren't any objections.

@kvark
Copy link

kvark commented Oct 29, 2020

I just cringe every time I see "more v1.0 crates", since versioning is very loosely related to the actual level of completeness and polish. Some authors kick off with a 1.0 release (e.g. https://crates.io/crates/density-mesh-core). Some crates go nuts and just keep releasing major versions ("nom", "intl_pluralrules", "triple_buffer", "raw-cpuid"). Extreme examples reach double-digit major versions without any trouble (looking at you, "core-text", "wast", "glean-core"). Is that what the ecosystem needs, really?
I'd rather have more polished v0.x crates than just more v1.0 crates :)

@erlend-sh
Copy link
Member Author

erlend-sh commented Oct 29, 2020

Some crates may diverge from the SemVer standard, but afaik it's still the best available measure we've got for API stability. What I'm trying to get at here is that in a year from now, we'll want to measure somehow if the overall stability/polish of crates went up in a meaningful way (and if so, then dig into the cause behind the effect). We definitely don't need to constrain ourselves to any single data point; the more the merrier.

I've changed the text to "More stable crates, less churn".

@aclysma
Copy link
Contributor

aclysma commented Oct 29, 2020

Feedback for https://www.surveymonkey.com/r/F2JYRFF:

  • For contact details, I would put (Optional!) on the header. (yeah it has no asterisk, but I think it changes the tone of the question for the better)
  • Hobbyist or developer - I would use the word "professional" instead of developer.
  • For most important ecosystem maturity, I'd change options:
    • Documentation: Documentation/Tutorials
    • More 1.0 crates, less churn: More polish/API stability in crates that currently exist
    • Add new option: New crates that fill gaps in the existing ecosystem
  • "Wide cross-platform support" could maybe be "portability"
  • "Iteration time" instead of "iteration times"
  • Highest priority publication target
    • I wouldn't exclude PC, I'd like to see PC vs. web

Do we have multiple people with access to the results of this survey?

Thanks for putting this together!

@erlend-sh
Copy link
Member Author

erlend-sh commented Nov 5, 2020

Do we have multiple people with access to the results of this survey?

I'll share it with anyone in the WG org who requests it. My email is public on my profile.


Survey is out! 📬

https://twitter.com/rust_gamedev/status/1324349001706229760
https://users.rust-lang.org/t/rust-gamedev-ecosystem-survey-2020/51094
https://www.reddit.com/r/rust/comments/joj5e0/rust_gamedev_ecosystem_survey/

@neachdainn
Copy link

It's probably too late for this but I would like to point out one thing in regards to the survey questions:

There is a non-zero percentage of the Rust game-dev community that aren't making games but are making simulations. For the most part, the two overlap in that they utilize much of the same techniques and technologies. However, the end goals are different - my programs are likely to only be used by me and aren't intended to be entertainment. The survey is currently written in a way that makes it difficult for me to answer in a useful way. Am I using Rust for game development? Well, no. Am I using Rust professionally (academia) for simulation? Yes.

It might be the case that I'm talking about a demographic so small as to not be interesting, or maybe I'm mistaken in simulation also falling in this working group, but providing a way to indicate between the two might provide useful information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests