Proposal: redesigning the docs for Godot 4

[NathanLovato]

Proposal: redesigning the docs for Godot 4

This high-level proposal drafts a first project to review and rework the documentation by Godot 4's release.

It stemmed from a long discussion with @akien-mga where we went over broad issues with the current manual.

Table of Contents

• Proposal for the Godot docs
  • Purpose and content of the proposal
    • The proposal in short
  • 1. Focusing on documentation
    • The state of the documentation
    • What should be in the documentation, and what should not
  • 2. Defining clear content guidelines
    • Examples in manual pages
  • 3. Reorganizing the documentation to match the guidelines
    • Improving the landing page and access to content
  • 4. Setting up a writing pipeline
    • Spell-checker
    • Grammar checker
    • Prose linter
    • Screenshot automation
  • 5. Using issues to track content to write systematically
  • 6. Redesign the getting started series
  • Moving forward

Purpose and content of the proposal

This proposal reviews the documentation at large, with the following goals:

1. Improving navigation and content discovery.
2. Improving the quality and consistency of the content.
3. Completing the manual.
4. Making the maintenance load manageable. That is to say, keeping the docs up-to-date.
5. Solidifying the docs' tooling and pipeline.

As part of this project, we would like to collect the issues users from all horizons have faced with the documentation. We could do so using a form so we can have both qualitative and quantitative feedback. We should have a list of the most common and the most frustrating issues the users face with the current documentation to set our priorities well.

The proposal in short

Here is find an overview of the proposal. Please read the entire document before sharing feedback, as the sections below covers the challenges and reasons for each suggestion in greater detail.

Here are the high-level problems this proposal attempts to address:

1. The documentation is incomplete.
2. We have limited contributions and time to improve the documentation.
3. The content often doesn't follow our guidelines.
4. The content is inconsistent both in terms of quality and in its nature.
5. New features can ship in the engine without any documentation.

In short, we could:

1. Focus on documentation.
   • Move tutorials out of the manual, delegate them to teachers and tutors in the community.
2. Write clear goals and content guidelines.
3. Reorganize the documentation to match the guidelines and improve navigation.
4. Set up a pipeline and tools to check for the docs' quality.
   • Spell-checker, grammar checker, prose linter, code formatter.
   • Pair writers with an editor, like professional writers and technical writers do.
5. Use issues to track new, modified, or undocumented features systematically.
6. Redesign the getting started series.

1. Focusing on documentation

The state of the documentation

Until now, the official documentation has been a repository for different types of content:

• Step-by-step tutorials.
• Guides.
• Technical manuals.
• Code reference.

Trying to have all of these together in a single place comes with its challenges and issues. By order of importance, here are the most important ones I see:

1. Tutorials in the official documentation have a significant maintenance cost. They not only need fixes: as Godot evolves, some need complete rewrites.
2. Having mixed content makes finding information difficult. The search feature often gives answers that don't match our expectations.
3. We have to split our time between all content types. In part due to that, many features are still undocumented.

Tutorials bring other subtler issues. One is that they may set the users' expectations in a harmful way. If you find some detailed tutorials about some types of games, like the FPS tutorial series, you might come to think that the documentation is the place to learn about all sorts of mechanics or types of games. Another one is that numerous issues on the docs repository come from the tutorials.

What should be in the documentation, and what should not

Every page in the manual adds maintenance work. Godot changes, the pages have mistakes, and when doing an edit, we can introduce new issues inadvertently. This is particularly true of tutorials, where a contribution that fixes a bug at one point can introduce issues later down the page.

The content's inconsistency is both a source of frustration for docs users and a maintenance burden.

Another issue comes from hosting tutorials with Godot projects. As we distribute them as zip files, any change in the source code increases the repository's download size by hundreds of kilobytes to dozens of megabytes. It makes it hard to update the projects' source code.

We talked about these known issues at length with @akien-mga. We concluded that we should:

1. Link to tutorials on trusted external sources and limit tutorials to a minimum in the official manual. There are some fantastic learning resources in the community. For example, the Godot recipes by @cbscribe. He has more tutorials there than we have in the entire docs.
2. Focus on guides and manual pages in the documentation.

ℹ Note: if we moved tutorials out of the docs, we could still provide great code or usage examples in manual pages. But not step-by-step, long-form tutorials. See Examples in manual pages below.

We believe that curating content from the community like so would:

1. Take us less time than writing and maintaining tutorials, allowing us to shift our focus to keeping the docs complete and up-to-date.
2. Benefit content creators who make high-quality tutorials for Godot, helping them get new readers or viewers.
3. Improve the experience of the documentation's users, making the content they find consistent.

This is a model that worked well for Blender 3D. For over two decades, they delegated tutorials and courses to users, teachers, and other learning portals.

2. Defining clear content guidelines

The docs' content is inconsistent. Some pages are old, some outdated. Some guides were written without any review or proposal. The quality and the style of each page varies a lot as a result. Some pages are long and hard to read, others are incomplete or unclear. Many pages don't follow our writing guidelines.

That is in part because we do not have content guidelines. As reviewers, we often don't know what we should merge or not.

The content guidelines would be a short document, a starting point for all contributors that defines:

1. The type of content we do and don't want in the documentation.
2. The recommended length for pages.

The page should link to other guidelines, like the code style guides and writing style guides. Also, they could link to useful tools to help you assess the quality of your writing and proof your text before uploading.

Examples in manual pages

Like users need code examples in the reference, the use cases or power of many features may not be clear.

We should set a guideline to keep examples short, for example, under 400 words, self-contained, and based on real-world game code.

To me, we should ban the foobar examples from the official documentation:

1. Not everyone has the same experience or capacity for abstraction.
2. There are always alternatives that better illustrate the example at hand.
3. Users could ideally copy or replicate snippets in their projects.
4. We could copy and adapt examples from real productions.

3. Reorganizing the documentation to match the guidelines

With a clear agreement on what the docs should be like, we can reorganize them and precisely list:

1. Pages to rewrite.
2. Missing content.
3. Pages to remove.

Doing so, using issues and milestones to manage content, would give contributors a clear path forward.

Improving the landing page and access to content

Navigation and content discovery is essential in an online documentation. I've looked at how other projects handle that over the years and taken lessons from the Write the Docs community. There are different levels we need to nail to help people find and share the content we make:

1. Redesign the landing page. In particular, we should have a clear visual hierarchy between links to provide a good experience for certain types of users.
2. Use cross-references in content pages. People would navigate docs with links inside the content more so than using side menus.
3. Improve the search. To do so, in part, we might need to separate the code reference from the manual, as it tends to take over other pages in search results.

4. Setting up a writing pipeline

A lot of maintenance work is caused by typos in prose or code, grammar mistakes, and the lack of testing before pushing content.

I can safely say most docs contributors don't even use a spell checker, even though doing so would save a lot of review work. Also, considering the number of build errors and sphinx warnings we've seen pop over the years, it's clear that not everyone builds the docs or checks for errors before opening pull requests.

This both hurts the documentation's quality and adds maintenance grunt work.

We use linters, code style checkers, and other Continuous Integration in the code. Similar tools exist for prose. We could use them for the documentation. Besides text, code and screenshots need updates with every major release.

We could share a writing workflow and set up a pipeline and tools to:

1. Check for typos, grammar mistakes, and common prose issues with a spell-checker, a grammar checker, and a prose linter.
2. Run code examples through relevant parsers and linters when preparing new releases.
3. Run code examples through code formatters. We have a great GDScript code formatter by Scony.
4. Use a tool for screenshot automation.

Spell-checker

GNU's aspell, with a good dictionary for class names and technical terms, catches typos and orthographic mistakes. It's lightweight and fast in my experience. It also offers decent suggestions through code editor plugins.

Grammar checker

I don't know any Free and Open-Source grammar checker other than languagetool, the grammar checker used in LibreOffice. In my experience, it catches enough errors for our needs.

Prose linter

We can't lint prose as strictly as code as even our simple writing guidelines should have exceptions. But a prose linter can give the writer useful insights, and systematically catch patterns contributors should avoid, like writing only in the passive voice.

I've looked at proselint, textlint, and writegood. In short:

1. The three linters all cover common issues and cases. We can disable the rules we don't need to maximize speed.
2. Proselint is the only tool written in Python. The other two use javascript and typescript (if we want to add our rules).
3. Proselint is not in active development right now.

Screenshot automation

We have many screenshots in the docs. There again, they look inconsistent. Some are poorly cropped, some have annotations in a given color, others in different tones. Also:

• It's laborious, error-prone, and time-consuming to update screenshots.
• When we replace screenshots, the new ones may not have the same style or cropping, making them inconsistent across the documentation.
• People forget to or don't know they can compress images to reduce the docs' load time and download size.

We could gradually Introduce robot automation to generate and update screenshots. We need a framework or tool that can:

1. Get the interface to a given state reliably. Do things such as opening a scene, expanding a docker or a node tree.
2. Take a screenshot.

Here are three options for technologies that might work:

1. Using GDScript. We can take screenshots from within Godot and control the interface through GDScript. We would only have to write some proof of concept tool but it could work. That would also be great for tutorial makers and for the community to use it in general.
2. Robot framework: a robotic process automation framework. It can do GUI automation and take screenshots. There is a library that uses pyautogui under the hood to do GUI automation with image recognition.
3. Pyautogui and some library to take screenshots. We could have a simpler markup than the Robot framework's dedicated to our documentation's needs.

Then, we could pipe the images to relevant compression programs to automatically compress them.

In both cases, the tools are Free Software, and in Python. So they're in the same language as other tools used across the Godot repositories.

Other suggestions are, of course, welcome.

5. Using issues to track content to write systematically

We should keep track of the content to write and use clear milestones so everyone knows what to work on, and what our roadmap is. We can do so using GitHub issues and milestones or projects.

Every time a new feature or a group of features is ready to document in Godot, their maintainer should either open an issue or ping the docs team.

I think we should require developers to at least some rough notes on how a feature is meant to work or the problem(s) it solves for the docs writers. Or at least, we should have their contact information. Even quick and dirty usage notes or explanations save much trial-and-error or trying to figure out why a given feature is there in the first place.

6. Redesign the getting started series

We discussed the need for a redesign of the first-time user experience for Godot users with @akien-mga as it's inconsistent. It mixes general and theoretical information with step-by-step tutorials, catering to different types of audiences. It's neither beginner-friendly nor developer-friendly.

We should review it and define the types of readers we want to cater for here, as we cannot make it work for everyone. We could provide a nicer experience to new users by separating step-by-step tutorials for programming beginners and a general intro to the engine and its concepts.

Following the points above, we could also link to beginner tutorials from the community instead of maintaining them in the docs or consider adding video tutorials to Godot's YouTube channel, for instance.

Moving forward

This proposal is a starting point to plan concrete projects. It's open to feedback and suggestions.

To address the issues outlined in the introduction, we will need to audit the entire docs, reviewing pages one-by-one. But content guidelines and goals should come first.

Also, in parallel, I think we should gather user feedback through an anonymous form so that each user can share their experience. Doing so would help us set our priorities.

[NathanLovato]

I updated the document to clarify some bits and formatted the text.

[moonpixelgames]

This is a very good proposal, I would like to see some simpler way to contribute to the docs and still keep within the guidelines. If this is possible.

[clayjohn]

Great work! This is an incredibly comprehensive proposal. To properly do everything we are going to need to spend a lot of time laying the groundwork.

Regarding section 1. I think we need a very clear separation between tutorials on how to do something vs. tutorials on how to use a feature. While these often overlap I think it is important that we focus on explaining what a feature does and how to use it rather than explaining what combination of features is best for a particular design goal.

At the same time, I think there still is a place in the docs for long form tutorials that explain how to use a given feature. Many features cross boundaries between different nodes and resources. For example, a tutorial on using particles in 3D needs to touch on the Particles3D node, The ParticlesMaterial resource, and the StandardMaterial3D. The concept of a particle system extends across multiple classes and so is unsuitable for the class reference. But maybe this is the type of tutorial you have in mind when are mention manual pages?

Regarding code examples within the class reference, do you have an idea how to format the UI to make the reference readable while still providing code snippets. If I remember correctly, pretty much anytime we have discussed adding more code snippets to the class reference we hit a wall because we don't have a convenient place to put them without making the class reference nearly unreadable.

[cbscribe]

I'm on board. :)

My comments/thoughts as I come to them:

Focus on documentation.
* Move tutorials out of the manual, delegate them to teachers and tutors in the community.

The available population of those able and willing to write create quality written tutorials is vanishingly small. There are lots of videos out there but the quality varies wildly. I drive new users exclusively toward the step-by-step guide as a way to mitigate this - at least I can say that it's official and therefore to be trusted.

Regardless of how much gets offloaded to external sites, there needs to at least be an "Official Tutorial" that establishes a baseline of knowledge that the beginner needs to know. For example, by far the number one issue seen every single day on the help channels is "null instance" questions - beginners just don't understand how to use get_node().

This is why at least "Dodge the Creeps" should stay. It's the baseline of a working Godot project we can all point to.

Tutorials in the official documentation have a significant maintenance cost.

This is true whether they're part of the official docs or not. The problem doesn't change whether the tutorial is local and out-of-date or we're pointing users to an external site that's out-of-date. To the reader, the result is the same.

We have to split our time between all content types.

Do we? Just like on the development side, contributors are going to work on what they want to work on. If I'm exclusively interested in writing beginner tutorials, then it really doesn't matter to me how separate or integrated the API manual is, because I'm not going to work on that.

Tutorials bring other subtler issues. One is that they may set the users' expectations in a harmful way.

I think there is definitely a place for them, especially if they're small, concise, and demonstrate good coding and design practices. The biggest problem with the FPS tutorial is that it's big - too big to properly check, which is why it generates so many issues. I'd rather see several small tutorials on FPS character setup, camera control, etc.

"Dodge the Creeps" aside, making a full game is definitely not in the scope of the docs.

Another issue comes from hosting tutorials with Godot projects. As we distribute them as zip files, [...] It makes it hard to update the projects' source code.

Agreed. What's the solution though? godot-demos is already unwieldy - there are just too many projects in there. Maybe separate repos for individual projects?

There are some fantastic learning resources in the community. For example, the Godot recipes by @cbscribe. He has more tutorials there than we have in the entire docs.
Focus on guides and manual pages in the documentation.

Thanks for the shoutout. I'm happy to open the site up to more contributors. The only reason I haven't so far is the lack of interest. Again, the problem of the relatively tiny pool of writers.

Focus on guides and manual pages in the documentation.

What exactly is the distinction between a "guide" and a "tutorial"? Where is the line? I'm looking through the list of docs and it gets pretty fuzzy.

Improving the landing page and access to content

Agree wholeheartedly.

Setting up a writing pipeline

Agreed. I use aspell and write-good as linters in VSCode when I write.

I think we should require developers to at least some rough notes on how a feature is meant to work or the problem(s) it solves for the docs writers.

Figuring out a good way to encourage this would be a huge win. This is a big factor to why the docs fall behind. There's a lot of leg-work involved with just finding out what's new/changed and then tracking down how it has. The changelog, even for minor releases, can be a firehose.

Redesign the getting started series

Agreed, with the stipulation that there need to at least be two tracks:

1. Experienced, but coming to Godot for the first time.

These people just need to see how the engine works - how the editor is setup, the node/tree system, signals, and most importantly: how to find/use the API docs.

2. Entirely new, possibly without much/any programming experience at all

These are still going to be the bulk of the incoming traffic. By definition, they're not going to know where to go or how to start. If they're completely new programmers, we should steer them to existing Python tutorials and highlight the need for basic skills. Either way, they need at least some stretch of runway to get up to speed.

These two options can be big, friendly buttons on the landing page.

 "I know how to code,        |   "I'm new, 
  show me how Godot works"   |    I have no idea where to start." 

[NathanLovato]

Answering your feedback.

@moonpixelgames

I would like to see some simpler way to contribute to the docs and still keep within the guidelines. If this is possible.

We can certainly do things such as:

• Using a CI to check community contributions.
• Giving you a simple program to download and/or run all tools on given files.
• Writing a concise guide to set up writing tools. For example, editors like VSCode have plugins for spell checking and prose linting.

Each with its pros and cons, maintenance vs efficiency. Ideally, contributors should read the contribution guide first, but in practice, a contributor might edit files and push without even looking at open issues or guidelines. So the CI, linting only the users' changes, seems almost necessary.

@clayjohn

The concept of a particle system extends across multiple classes and so is unsuitable for the class reference. But maybe this is the type of tutorial you have in mind when are mention manual pages?

Yes. I'd try to keep them short and to-the-point, but at the same time, I don't find manuals that look like pure references that useful - as a tutor, I end up having to experiment a lot to figure out the intended use cases of a given feature. To me, use cases should be part of the manual.

Regarding code examples within the class reference

Just to clarify: this proposal is for this repository, not the class reference.

do you have an idea how to format the UI to make the reference readable while still providing code snippets.

If we want comprehensive examples, we could use foldable sections controlled by a dedicated "example" tag. Html supports that natively, and Godot has the control nodes we need. That way, we could have detailed code examples without cluttering pages. We'd also need support for code tabs in the code reference, for GDScript and C#: we have it with Sphinx, we'd only need to add support for it in the editor.

[mhilbrunner]

Great proposal, coherently sums up a lot of similar thoughts I had. I'm willing to help where I'm able to :)

This is why at least "Dodge the Creeps" should stay. It's the baseline of a working Godot project we can all point to.

IMO we should have two tutorials: 2D and 3D.

For example, keep the Dodge the Creeps as the 2D one, and repurpose and simplify the FPS one as a more general 3D tutorial.

The 3D tutorial should assume you did the 2D one first, so the 2D one can introduce the general editor and workflow. (Nodes and such)

The tutorials should be step by step but as concise as possible, with as few features as possible. Don't show off engine features, just how to set up a starter project with sane defaults every 2D or 3D project needs.

Any other tutorials about specific features should IMO be

• small enough to keep on the API reference for the corresponding classes, like code examples
• or be crowdsourced to the larger community and merely referenced by us

As an "advanced" user I'd then be fine just perusing the class reference with examples.

[NathanLovato]

@cbscribe

The problem doesn't change whether the tutorial is local and out-of-date or we're pointing users to an external site that's out-of-date. To the reader, the result is the same.

It's about the maintenance load on the docs contributors and the responsibility we have as maintainer. I've done quite a bit of review work.

I think that tutorials like dodge the creeps, or more than that, the UI tuts I wrote for instance, require a lot of maintenance work. Here's how I see it:

1. We might be able to keep the docs complete and up to date without the large, unattended tutorials. The FPS series for example, but there are more of them hidden in various categories.
2. To the reader, the result is different in that outdated tutorials in the docs are the Godot team's responsibility.

The persons who think that Godot's docs are bad might say it for these two reasons, maybe others.

There's a lot more to say there, but in short, I think that many of the tutorials we have now are a burden, and some are not good learning resources at all.

For the few long-form tutorials we keep, I would also split them in a series, without changing the content a lot, to ease review and maintenance. Dodge the creeps is a good example: it's your tutorial, so you could review PRs easily, but for any other reviewer, it's difficult to assess the validity of a PR in a 1000+ lines document. And it got changes from 30+ persons.

This is more of a technical detail on how we structure content and limiting the length of a given page. In short, I'd make a tutorial like Dodge the creeps a series of pages rather than a single page, and avoid going over 1000 or 1500 words on a single page. In my experience, it's easier to review and maintain.

there needs to at least be an "Official Tutorial" that establishes a baseline of knowledge that the beginner needs to know

Having one or two focused step-by-step tutorials or series with a clear purpose, sure. Ideally, we'd have a learn portal separate from the manual with tutorial series, video courses, etc. But for now, we don't.

contributors are going to work on what they want to work on

Sure. And maintainers have to maintain everything that goes in the docs. On the godot engine repo, maintainers reject PRs that bloat the engine or go in the wrong direction. The idea is to do the same for the docs. I'm sure you've also come across pages that shouldn't be there.

separate repos for individual projects?

Yes, that's the idea.

What exactly is the distinction between a "guide" and a "tutorial"? Where is the line? I'm looking through the list of docs and it gets pretty fuzzy.

For the lack of better words, here's how I use them:

• Tutorials refer to the idea of tutoring or tutors, that is, someone with their own voice, and the idea of teaching one-on-one, possibly taking the user by the hand. I'd also expect many users to perceive the world tutorial as meaning "showing me every step", from our experience with the channel and tutoring.
• Guides provide information or instruction about a topic.

I feel that the term tutorial is tainted and can bias people's expectations. I've been told quite a few times that some of our intermediate-level content was not a tutorial because we didn't start from an empty Godot project or because we gave overviews of some code on GDQuest.

---

And I generally agree with everything. I haven't talked about the getting started part too much because I'd get user feedback using forms for that.

[NathanLovato]

I'll add a general note about tutorials and courses: the background, behavior, expectations, and wishes of the users are all over the place. We couldn't cater for everyone and everyone's needs, even with a full-time tutor on Godot.

That's why I insisted so much on the tutorials and their problems in the docs. The idea is not to say we forbid them for the sake of consistency, but that we should be very careful and deliberate about what we choose to keep in and keep up to date.

Regardless of what we have, it won't be enough for many users, neophytes in particular. There, we can curate and recommend external sources, which takes way less time than creating the content itself.

We've done that on GDQuest with two curated learning paths:

• path for beginners
• path for developers

We already have a page that links to various content creators at the bottom of the Godot docs' menu.

[skyace65]

This sounds good to me. Since it's going to take a while to automate images for the docs, and I think we should prioritize other things like sorting through tutorials first, we should probably come up with a guideline.

[nhold]

In short, we could:

1. Focus on documentation.
   
   * Move tutorials out of the manual, delegate them to teachers and tutors in the community.

Are you saying remove that information from the manual entirely with no official replacement? A lot of those tutorials are really just covering concepts and use cases (I.e 2D movement) which I don't think should move. If anything we should do the inverse, encourage teachers and tutors to contribute to help keep them up to date, while not cannibalising their revenue streams.

If we were to do that (remove information and rely on outside source links) they would have to at least include C# versions of their tutorial to be included (Similar to how the docs themselves work now).

[navett52]

I really like this idea. I'm relatively new to Godot, but I'm learning. I'm mainly focused on 3D and can relate to the fact that most of the focus as far as tutorials in the docs seems to be 2D. I have plans to hopefully start becoming a good educator for Godot. I want to funnel all of my learning into tutorials. Anyway I'm really just here to say I support the idea as most of my frustrations with the docs have been noted here, and I'd like to see it change and help if possible.

[NathanLovato]

@nhold no, but I don't have a word to differentiate the types of content I'm thinking about. Let's say tutorials about specific game genres, or creating certain types of games.

We'd have to audit the docs and list that page by page. Then, the idea isn't to just remove the tuts, but to let the author to move them to their website or a community website. Especially if they're outdated and we don't intend on rewriting the content, although not only.

[puchik]

The docs' content is inconsistent. Some pages are old, some outdated. Some guides were written without any review or proposal. The quality and the style of each page varies a lot as a result. Some pages are long and hard to read, others are incomplete or unclear. Many pages don't follow our writing guidelines.

Some of the pages use different "tones" as well which can be off-putting. Some are written in first-person and read like... well... an unofficial tutorial!

It's about the maintenance load on the docs contributors and the responsibility we have as maintainer. I've done quite a bit of review work.

I think that tutorials like dodge the creeps, or more than that, the UI tuts I wrote for instance, require a lot of maintenance work. Here's how I see it:

1. We might be able to keep the docs complete and up to date without the large, unattended tutorials. The FPS series for example, but there are more of them hidden in various categories.

The FPS tutorial for C# is incomplete past the first part, as well. You can do the first page, but then after that, only gdscript is available... I've been working on translating the rest bit by bit but that also brings up consistency issues since working in gdscript and C# you'd name your signals differently, but only the code tab can be switched from one language to another...

I will say, though, that having the tutorials in the docs was pretty useful for me. I'd think it might at least a good idea to just separate them better... There's a huge section called "tutorials" right now, but a lot of the pages aren't actually "tutorials". For example, the GI Probe page: it's on the same level as the FPS tutorial, but the FPS tutorial is a multi-part step-by-step series, whereas the GI Probe page is just a description of the GI Probe.

I feel like a lot of things under "tutorials" should be under "general" or something like it.

But to add to that, the FPS tutorial is still very valuable imo... It might be good to have an official resource of that size readily available for an extensive lesson on the engine.

[shayneoneill]

One thing I think might be worth looking into as a longer-term thing is the Doctest idea from the python world. Basically the idea is each method, class, etc you do a heredoc documentation in the code, but you also include examples.

Ie

"""
The bogus method takes two parameters, and returns the sum of them

> Bogus.add(10,20)
>> 30
"""

This first off provides really usefull class documentation with examples. PLUS doctest can then run all the examples and they essentially form unit tests. If the example has to be a bit longer to set up the test, all the better because it provides a more in-context example.

Not sure how you'd implement this with GD script (Or C#) but it essentially folds documenting and testing into each other making lighter work for everyone.

Worth thinking about down the track!

[Calinou]

PS: I opened a PR for continuous integration recently, but it doesn't abort on warnings yet as there are still some warnings present when building.

I think we should require developers to at least some rough notes on how a feature is meant to work or the problem(s) it solves for the docs writers.

With @akien-mga, we decided to have a rule to require contributors to document newly exposed properties/methods, but it's not always enforced yet, unfortunately. We should write it out in CONTRIBUTING.md so people are actually aware of this requirement. This will probably seem tedious to some, but it ensures the class reference coverage never decreases.

@shayneoneill That can technically be done with an external tool that reads GDScript code, but I don't think it's particularly suited for "manual" documentation.

[nezvers]

I still love the way the GameMaker manual is laid down. Clear hi-level sections, that branches down to functions and variables that have description, syntax, return type, and example code.

[davthedev]

Here are my thoughts:

Writing style

I totally agree the need to have a consistent writing style. As the documentation grows internationally, so should the writing style be checked against rules in each different language. These rules should have contributions by native speakers of the corresponding language.

Separation between docs & code reference

Separate toplevel sections for code reference and actual documentation would improve information search. Most big projects do it; look at most Google product docs for instance.
Code reference is useful on how to use a specific class; it does not help when looking for general, big-picture advice.

Tutorials

Sure, as the engine evolves, some tutorials get outdated and it's a burden to rewrite these. Especially if the original maintainer has left the ship.
By moving tutorials out of the platform, we reduce our burden as documentation maintainers. On the other hand, some content writers who only want to share their knowledge in a step-by-step way already have a place to write as it is now. By officially stating to move out, they will have the extra burden to set up their own blog / Medium account / whatever, deal with SEO... Sure, it can be rewarding but not everyone is willing to do that.

When I started with Godot, I learnt the basics with... a tutorial. I don't forget it. These are essential entries for making new users get confidence with the engine and remain.
A few day ago, I wanted to explore the realm of GDNative. What brought me confidence on the topic is... a tutorial. And something that allowed me to pickup something as advanced as rebuilding the engine with my own module is... once again, a tutorial!

A solution in the middle would be to have some kind of official tutorial repository. Either a comprehensive database of links with summaries, or a dedicated writing platform where one can setup an account and begin writing. Furthermore, a "quality seal" for tutorials could be created to encourage consistency in the contents. The tutorial repository would show the seal along the tutorial links or pages that comply with the guidelines.

My proposal for such guidelines would include:

• Consistent screenshots like in the original proposal
• Consistent writing style like in the original proposal
• Does not begin by "get that boilerplate code from my GitHub repo first and we'll build on it" (possible exceptions for getting assets like art and audio to work with, or libraries). This is something that impacts most of the web development tutorials, which become unusable as the said boilerplate repos get outdated fast.
• Operating system agnosticity. Godot is multiplatform; so should be the tutorials. (Exceptions for, maybe, using a library that only makes sense in a specific operating system). Installation instructions, command-line examples should be declined in specific OS sections if necessary. An issue in the web development community is that lots of tutorials suppose the reader to use a Mac, ruling out those who work on Windows or Linux. The official docs are a good example on that point.

In addition, the tutorial repository would mention which version of Godot the tutorial is written for. This would make it easy to spot knowledge at risk of being inaccurate by the reader.

It would also encourage the presence of more tutorials in written form. Even if its main purpose is to be followed step-by-step, a written tutorial can be quickly scrolled & searched for the bit of knowledge you need right now that may take you out of the ditch.
In contrast, a video must be watched in its entirety to get to the information you want and you cannot copy-paste code from the still frame on YouTube. Note to content creators out there : videos are here to stay, the community already does an outstanding job on that point and I encourage both forms of media to exist.

[NathanLovato]

Let's not lose focus of the task at hand: redesigning http://docs.godotengine.org/en/latest/ by the Godot 4.0 release.

The idea of making a tutorial repository, a learning portal or website for Godot, separate from the docs, etc. makes sense. But one thing at a time.

Keep in mind that what we have now, the incomplete docs with its problems, is the result of 5 years of work by close to 500 contributors. We have limited output and a maintenance problem here and now. Godot 3.2's new features are largely undocumented. Godot 3.1's features were not all documented either last time I checked.

That's why I insist so much on trimming the docs and setting strong guidelines with the goals to improve quality, maintainability, and completion.

Once we have that going, we can think about the next step.

[YuriSizov]

Would it be good to add some form of a banner on top of each tutorial/guide/manual/whatever telling whether is has been proofread and updated for the current version? Kind of like Wikipedia warns against latest contributions not being checked yet.

In the article's layout it can be a special macro that is fed the latest supported version of the engine it has been validated against. If it matches the branch the article is read at, it's a green box telling that the quality is good. If it does not match, then the box is orange, telling to read with caution, report it and ensuring that we are going to check it soon.

I don't know if it's going to reduce confidence in our documentation, but it can at least be a marker for everyone and a kind of a temporary disclaimer. And, if implemented properly, can be easily searchable by tools. Like, the macro can be as short as

.. valid-for-version:: 3.2

[pyacier]

This is a good proposal and I will only react on the tutorials subject.
As the others, I think the management of the tutorials is a tricky part. I agree that there is advantages to move part of the tutorials out of the documentation.

But I see some possibles disadvantages :

• When i see the documentation translation completion, it make me think that the outsourced tutorials will never be translated, not great for the no-English speaker.

• It could make finding tutorials more difficult.

Apologies of my lack of understanding of the organization around the whole Godot project, but, is this problem not highlight the lack of an helpers team, other than the dev/contributor team, a team doing things like writing tutorials, translating, maybe community management ?
I say that, but I understand that it’s hard to find volunteers.

Apologies for my english.

edit: "simply" was a poor choice of word.

[mhilbrunner]

That's why I insist so much on trimming the docs and setting strong guidelines with the goals to improve quality, maintainability, and completion.
@NathanLovato

Agreed. Strongly. :)

Again, IMO we should focus on a maximum of two tutorials - one 2D, one 3D. No more. And simplify those as much as possible.

is this problem simply not highlight the lack of an helpers team, other than the dev/contributor team, a team doing things like writing tutorials, translating, maybe community management
@pyacier

Yes, of course. "Simple" is relative, however. There are hundreds of docs contributors, yes, but only a small group of people doing maintenance and proofreading work and updating and fixing the docs with each Godot change.

That small group grows very slowly. Sure, if we suddenly had 100 docs writers who solely worked on keeping everything up to date, this discussion would be moot.

Thats not the case however, and it won't change easily or anytime soon.

So we need to find a way to keep the work load manageable. Or live with very outdated docs, I guess.

[pyacier]

@mhilbrunner : I do understand it’s not simple, I translate the documentation for some time now, “simple” was a poor choice of word. I apologies to the people that work on the documentation that could have feel insulted.

What I wanted to say, is that maybe we need to ask the community for more help. Maybe some people want to help but don’t know how, maybe we need to make more rewarding, I don’t know.

This proposal seem to be good and will probably make the documentation more efficient to maintain.
But the project is growing so maybe we also need more people.

[willnationsdev]

Great proposal.

Pages to rewrite.
Missing content.
Pages to remove.

After getting the content guide written up, how do you propose people track/catalog who has reviewed/categorized which pages to be sorted into these categories? Use a HackMD page like I've seen in the past for docs contribution events? I feel like you'd need a finite set of approved editors to give the final say, but at the same time, you want the general public to submit proposed changes. In which case, it's more like generating an issue for every page and then having pull requests that directly close specific issues of pages. But that feels like overkill.

---

I also agree that the biggest priority should be in standardizing and automating the community's contribution expectations (content guide, style guide) and toolkit (linter, CI, etc.). Once that work has been done, we will have overcome the bottlenecking operations that prevent other people from effectively contributing to the documentation. At that point, it will be easier to crowdsource the work and make a call for contributions in the form of auditing/peer-reviewing all of the existing documentation to sort it into the appropriate categories.

[NathanLovato]

but at the same time, you want the general public to submit proposed changes

At first, if you want quality and consistent work, I'd say no, you don't want the general public involved in writing content. Just like you don't want a wiki or a way for beginners to edit your files directly in the docs of complex software.

In short, training a technical writer or a tutor takes at least months full-time. Editing a poor starting article often takes more time than writing it from scratch for an experienced writer. I've experienced that more than once, having done my share of editing in the docs, and doing editing work with my team.

That is even with tools, as the most difficult part of writing docs and tutorials isn't English grammar, it's pedagogy.

I did my share of review work to help bring people on board and train them, but it's a time sink. Coordination is a time sink.

I'm thinking of a project like this a bit like the big refactoring happening in Godot 4: if we're going to make big changes, you need people who are clear about the goals and tasks at hand to break stuff and lay down solid foundations.

Once you have the groundwork done, you can open the branch to contributions once again.

[Sauermann]

As someone with a decent development background who started playing around with Godot just recently I have to say that the documentation at it is today helped me to get very quickly to a state where I can work with Godot. Thank you for the current state of the documentation and your ongoing effort to keep it in top shape!

So here are a few notes from me as a documentation-consumer:

• Regarding outdated tutorials and guides, to me it is all about expectations. If I see that a tutorial is annotated as "based on Godot V2.0", I would read it with the assumption that it is a little bit outdated and use it only for general guidance, however it there is no version-annotation, then I would expect that it works with the current stable version of Godot. Requiring "based-on-version X.Y" annotations for tutorials would help me to get a feeling for them before I start digging into them and help me find the reason, when something does not work as expected. This would not solve the problem of outdated tutorials, but would minimize the workload for maintainers as these tutorials are expected to get outdated over time.
• Separating the actual documentation from user contributed tutorials (for example by setting up a separate location on the Godot-website) would help distinguishing the both sections and give tutorial contributors the ability to contribute without needing to set up an own blog.
• Personally I do not care about the style of the documentation, I just care that it is concise and clear (I did not even notice that apparently the documentation has different styles)
• While learning the GDScript language I often need to look up the basic syntax and native functions. In the meantime I learned, where I can find them in the Documentation-menu, but initially I had my problems finding the sections about these topics, because the search engine would only rarely show them in the top-search-results. I would hope that a restructuring of the documentation would lead to an easier access to the GDScript basics for beginners.

[hectordominguez]

i completely agree

[m-trigo]

Thank you for this proposal and your work with the docs so far.
As an aspiring contributor, I would like to make sure my contributions are beneficial.

As a developer, my biggest grievance with the online docs is the usability of the sidebar. My preference is to use the built-in Search Help, but that's often not possible unless I know the name of what I'm looking for.

I support the sentiment of delegating tutorials to external sources and shiting effort into more maintainable documentation formats.

[Calinou]

I opened a PR to fix many typos using codespell in #3395. There are a few false positives, but these should be easy to manually exclude. I think we could integrate it in the CI process eventually.

This is the smallest exclude list I could use to avoid false positives after the PR:

codespell ^classes/**/*.rst -L doubleclick,dof,lod,que,raison,uint

As for LanguageTool, the following command works but still generates quite a lot of false positives (you need GNU parallel and the English ngrams installed):

parallel java -jar languagetool-commandline.jar --language en-US --languagemodel ~/Documents/ngrams -d ARROWS,COMMA_PARENTHESIS_WHITESPACE,DASH_RULE,DOUBLE_PUNCTUATION,EN_QUOTES,ENGLISH_WORD_REPEAT_BEGINNING_RULE,ENGLISH_WORD_REPEAT_RULE,MORFOLOGIK_RULE_EN_US,MULTIPLICATION_SIGN,PUNCTUATION_PARAGRAPH_END,PLUS_MINUS,SENTENCE_WHITESPACE,TR,UNLIKELY_OPENING_PUNCTUATION,UPPERCASE_SENTENCE_START,WHETHER,WHITESPACE_RULE,WORD_CONTAINS_UNDERSCORE {} ::: "$PWD"/^classes/**/*.rst

Edit: More typo fixes and grammar improvements in #3396.

Edit 2: Basic CI set up in #3404.

[realkotob]

I agree with most of the points, but I disagree with separating step by step tutorials.

When a user wants to save the game, they're not going to search for "godot file io json saving." I've visited this tutorial so many times over the years it's ridiculous. It's more relevant to be in the docs than the actual File IO reference.

If anything, I think there are some more tutorials that can be part of the official docs, such as Menip's multiplayer docs. The current High Level Networking docs are bordering on detrimental (they're well written but there are gaps that aren't addressed).

I can give many examples like these that would be downright criminal to remove based on a beurocratic formality.

I believe that outdated information will always be more useful than no information at all, only because there's rarely enough change to something that makes a certain guide completely obsolete.

I'd say that the step by step tutorials are a main reason why the docs are great, because it provides a way for the community to centralize its information.

If we urge content creators to host guides themselves, most of it will disappear into the void when they close their blog, and there's a 99% chance it won't be even listed by Google's search algorithm on the first results page.

If I find something outdated, it's very easy to search keywords in class reference and then fix it immediately via github's browser editor, or at worst post an issue for others to help me with.

Having maintainers for the docs also helps content creators make sure the quality of their content is friendly to read and parse, since if they host it on their blog it'll very likely become a technical mess. They might even make mistakes and provide inaccurate information. With no other similar guide to follow, this would be unpleasant for everyone.

While I do agree that maintaining such a huge volume of content can be difficult and tiring, I think the alternative is much much worse, and would be worse for the community overall.

The Arch wiki is a great example of a community wiki, since it has practically everything you'll ever need to fix your case scenario, not only for Arch but even on any other Linux distro.
No one has ever complained that the "Arch wiki" has irrelevant content, quite the opposite it's a point of awe and admiration.

I'm not saying we can't do something better than what we have now, but I urge you to not make a hasty decision that would cause more harm than good.

I've been using godot in hobbyist and professional capacity for half a decade, yet still when I search for something of intermediate to advanced difficulty on Google, the only results I have are the official docs, the Q&A forum (often unanswered question), or a reddit post.

We haven't reached a threshold amount of online 3rd party resources that make this separation a wise choice. Maybe soon, after 4.0 is released, we'll see a major influx of new intermediate-advanced content creators. But definitely not now.

Edit: Just today I was looking for something and came across several dead pages already, like here and here. I was able to see the content through the google search cache but it's only a matter of time before it disappears.

[follower]

Or is it done on a server like Travis? [...] While in the second case we would need a server with GUI, and that would be an overkill just for that.

A "headless" solution like xvfb ("X Virtual Frame Buffer) would mean this shouldn't be required. (I've used it in the past for initial testing of the Arduino Java-based IDE which originally wouldn't run without GUI support and more recently for running (either Linux or Windows--I don't remember) Godot via a SSH link without X support. (It was required because running a script from CLI still requires GUI support on Linux, from memory.))

It may be an issue with 4.0 & Vulkan however: godotengine/godot#38428

[follower]

In terms of screenshot automation another possible tool is SikuliX:

• A little unclear but I think the most actively developed version is: https://github.com/RaiMan/SikuliX1

• Written in Java but has some level of Python support via: https://github.com/RaiMan/sikulix4python

Having said that, I suspect that a GDScript-based approach (using maybe internal-only functionality) is probably preferable.

Exposing UI structure for screenshot automation would also be an opportunity to improve accessibility for screenreaders.

[omicron321]

concerning 3. Section. Improving the landing page and access to content

In doc navigation left column, some sub tree might be unecessarily deep: we don't always know if it is having deeper pages or if it is displaying links to current page subtitles and require additionnal clicks as menu trees are collapsed by default.

At end of page (apart next button), it is uncertain which section will show up, or if we missed a page, before manually opening those menu trees.

One suggestion would be:

• flatten menu trees when possible, and/or, expand them by default (probably the cheapest to do)
• and update highlight menu tree item, whenever new anchor inner link reached (= when it goes of top position of page)

[NathanLovato]

@Omicron666 These are some good suggestions. Could you open two new issues for these two proposals? I'm afraid they'll get lost in here otherwise.

I'll close this issue because the proposal's already been accepted and I already did some work from it, so this discussion's already kind of over; there are dozens of new open issues to tackle that stem from this proposal.