Reapproach to Documentation

[innovate-invent]

Copy pasted from gitter:

Nate Coraor @natefoo
@innovate-invent It's a big, complex system and sometimes it's easier to show by example than by exact specification, and we definitely do have issues with multiple sources of information covering certain topics. We've tried to develop as much of both documentation and tutorials as possible, if there's areas that are lacking please compile a list and open an issue or present them here or on the dev channel. And of course, we openly welcome corrections and improvements.

Nolan Woods @innovate-invent
Basically Galaxy needs every config file variable, api endpoint parameter, and tool/plugin interface parameter documented. If galaxy parses it, it should be specified in a list somewhere. The closest thing currently available is the tool spec but even that has undocumented attributes or unspecified domains for the ones mentioned.
The documentation should also exist independent of tutorials or examples.
The code also needs to be more thoroughly documented. Module docstrings describing their contents, architectural documentation, and internal data model documentation. Every time I try to make a contribution to the Galaxy codebase I give up because it is overwhelming trying to orient myself in it and confidently make changes that don't have long range effects elsewhere.

Nate Coraor @natefoo
Almost all possible config file vars are documented - we don't always document every single one because there's a balance to strike between overwhelming deployers and easing the path for new developers, so the few that aren't documented tend to be only used in very small corner cases only relevant to development or are alpha-quality and that we wouldn't ever want anyone to use in production, or they are config options that have been deprecated or superseded. That said, we could have multiple sample configs, one with the most important options, and one with every single one. FWIW, this situation has improved considerably over the past few years, since now the sample and the documentation are automatically generated from a schema.

Nolan Woods @innovate-invent
There is also the issue that documentation is scattered and fragmented across the Galaxy docs, Planemo docs, and Bioblend docs.

Nate Coraor @natefoo
Yeah, that (fragmentation) is the "multiple sources" I was referring to. It's definitely one of our most important areas of need.

Nolan Woods @innovate-invent
@natefoo the documentation buried in the sample configs isn't documentation, it is a well documented example. It is not indexable or linkable. When I refer to documentation I mean something along the lines of that tool spec link I provided.

Nate Coraor @natefoo
A lot of documentation does exist independent of tutorials/examples, but as I mentioned, if you know of specific cases, please make an issue for them so they can be addressed.
That's why the documentation for the galaxy.yml config options is also in the docs proper, not just the sample. However, it's not currently the case for e.g. the other non-galaxy.yml configs, which is something we need to address.

Nolan Woods @innovate-invent
There are docs for galaxy.yml?

Nate Coraor @natefoo
But there's a YAMLification under way that has put that somewhat on the back burner.
Yes: https://docs.galaxyproject.org/en/latest/admin/config.html

Nolan Woods @innovate-invent
neat, bookmarked
except there is no index for that document
here is an example of what I have in mind: https://vuejs.org/v2/api/
It is not perfect, but it exhaustively itemizes every element, indexes them, and is centralized.

Nate Coraor @natefoo
Sphinx would probably generate a TOC for each option but there are so many that I'm not sure how useful that would be?

Nolan Woods @innovate-invent
it is very useful, when you are trying to figure out how to do something it is very handy to have a index at the top so you can find functionality inferred from the name. Currently you have to scroll through that entire document to find anything.
The thing that is missing from the vue example and the galaxy docs is specifying the domain of the parameters

Nate Coraor @natefoo
I find that search in page works pretty well for that, but yeah. I think if we could group the options it'd help.

Nolan Woods @innovate-invent
search in page only works if you already know what you are looking for, you have to look at it from the perspective of someone trying to learn to do something rather than someone just looking to verify against a reference

Nate Coraor @natefoo
It's certainly not perfect.

Nolan Woods @innovate-invent
'not perfect' is an overstatement

Nate Coraor @natefoo
We unfortunately don't have the luxury of a documentation team. The core development team has invested as much effort as we've been able to so far, and the wider community has made outstanding contributions, which are always very welcome.

Nolan Woods @innovate-invent
I guess the question is, what would it take to get the core team to do this? It can't be someone else as most of the information is in your heads.

selten @selten
What would also help, is when you figure out what a feature does or how it works, to write the documentation in a pull request. Someone might see it and add upon it.

Nolan Woods @innovate-invent
True, but the current structure (or lack thereof) doesn't allow for this

Nate Coraor @natefoo
I guess what I'm saying is, the core team is already overextended on writing documentation as it is. What I think would be helpful is for someone to lead a documentation organization effort. We could potentially propose a hackathon for it similar to the GTN hackathons.

Nolan Woods @innovate-invent
Except there is still the issue that much of the information is in your heads
approaching this forensically is the long way around

Nate Coraor @natefoo
I'm not sure what about it doesn't allow for contribution? Outside contributors make documentation PRs all the time. What @selten says is spot on - we as developers being intimately familiar with the software often can't spot documentation gaps or errors nearly as easily as the documentation consumers - who are best positioned to provide corrections and improvements.
Or at least point out specific gaps to fill.

Nolan Woods @innovate-invent
It is spread across three projects, the docs need to be unified first
The sphinx template needs to be reworked as well

Nate Coraor @natefoo
I think we've already established agreement that it needs to be unified, not sure that I agree that improvements can't be made before that point.

selten @selten
WOuld be nice if it was markdown instead of rst :P

Nate Coraor @natefoo
That's an effort underway! There should be an issue.

Nolan Woods @innovate-invent
@selten I am on the fence about markdown vs rst
rst is more robust and extensible
markdown is simpler

Nate Coraor @natefoo
You are free to write either MD or rST but the preference for new docs is MD (even though my personal preference is rST for its expressiveness).

selten @selten
I prefer MD as it's what I'm used to

Nate Coraor @natefoo
However Sphinx's MD support allows you to embed rST in fenced code blocks to get that back.

Nolan Woods @innovate-invent
Would a PR that massively restructures the doc folder be accepted?

Martin Cech @martenson
Size says nothing about the PRs merits.

Nate Coraor @natefoo
For major projects I'd suggest laying it out as a proposal, probably in an issue, so that the committers can help guide the process and avoid you having to do a bunch of revisions and suffer wasted effort.

[innovate-invent]

@eknorg It would be great to have your experience documented here. I wouldn't mind starting a proposal after gathering other input.

[innovate-invent]

I think a good direction would be to generate JSON schemas (it supports YAML) for all config/tool files that are parsed by Galaxy and render them with https://pypi.org/project/sphinx-jsonschema/
The same can be done for the api but it would be better to pull in the OpenAPI stack for that.

[alaninmcr]

I agree that a JSON schema should be made available for the Galaxy file formats. It would make integration/mapping of Galaxy far easier

[natefoo]

It's worth noting that this is already done for galaxy.yml and the new job_conf.xml replacement, job_conf.yml, using pykwalify.

[innovate-invent]

@natefoo Does pykwalify have sphinx integration?

[martenson]

My reading of this issue is that it has been agreed that having a (json) schema for galaxy-specific formats and syntax would be beneficial. This is in alignment with what the project is already doing with config_schema.yml, https://docs.galaxyproject.org/en/master/dev/schema.html, and similar cases linked above.

I am therefore closing this issue in favor of something more specific and actionable, preferably a PR or at least a more concrete need.

[innovate-invent]

Another issue is the way the documentation is published. The devs want the public to contribute to the documentation, but there is no system in place to do so. A wiki style framework is required that allows users to easily contribute. Currently someone would have to fork, clone, branch, locate the documentation in galaxies deep (and somewhat confusing) directory structure, edit, commit, push, and write up a PR, all for a single line contribution of text. Contributions aren't going to happen with that kind of a workflow.

[natefoo]

@innovate-invent for trivial doc changes like that, would editing in Github not be a reasonable approach?

[innovate-invent]

It would be, if the docs were presented with a edit link that takes you directly to the editor page of the file being viewed. I expect that contributors are going to be people who are busy working on their own projects. If it isn't simple and quick to add something they discovered then it won't happen.

[natefoo]

@innovate-invent good idea, I created an issue in #8547.

[innovate-invent]

@natefoo You should also have a look at githubs wiki system. It stores the wiki data in a orphaned branch. You can work with that branch locally, centralizing all of the docs into it.

[natefoo]

@innovate-invent I've used Github wikis quite a bit. We also moved a fair amount of our documentation and galaxyproject.org from a wiki for a number of reasons, so I don't think we are looking to move back. The PR process provides a consistent, familiar flow for the committer team to review, provide feedback, and incorporate doc changes.