[RFC] Optional documentation generation for crystal version, stdlib and shards.

[didactic-drunk]

Can the following options be added to the documentation generator to further my proposal in #5215 and for general use?

crystal docs

• --shards Also generate documentation for all shards.
• --stdlib Also generate documentation for required stdlib files.
• --stdlib-all Also generate documentation for all stdlib files.
• --core Also generate documentation for Int | String | Etc.
• --all All of the above.

Documentation for each class or method would ideally be marked/colored with the shard it came from, stdlib, core or with the most prominent indicator for the current application or shard.

[RX14]

Having a crystal docs --all which provides a single combined documentation for developers working on a project is something I've thought about and wanted before. With well-documented dependencies (ha!) it should be the only reference you ever need - and available offline. Adding indicators of where each type came from is an improvement which can come later.

[didactic-drunk]

I've tested generating --all documentation locally. Just like the crystal documentation it's useful but the list is too long when looking for information. Especially so when trying to reason about what an application does or where the boundaries are between shards, the app, and crystal.

A filter with the following may help:

All	App	Crystal	Shards
Shard1	Shard2	Shard3	Etc

Why each shard? Sometimes your hunting for basic documentation. Other times It's nice to know a shard adds methods to String or another base class.

[jhass]

Rather than having to regenerate documentation I would like to see project grouping support in documentation browser. So adding another hierarchy level on top when there's more than one project. Similar to what devdocs.io does for languages, so you can search across everything in the top level but also move into the project and limit the search there.

[straight-shoota]

As nice as this all sounds, I seriously doubt whether that fits into the scope of the docs generator. The current concept is quite clear: Provide docs for all features in src folder for the given project (shard) and the current version. That's a very clearly defined mode of operation, and yet has many aspects to cover (and lots of improvements waiting). Adding another layer on top of that, adds a lot of complexity that doesn't cover the main purpose.
As an alternative approach I'd suggest to try to improve connecting between individual API doc instances for separate shards. The main feature necessary for this would be to have way to map namespaces not included in the current API docs build to external locations where these namespaces are documented. That way individual code packages (=shards) would still have their encapsulated API docs but you can easily navigate to related API docs.
This wouldn't properly work for reopened types because shard A would list only the base features and shard B only the features added through reopening, but not the combined end result. But I think that's acceptable. Reopening between shard boundaries should be avoided anyways.

It wouldn't be difficult to build a tool on top of that that combines individual API docs to a bigger package, similar to devdocs.io.

[straight-shoota]

Some context: I'd like to have a docs build service similar to https://pkg.go.dev/ which provides automatically built API docs for shards. Setting up and maintaining docs for all your shard releases is a tasks that doesn't sound too complicated, but it's still some work that shard authors shouldn't have to worry about. They can obviously still provide their own docs. But currently most shards don't. So this would also be a service to the users who don't have to rely on authors to provide API docs.

In this use case, there would be docs for hundreds of shards sitting around next to each other and many of them have dependency connections between them. There should be a simple way to link namespaces and other features between shards while still keeping each shard's documentation a separate piece from all the others.

[didactic-drunk]

This wouldn't properly work for reopened types because shard A would list only the base features and shard B only the features added through reopening, but not the combined end result. But I think that's acceptable. Reopening between shard boundaries should be avoided anyways.

My main use case is generating documentation for an application with all API's in a single place.
Why? A 2 year old project is many crystal and shards versions behind. I may not remember the API diffences when working on multiple crystal projects with different versions.

If I need to quickly fix a bug (or security hole) I need to see the documentation with the correct versions for everything including private shards preferably without hunting around.

There may be private shards or forks that aren't available on the web which I also need documentation for.

This wouldn't work for:

• Offline use (while traveling).
• Private shards.
• Private forks.
• Reopened classes.
• Quickly seeing the entire API available to an application at a glance.

You can almost generate this sort of documentation by passing extra files on the command line to the doc generator. The only issues are:

• You have to do it yourself customized for each project. Solution: add --options to the doc generator.
• The output doesn't highlight which is a shard, base, or application API. Solution: Show the shard name and color it.
• Filtering isn't available. Solution: js filter or future feature.

As far as I can tell the work is minimal.

[RX14]

There's no reason why having one solution wouldn't preclude the other - I'm still in favour of adding an --all flag to the docs to document the entire require path.

[didactic-drunk]

There's no reason why having one solution wouldn't preclude the other - I'm still in favour of adding an --all flag to the docs to document the entire require path.

• @jhass suggested project grouping.
• I suggested a single page with minimal modification.
• @RX14 wants --all (a subset of what I suggested).
• @straight-shoota suggested a single website that hosts all shard documentation.

@jhass and mine are semi mutually exclusive. @RX14 I think provides a minimal starting point. --all could work with project grouping or without. Perhaps start there?

@straight-shoota's suggestion is independent.

[straight-shoota]

Just including all public API features into the docs output, should be really easy to achieve.

[RX14]

Including everything required should be easy, doing the complete stdlib will be harder I think. This is why docs_main.cr exists.

[straight-shoota]

I don't think this should be a problem. docs_main.cr is available when generating docs for a shard.

[nobodywasishere]

This command should allow you to get some of this functionality, just add/remove what you want/don't want. AFAICT flags like --include shards would be aliases for this regardless, though having the flag would be more convenient

crystal doc \
    src/**/*.cr \ # Repo api docs
    lib/*/src/{*.cr,**/*.cr} \ # Shards api docs
    /usr/lib/crystal/docs_main.cr # Stdlib api docs