Re-organize API reference docs to be one page per class

[bryevdv]

(This issue mostly pertains to the contents of bokeh.models)

Our reference docs are currently hand-curated into topical groupings. The original reason for the the groupings was that early on we had issues supporting and effective search capability, and because the API surface was much smaller and easier to organize by hand. Search has since gotten better and the API surface has grown considerably.

In addition to posing a manual maintenance burden, this organization also results in som very heavy pages in our docs. These large page sizes can cause problems or block new features:

• Refdocs anchors do not work #13799
• [FEATURE] Inheritance diagrams in docs #12707

I think we must split the API docs into a one-class-per-page organization. I.e. instead of:

https://docs.bokeh.org/en/latest/docs/reference/models/tools.html#bokeh.models.TapTool

there would be a dedicated page like:

https://docs.bokeh.org/en/latest/docs/reference/models/TapTool

This organization is common in other projects and is supported by some tools e.g. autoapi and others. It will take some research and experimentation to see whether these tools can be used "out of the box" or if we will need to make other changes to accommodate them or (worst-case) develop our our tooling. I'll try to do some minimal investigation to see how/if existing tools currently function.

There are some questions that probably intersect with this, such as the idea to split up the monolithic bokeh.models namespace itself at the user-facing API level (cc @mattpap I thought there was an issue/discussion about this but I can't find it)

cc @bokeh/dev for comments (not triaging as "discussion" since I think the necessity of change is making itself evident)

[ianthomas23]

I am very much in favour of this.

There might be some overlap with BokehJS if we manage to elevate that to a standalone NPM package as we'd need API docs for that, and ideally both sets of API docs would be organised along similar lines for ease of mentally switching between the two. The BokehJS model classes are already split into sensible groups so it would be good to do similar on the Python side.

[bryevdv]

@ianthomas23 can you elaborate? The structure on Bokeh sides should still be identical AFAIK. The only difference is that on the Python side model definitions are small so a module like models/ranges.py defines several models in one file. But implementations on the JS side are typically much larger, so are split out one more level, with a models/ranges directory, that contains individual files for each model implementation.

I'm not sure how that pertains here, though. These things are all distinct:

• API presented to users
• file/module layout on disk
• organization of the docs

So just being clear, this issues is not about expecting users do from bokeh.models.ranges import Range1d instead of from bokeh.models import Range1d. IMO that is Bokeh 4.0 territory. It's also not about splitting up ranges.py to a ranges directory to match BokehJS (that seems excessively fine-grained on the Python side) This issue is just about the docs presentation and can be summed up "stop generating enormous pages".

[bryevdv]

Just noting that I can get a basic configuration with autoapi building (with some new warnings) but for the results are still mixes, e.g. the help strings for properties are not currently being rendered, so some continued investigation is needed.

[ianthomas23]

@bryevdv I assumed that

There are some questions that probably intersect with this, such as the idea to split up the monolithic bokeh.models namespace itself at the user-facing API level (cc @mattpap I thought there was an issue/discussion about this but I can't find it)

meant splitting up the Python directory/class structure to be more fine-grained, but you have said this isn't so. My mistake.

[bryevdv]

meant splitting up the Python directory/class structure to be more fine-grained, but you have said this isn't so. My mistake.

@ianthomas23 no mistake I just hadn't imagined that was something people wanted, since it seems like it will result in hundreds of very small files. I'd like to understand what benefits you identify in splitting things up more are? Entirely possible I just don't see them, but I can learn :)

[thomktz]

I think the docs over at @snehilvj/dash-mantine-components have a great way of displaying each widget with examples and full docs, as shown here, with one page per widget.

[bryevdv]

I don't think autoapi is going to be an option, AFAICT it is just completely independent of autodoc directives meaning all our custom extensions for models, properties, enums, etc. will not function. I will look at sphinx-apidoc and/or sphinx.ext.autosummary next, which should explicitly work together with autodoc.

[bryevdv]

@ianthomas23 I am actually starting to come to the conclusion that "one page per model" may in fact not be possible unless we adopt the fine-grained split you suggested... (or else write yet another custom extension or literally hand-author hundreds of ReST files, but I am loathe to consider either of those options)

[bryevdv]

unfortunately autosummary is also currently a non-starter due to a Sphinx limitation sphinx-doc/sphinx#12021

Bokeh has unusual requirements, and I am not sure there is any immediate path forward without more custom tooling. I'm not sure when/if I would have bandwidth to look at that.