[meta] Roadmap

[leostera]

Why. There's quite a few things ongoing/planned at the moment but there isn't very good visibility of what's next or dependencies across tasks. Grouping issues by milestones and resolving dependencies enables mainly 3 things:

1. current contributors can focus their efforts independently and efficiently,
2. new contributors get a clear picture of what they can jump on, and
3. odoc can communicate with stakeholders uniformly about what to expect in the near future

I talked to @aantron about this and he suggested using the Wiki when I proposed using Github Projects. In the past I've talked about this with @rizo as well, and recently with @ryyppy. Either way, I'll start this issue to discuss and we can move things to either tool.

Proposal. Here's some milestones with their respective issues/prs; dependencies are not by list order but indicated in each item.

---

Milestone: Reason/BuckleScript-ready

Brief. Cut a release that is distributed in npm and lays the foundation for promoting odoc as the primary documentation tool for the Javascript side of the ecosystem.

• Publish as an npm installable package #195: Publish as an npm installable package.
  Might depend on Improve support-file management #155 and using ocaml-crunch, and for easier adoption will require a high-level running script as suggested in odoc should drive itself #137. This script could be included only in the npm package and deal with other things like [BuckleScript] Automatic namespacing breaks linking of modules #209
• Bring Reason Syntax up to par with Refmt v3 #199: Bring Reason Syntax up to par with Refmt v3 — this has been a problem for bs-react-native, who have started to use odoc but have stumbled upon inconsistencies that make the docs harder to go through. Related to Reason/OCaml syntax switching #129
• Reason/OCaml syntax switching #129: Reason/OCaml syntax switching — I know @ryyppy has been further exploring this

Milestone: Better Usability

Brief. Improve readability and accessibility of content in generated docs and in odoc's documentation itself by including common documentation features and facilitating guides for writing good documentation.

• Docs-wide search #124: Docs-wide search
• Make sure in-page search works, despite any markup #125: Make sure in-page search works, despite any markup
• Improve navigation (TOC, index, etc.) #127: Improve navigation (TOC, index, etc.)
• Link from docs to source files #128: Link from docs to source files
• Document .mld files #146: Document .mld files. — Having these documented makes an incredible difference for documentation writers, possibly resolvable through Begin new odoc manuals #203.
• Document odoc, help people write good docs #120: Document odoc, help people write good docs — linked to Document .mld files #146 and Begin new odoc manuals #203
• Restore module summary for modules #161: Restore module summary for modules

Milestone: Better Interop

Brief. Define a standard interoperability mechanism to allow other features to be built outside of the core of odoc.

• odoc should drive itself #137: odoc should drive itself
• Stable structured output (a.k.a odoc json ...) #196: Stable structured output (a.k.a odoc json ...)
• Missing Issue: @rizo's package proposal?

---

That said, these milestones are not written in stone, the issues lists are definitely not exhaustive, and a release might as well cut horizontally across these milestones.

Thoughts?

cc/ @aantron @rizo @ryyppy @pmetzger @dbuenzli @grabbou

[dbuenzli]

Thoughts?

I think the basics of the tool should rather be first settled down --- I had to stop working on my odig rewrite because basically most of the driving commands are broken (see my recent issues).

Besides in general I would rather work on stabilizing a markup, especially on the problems that have been there for ages which haven't progressed for a year now and resolving the regressions that the nth rewrite of the markup seems to have introduced.

[lpw25]

Thoughts?

I would suggest that including some bug-fixing would be a good idea. At Jane Street we still use original odoc -- including adding new features to it rather than here -- because this new version can't read our code base without crashing. #108 and #178 would be a good start.

It also might be good to put the fixes needed to the model somewhere on a road map. I wrote a full description of both how the model should work and what changes are needed to get there for OCamllabs when they were considering addressing this issue. I believe that work is currently on hold but that it is intended to be started up again in the not-too-distant future. For context, these fixes -- which involve rewriting some larger parts of the model -- are required before libraries like core_kernel will look readable.

[grabbou]

Thoughts?

This looks very promising to me. I am still relatively new to the codebase, but I am happy to help as much as possible. We are already using odoc extensively in bs-react-native and want to provide a first-class documentation to ease the on boarding experience to bare minimum.

That said, I am really happy to see the list of issues focused on improving overall user experience.

[rizo]

I wrote a full description of both how the model should work and what changes are needed to get there

@lpw25 is this description available somewhere for contributors?

[lpw25]

is this description available somewhere for contributors?

Probably not. If you have somewhere sensible to put it, I'm happy to put it there.

[aantron]

@lpw25 Would the odoc wiki work?

[aantron]

Thanks for grouping the issues, @Ostera. I turned the milestones from this issue into projects in the repo: https://github.com/ocaml/odoc/projects.

Releases will cut across the projects, because we want to be able to release without completing everything, and some projects will span a long time. I created a 1.3.0 milestone for now, and put only #129 into it, because @ryyppy would like to address that issue, and I don't personally want to block the release on anything else.

We should probably add some links to projects and milestones to the README and/or CONTRIBUTING.md.

[leostera]

Happy to see 1.3.0 go out ASAP with @ryyppy's fixes! I'll update the projects with the briefs from this issue so it's easier for other people to see what they are about.

@dbuenzli: Besides in general I would rather work on stabilizing a markup
@lpw25: I would suggest that including some bug-fixing would be a good idea

Can I begin by bucketing them in a LongTerm/Stability/Scalability project and we can take it from there?

[aantron]

Ok, I added an initial draft roadmap to CONTRIBUTING,md, and linked it from the README.

Rather than being lists of specific issues, it is more "thematic" in nature, with sections Project status, General direction, Not supported in the near term, Releases, and Issue organization. We can start adding detail about specific plans to this, or just open milestones.

For me personally, right now, the General direction section is sufficient for finding things to work on and staying on track, meanwhile it's hard to know ahead of time who will want to work on what, when, and how much, so I hesitate to schedule specific issues, rather than just putting them in projects. I understand that other people might have a different view of this, so please discuss.

We can use assignments to prevent people working on the same thing. However, keeping track of this hasn't been a problem yet, so this may be a premature concern. Still, if you have access and are going to work on an issue, please assign yourself :)

I created 1.3.1 for the next release, which will probably be a bugfix release. If we end up merging a feature, we'll rename to 1.4.0.

I don't know if it's better to move the roadmap elsewhere, so please discuss.

cc @Ostera @rizo @dbuenzli @lpw25 @trefis @ryyppy

[dbuenzli]

Regarding "not supported in the near term" I think you could have been a little bit more subtle than that, but anyways. Here are a few notes:

1. There's nothing complex in supporting theming and @rizo (thanks!) already has done most of the work.
2. I don't see why odig compatibility should not be on the roadmap while bsb or dune should. odoc is just a tool to be driven by build systems and that's the way it should be developed. Besides I want to stress again that there's nothing special needed in odig beyond what a good build system needs.
3. I will have to ask you to remove the statement that odoc is in beta. It has been working pretty well for many people since at least 1.1. This project should not be treated as beta software but as a tool whose outputs are used by many users every day which should be treated with respect. It's not because you maybe have not been using it that it is beta.
4. I strongly disagree with the idea that the markup should not be stabilized, the latter has a big impact on the linking structure of the docsets and we don't want to break the links of the hundreds of docsets that are being published online on each odoc release. Stabilizing the linking structure and markup should be a priority.

Regarding 2. I would also like to remind you that odig is currently the only way to get a reasonable cross-linked set of API documentation for packages accross the whole OCaml eco-system. It is also the only way to thoroughly document the opam switch you are working in for your project and this across documentation artefacts (readme, changelogs, API refs, etc.). By explictely leaving it out and showing contempt for it you are doing a disservice to the OCaml community.

I think there should be a wake up call here: odoc is no longer in beta and hasn't been for a very long time now. It is a working tool with users which should gradually be improved.

[lpw25]

The main thing missing from the "General Direction" is the original use case for which odoc was designed and built. That is being able to run it over the installed contents of a group of packages to generate documentation for them. The build system integration is less important than that use case because you can already use ocamldoc for your own packages -- it is documenting an cross-linked set of packages for which odoc was created.

There are two implementations of this use case -- one is odig and the other is internal to Jane Street -- both of which were broken by the latest release. I think this use case needs to be explicitly recognized and supported in the road map. It doesn't require much more than what the build system integration does -- just a more relaxed approach to errors and a commitment to never require source files for building documentation -- and it is a prerequisite for the long term goal of online opam documentation.

[lpw25]

@lpw25 Would the odoc wiki work?

Do you mean the Github wiki for this repo? It's not currently enabled but I'd be happy to put these descriptions there once it is.

[aantron]

Yes. IIRC I disabled it at some point during the discussion. I've re-enabled it now.

[lpw25]

Somehow I missed your reply. I've just put the two documents up on the wiki, with a quick find-replace of "doc-ock" with "the model" so that it makes more sense with the new code layout.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.