The next direction for Haddock

[Kleidukos]

With the landing of Hi Haddock, I'd like to take some time to think about what we want to focus on for the year 2022.

Some things that need attention

• The CI: it's completely borked at the moment;
• The HTML output: There are many contributions to it, but very few people have real expertise in web design & integration;
• The 42 open bugs
• The multi-branch flow that leads us to push releases from old branches instead of the current ghc branch

Some R&D things that would be awesome to have

• Markdown support
• Native integration of tintin (Hi @javiertoledo!)

What can we do from here?

• I personally can't understand why the CI is broken, so I'm going to need someone familiar with the beast.
• Should we have our own CLI tool for release preparation that checks that we're indeed on the proper branch when producing Hackage releases?

I feel like these are pre-requisites to onboard more people in the development cycle of Haddock.
There is also some architectural design that needs to take place so that the whole thing is better understandable and maintainable by the next generations.

• I know that we can invest in Frontend expertise through an HFTT process, so I'm not overly worried about this.
  • But having a fixed list of things would be better.

At the moment we are in a difficult situation, and we need to prepare the grounds for the next generation of maintainers. Being constantly in "damage control" mode is an endurance race that we cannot win. And while I'm grateful for the guidance I've received in the past year, I don't feel qualified to teach the next maintainers.

cc @harpocrates @alexbiehl @bgamari

[javiertoledo]

Hi @Kleidukos, thanks for thinking about tintin, integrating it natively with Haddock would make a lot of sense, what can we do to help? We'd love to understand what you have in mind! /cc @NickSeagull

[ulysses4ever]

I was looking into Markdown support some. There is a working parser into the Haddock AST (credit to Alec), but so far I don't see an easy way to swap the parser in depending on the flag. It's probably just me not knowing the codebase enough, and I fail to find time to get myself more familiar with it. It should be a low-hanging fruit!

[harpocrates]

What's the new state-of-the-world regarding where to contribute patches? IIUC, https://gitlab.haskell.org/ghc/haddock is the place where GHC related changes go (and where Hi Haddock landed), but is there any plan for getting changes from here to there (or vice versa)? Now that Hi Haddock has landed (and the massive cloud of potential merge conflicts is no longer hanging overhead), I wouldn't mind trying to resurrect/fix some older PRs and issues, but I realize I'm not sure where those patches even go anymore.

I've messed with the CI process before and wouldn't mind doing so again (but again, I'm not sure where the patches go). There's some amount of --allow-newer (and finger crossing) needed for the testsuite to build, since Haddock tends to track the latest GHC release and the transitive dependencies of the tests tend to take a little bit of time to catch up.

[Kleidukos]

What's the new state-of-the-world regarding where to contribute patches? IIUC, https://gitlab.haskell.org/ghc/haddock is the place where GHC related changes go (and where Hi Haddock landed), but is there any plan for getting changes from here to there (or vice versa)?

@harpocrates Non-GHC patches arrive here, which gets synchronised with ghc-head on the GHC mirror by the GHC team.

Now that Hi Haddock has landed (and the massive cloud of potential merge conflicts is no longer hanging overhead), I wouldn't mind trying to resurrect/fix some older PRs and issues, but I realize I'm not sure where those patches even go anymore.

Fortunately there is a simple heuristic: If the patch is targeting ghc-head or is needed for a GHC MR, chances are that the patch is more appropriately moved to the GHC Gitlab. :)

[harpocrates]

@harpocrates Non-GHC patches arrive here, which gets synchronised with ghc-head on the GHC mirror by the GHC team

Does this mean the GHC team manages cutting new branches for this repo, updating the default branch, etc.? They presumably need to know which branches to merge into ghc-head, and contributors here need to know when they should no longer open/merge PRs against a certain branch (because GHC team won't be merging that branch back into ghc-head anymore).

[Kleidukos]

@javiertoledo What I have in mind is quite specifically the same experience provided by ExDoc in Elixir, which gives us the possibility of a Guides section along with the generated function documentation.

https://hexdocs.pm/phoenix/overview.html :)

[Kleidukos]

@harpocrates This new scheme is relatively recent so I don't have the experience of time to tell you if it's viable or not.

However I have the strong feeling that we should definitely steer towards a model where a main branch is developed, from which the ghc-head branch would sync. What do you think?

[NickSeagull]

@javiertoledo What I have in mind is quite specifically the same experience provided by ExDoc in Elixir, which gives us the possibility of a Guides section along with the generated function documentation.

This can be relatively easily done by integrating haddock generation into tintin or viceversa, it could be very nice :)

One thing that I also had in mind for a long time is to have interactive exercises built into it like scala-exercises. Maybe for this we can use something like hint to evaluate/assert the exercises results @Kleidukos

[Kleidukos]

@NickSeagull I think it would be wonderful to have haddock natively generate HTML bundles with Tintin included. :)

Would you be open to have a call at some point so that we can lay out the ideas and determine where to go from here?

Regarding the usage of Hint, you should partner with @gelisam to make it happen, that would indeed be incredible to have!

[gelisam]

Happy to help! I would also like to point out that School of Haskell had interactive Haskell snippets and has been open-sourced, so it might be easier to reuse that than to start from scratch.

[michaelpj]

One direction that might be interesting to investigate is moving further towards nicer multi-package support. Now that cabal v2 is pretty well-established, I think it's natural for people to expect Haddock to be easy to use on multiple packages, whereas it's mostly geared towards a single package.

There's been some work in the direction of multi-package support, but I think it could be more seamless and what would be really nice would be a nice multi-package index, like Hackage has.

The top-level goal here might be for cabal v2-haddock to produce a beautiful, properly hyperlinked, multi-package documentation site for all the packages in the project.

---

All that aside, I think lowering the barrier to new contributors is probably the thing that will bear the most fruit in the end, but it's hard to get away from the GHC/Haddock fraternal friction...