Allow Haddock to show more coverage information

[altaic]

It would be nice to have a --coverage option, which would generate statistics about haddock coverage, but not generate the actual documentation.

That is, for one file it might generate something like "20 3 76 9 1 0", indicating that out of 99 top level definitions, 20 are fully documented, 3 are partially documented, and 76 are undocumented; of those, there are 9 fully documented, 1 partially documented, and 0 undocumented, which are exported.

This would be great for visualizing documentation status/progress for large projects.

[Fuuzetsu]

Why such an ugly format and no locations? Are you hoping to use it to generate reports?

What does it mean to be partially documented? We report no such statistic at the moment.

[sol]

If you run haddock without specifying an output format it already prints coverage without generating documentation. So I think we already have this.

So maybe we just need an other option to specify the output format? @altaic Do you want it in a machine readable way?

[altaic]

@sol Whoops, sorry for the delayed response. I hadn't realized running haddock with no output format produces coverage information. That's exactly what I needed (machine readable isn't necessary), thanks.

@Fuuzetsu Regarding partial documentation, please see http://www.haskell.org/haddock/doc/html/ch03s02.html. For instance, perhaps a function is described but its arguments are not. I come across that particular case fairly often, and wish library authors would be mindful that such partial documentation is usually insufficient.

The reason I opened this ticket is in the hope that, should haddock provide an at-a-glance documentation metric, authors may better judge the state of their documentation. To that end, I am working on a tool to generate reports for documentation, test suites, coding style conformance, etc., such that one may easily survey a patch set (or pull request, or whatever). That way, larger projects can easily track and prioritize progress and regress in those areas, as well as set up rules to automatically execute actions for patch sets.

[Fuuzetsu]

It'll be more helpful if you can collect all cases you'd like to have included in the ‘partial documentation’ scenario. I think that what we do currently is fairly sufficient but clearly some people want more: I'm not against that but I need to know precisely what's wanted before I can start implementing things.

[altaic]

I believe the page I linked lists the documentation criteria:

• class methods
• constructors and record fields
• function arguments

I'm not sure how the module description counts in the current haddock coverage, but that's less significant to me than the above.

[Fuuzetsu]

I see. Even if I implement this (I think it shouldn't be too difficult, not sure about exact locations), it will not be on by default: there are many cases where documentation is either not necessary or doesn't make sense: argument to id :: a -> a or in fact most cases concerning type variables. Other cases include already-descriptive type names or functions: data N = NumberOfNodes { numberOfNodes :: Int } doesn't exactly warrant a further comment on numberOfNodes.

I don't want to dump huge amounts of text back at the user (possibly obfuscating other, more important things) because they'll just mentally skip it every time or switch it off all together: no one wants to be forced to annotate arguments to const :: a -> b -> a just to silence the documentation engine.

However if a project wants such amount of info then they can pass a flag to Haddock through Cabal or however they want to. Is this satisfactory?

Module description is already accounted for which is why you'll get 0/1 in an empty (bar module line) module.

[altaic]

That would be perfect, thank you.

[taruti]

How about using
taruti@d75a351
for this.

[Fuuzetsu]

It does not add any new info which is what the ticket is about, we already report what that backend does just in a different form, as standard operation.

[taruti]

Would you prefer for client code to parse the current Haddock progress output? That seems very fragile.

[Fuuzetsu]

Oh, forgot the OP wanted some better machine format output. In any case we need to first gather the requested stuff and we can worry how to output it later.

[ivanperez-keera]

Hi

I ended up here while trying to parse coverage info and integrate it in some regression tests.

This commit adds a --csv-coverage flag to haddock: keera-studios@d5d7529

Criticism is welcome (here or there, depending on how useful it is for the haddock project; I leave that to your judgment).