go/doc: improve headings, lists, and links

[rsc]

This proposal improves support for headings, lists, and links in Go doc comments,
while remaining backwards compatible with existing comments.
It includes a new package, go/doc/comment, exposing a parsed syntax tree for
doc comments, and it includes changes to go/printer and therefore gofmt
to format doc comments in a standard way.

For example, existing lists reformat from the display on the left to the one on the right:

URL links can be rewritten to change the display on the left to the one on the right:

And package doc comments (and others) can be rewritten to link to specific symbols:

The full design doc is at https://github.com/golang/proposal/blob/master/design/51082-godocfmt.md.

[gopherbot]

Change https://go.dev/cl/384263 mentions this issue: go/doc/comment: new package

[gopherbot]

Change https://go.dev/cl/384257 mentions this issue: all: fix various doc comment formatting nits

[gopherbot]

Change https://go.dev/cl/384267 mentions this issue: cmd/go: gofmt alldocs.go

[gopherbot]

Change https://go.dev/cl/384258 mentions this issue: all: fix TODO comment hanging indents

[gopherbot]

Change https://go.dev/cl/384265 mentions this issue: go/doc: use go/doc/comment

[gopherbot]

Change https://go.dev/cl/384266 mentions this issue: cmd/doc: use new go/doc APIs

[gopherbot]

Change https://go.dev/cl/384268 mentions this issue: all: gofmt main repo

[gopherbot]

Change https://go.dev/cl/384264 mentions this issue: go/printer: format doc comments

[gopherbot]

Change https://go.dev/cl/384259 mentions this issue: all: remove trailing blank doc comment lines

[gopherbot]

Change https://go.dev/cl/384260 mentions this issue: all: separate doc comment from //go: directives

[gopherbot]

Change https://go.dev/cl/384274 mentions this issue: internal/pkgdoc: update go/doc API usage for links

[rsc]

The top comment has been filled in now.
See #51082 (comment).

[jimmyfrasche]

This solves 90% of my problems working with docstrings and makes the remaining 10% trivial. I am ecstatic!

I'd be perfectly happy to have this accepted as-is but I do have a couple minor thoughts:

I'd s/Text/Inline/g and then s/Plain/Text/g to align with html. That could just be familiarity but it seems clearer to me.

Allow subheadings with ##, ###, and so on. I've seen a few packages since the original discussion that could use at least one extra level of indentation for grouping subsections (I did not think to record the packages in question, sorry). This could be added later, but if so it may be prudent to accept ## x as a heading now so it doesn't change from a paragraph to a heading later. Though at that point you're most of the way to an implementation.