Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/doc: improve headings, lists, and links #51082

Open
rsc opened this issue Feb 8, 2022 · 119 comments
Open

go/doc: improve headings, lists, and links #51082

rsc opened this issue Feb 8, 2022 · 119 comments

Comments

@rsc
Copy link
Contributor

@rsc rsc commented Feb 8, 2022

This proposal improves support for headings, lists, and links in Go doc comments,
while remaining backwards compatible with existing comments.
It includes a new package, go/doc/comment, exposing a parsed syntax tree for
doc comments, and it includes changes to go/printer and therefore gofmt
to format doc comments in a standard way.

For example, existing lists reformat from the display on the left to the one on the right:

URL links can be rewritten to change the display on the left to the one on the right:

And package doc comments (and others) can be rewritten to link to specific symbols:

The full design doc is at https://github.com/golang/proposal/blob/master/design/51082-godocfmt.md.

@gopherbot gopherbot added this to the Proposal milestone Feb 8, 2022
@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384263 mentions this issue: go/doc/comment: new package

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384257 mentions this issue: all: fix various doc comment formatting nits

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384267 mentions this issue: cmd/go: gofmt alldocs.go

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384258 mentions this issue: all: fix TODO comment hanging indents

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384265 mentions this issue: go/doc: use go/doc/comment

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384266 mentions this issue: cmd/doc: use new go/doc APIs

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384268 mentions this issue: all: gofmt main repo

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384264 mentions this issue: go/printer: format doc comments

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384259 mentions this issue: all: remove trailing blank doc comment lines

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384260 mentions this issue: all: separate doc comment from //go: directives

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384274 mentions this issue: internal/pkgdoc: update go/doc API usage for links

@rsc rsc added this to Incoming in Proposals Feb 8, 2022
@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 8, 2022

The top comment has been filled in now.
See #51082 (comment).

@jimmyfrasche
Copy link
Member

@jimmyfrasche jimmyfrasche commented Feb 8, 2022

This solves 90% of my problems working with docstrings and makes the remaining 10% trivial. I am ecstatic!

I'd be perfectly happy to have this accepted as-is but I do have a couple minor thoughts:

I'd s/Text/Inline/g and then s/Plain/Text/g to align with html. That could just be familiarity but it seems clearer to me.

Allow subheadings with ##, ###, and so on. I've seen a few packages since the original discussion that could use at least one extra level of indentation for grouping subsections (I did not think to record the packages in question, sorry). This could be added later, but if so it may be prudent to accept ## x as a heading now so it doesn't change from a paragraph to a heading later. Though at that point you're most of the way to an implementation.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 8, 2022

For now I think we should keep things simple and not increase scope past the previous discussion. There would really be no problem with having ## render as a paragraph during a later transition. Single # is rendering as a paragraph during this one.

@kortschak
Copy link
Contributor

@kortschak kortschak commented Feb 9, 2022

In numbered point 5 in the goals and non-goals, the absence of in-line image support is noted for a couple of languages and support by one, but no mention of Python (a much more relevant language than for example Perl). Here it does exist in some projects https://numpy.org/doc/stable/reference/generated/numpy.fft.fft.html#numpy.fft.fft and in general, Python doc strings may be valid reStructuredText which allows embedding of images.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

Python supports images in their docs, noted.
All the problems listed for images still exist though, and images remain out of scope for this change.

@ainar-g
Copy link
Contributor

@ainar-g ainar-g commented Feb 9, 2022

I am extremely excited about this! A couple of questions / suggestions.

  1. Currently, the pkg.go.dev renderer automatically converts text like RFC NNNN into a link to the corresponding RFC. From what I can tell, that's a feature of the website's renderer as opposed to one of godoc itself. Will that behaviour of the renderer remain intact?

  2. The point about ASCII double quotes doesn't mention if they are reformatted in the preformatted text. I think, it is worth explicitly mentioning that they won't be replaced with the Unicode ones in that position and adding test cases for that if there isn't one.

    (This may seem obvious, but anyone who ever tried editing a Shell script with the macOS TextEdit program will know that it sometimes isn't, heh.)

  3. In the numbered list section, it's currently not clear, what happens when N is above 9. I assume that it will become “␠NN.␠”?

  4. In the heading section, perhaps there should be an example of a line that begins with a space and then a #. Markdown allows a certain amount of spaces before the heading character, and if the new format doesn't allow that then it's a differrence worth mentioning explicitly, I think.

Again, really looking forward to this in any case.

@kortschak
Copy link
Contributor

@kortschak kortschak commented Feb 9, 2022

Overall this looks good, but I want to restate my concern about the accessibility issues I raised with the approach to marking links.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

Thanks for the questions, @ainar-g.

  1. I dont know for sure, but I think the idea is to drop the special case RFC NNNN support on pkg.go.dev and let people write links. The current support does not handle references to subsections well, for example. It was probably a mistake for pkg.go.dev to diverge this way. There is a bit more prior discussion at #48305 (comment).

  2. Doubled ASCII single quotes (`` and '') are reformatted. This is called out explicitly:

    The ASCII double-single-quote forms that have always been defined to render as “ and ” are replaced with those.

  3. If N is 10, then “␠N.␠” would be “␠10.␠”. This seems clearer than writing “␠NN.␠” - wouldn't that be “␠1010.␠”?

  4. Thanks for the suggestion. Added that as an example.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

@kortschak Noted re link syntax. I don't have anything more to add beyond #48305 (comment).

@kortschak
Copy link
Contributor

@kortschak kortschak commented Feb 9, 2022

It's worth noting that I did follow that up OOB with a detailed response, but did not receive a reply.

@thepudds
Copy link

@thepudds thepudds commented Feb 9, 2022

Overall, this is exciting to see.

A few quick questions about linking to headers and the relationship to the base URL.

  1. For linking to headers in the same package, would something like the following work?
See [Some Heading] for more information.

[Some Heading]: #some-heading
  1. Would that rely on knowing the default header anchor ID algorithm (or doing a copy/paste or similar)? From the API, it looks to be proposed as:
// The default anchor ID is constructed by dropping all characters
// except A-Z, a-z, 0-9, space, and tab, converting to lower case,
// splitting into space-or-tab-separated fields, and then rejoining
// the fields using hyphens. For example, if the heading text is
// “Everybody Says Don't”, the default ID is “everybody-says-dont”.
  1. Would you be able to link to headers in other packages without needing to know the base URL, such as something like:
[Another Heading]: /some/import/path#another-heading
  1. Anchor IDs removing non-ASCII letters certainly simplifies things, but if linking to something like this (one of the examples from #7349) is contemplated for the future, simply changing the anchor ID algorithm might then break other links that are mixed ASCII/non-ASCII headers?
# これは、ヘッダです

In any event, I think the ability to link to headers would be extremely useful, especially for complex packages.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals Feb 9, 2022
aalexand added a commit to google/pprof that referenced this issue Apr 12, 2022
golang/go#51082 changed how gofmt handles some
comments, so we need to slightly reformat them to avoid `gofmt -s -d .`
errors.
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs.)

For golang/go#51082.

Change-Id: I0cc2e6cac2e4ed975770aea78cc2f39c13d6f874
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399357
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs.)

For golang/go#51082.

Change-Id: Ife11502fe1e59a04d53dba9edccd3043e57f9ae8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399358
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs.)

For golang/go#51082.

Change-Id: I77809c80838cc8f4cdf43c3c42685e2fc695328a
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399359
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs.)

For golang/go#51082.

Change-Id: I229b90365998244cfa858ae678382f6089b1cdb9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399360
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs.)

For golang/go#51082.

Change-Id: I5e11f2946001b9b36b7bd3af4d42da9dcea7494f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399361
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs.)

For golang/go#51082.

Change-Id: I58a534f3e518dad2d3a867f81b08a551a76bd423
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399362
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

(There are so many files in x/tools I am breaking up the
gofmt'ing into multiple CLs. This is the leftovers.)

For golang/go#51082.

Change-Id: Id9d440cde9de7093d2ffe06cbaa7098993823d6b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/399363
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/example that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Ic98f647623f234cf5d36309c6204683e151820d7
Reviewed-on: https://go-review.googlesource.com/c/example/+/399596
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/benchmarks that referenced this issue Apr 12, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Ia404739fe81daca82a209ff82fc29ac3a064cea2
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/399595
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit to golang/mobile that referenced this issue Apr 14, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I9b4c287e2d25aa108adfa9fe2f972c8fd3d68fe1
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/399597
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
gopherbot pushed a commit to golang/exp that referenced this issue Apr 14, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Iac828c845b6d7ae0eab93fcf007f3ef8e16c8ed7
Reviewed-on: https://go-review.googlesource.com/c/exp/+/399614
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
@shuLhan
Copy link
Contributor

@shuLhan shuLhan commented Apr 24, 2022

@rsc

I think there is a bug with list formatting when comment start and end with /**/ in the current tip branch, or maybe a missing exception in the proposal since the spec still allow comment start/end with /**/.

Test code

See https://go.dev/play/p/WfIdp2WQphD?v=gotip

Expected result

/*
testCommentSlashStart method with header and list

# Header

Here is the list,

  - a
  - b
*/

Got result

/*
testCommentSlashStart method with header and list

# Header

Here is the list,

* a
* b
*/
func testCommentSlashStart() {}

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 25, 2022

@shuLhan Would you mind opening a new issue for that specific problem? Thanks.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 25, 2022

I believe that all changes described here have been submitted, so closing this issue. Please comment if you disagree.

gopherbot pushed a commit to golang/sys that referenced this issue Apr 29, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: I9a1c4b671c06820a1c383ee515f7884965fefa54
Reviewed-on: https://go-review.googlesource.com/c/sys/+/399602
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 2, 2022

Still one pending CL: the doc comment for the comment package: https://go.dev/cl/397287.

@bcmills
Copy link
Member

@bcmills bcmills commented May 9, 2022

The changes for this proposal appear to have introduced a crash in go/doc (#52783).

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented May 18, 2022

This is only waiting documentation now, right?

@gopherbot
Copy link

@gopherbot gopherbot commented May 18, 2022

Change https://go.dev/cl/407137 mentions this issue: all: gofmt main repo

gopherbot pushed a commit that referenced this issue May 19, 2022
Excluding vendor and testdata.
CL 384268 already reformatted most, but these slipped past.

The struct in the doc comment in debug/dwarf/type.go
was fixed up by hand to indent the first and last lines as well.

For #51082.

Change-Id: Iad020f83aafd671ff58238fe491907e85923d0c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/407137
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Accepted
Development

No branches or pull requests