Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/doc: improve headings, lists, and links #51082

Open
rsc opened this issue Feb 8, 2022 · 134 comments
Open

go/doc: improve headings, lists, and links #51082

rsc opened this issue Feb 8, 2022 · 134 comments

Comments

@rsc
Copy link
Contributor

@rsc rsc commented Feb 8, 2022

This proposal improves support for headings, lists, and links in Go doc comments,
while remaining backwards compatible with existing comments.
It includes a new package, go/doc/comment, exposing a parsed syntax tree for
doc comments, and it includes changes to go/printer and therefore gofmt
to format doc comments in a standard way.

For example, existing lists reformat from the display on the left to the one on the right:

URL links can be rewritten to change the display on the left to the one on the right:

And package doc comments (and others) can be rewritten to link to specific symbols:

The full design doc is at https://github.com/golang/proposal/blob/master/design/51082-godocfmt.md.

@gopherbot gopherbot added this to the Proposal milestone Feb 8, 2022
@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384263 mentions this issue: go/doc/comment: new package

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384257 mentions this issue: all: fix various doc comment formatting nits

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384267 mentions this issue: cmd/go: gofmt alldocs.go

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384258 mentions this issue: all: fix TODO comment hanging indents

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384265 mentions this issue: go/doc: use go/doc/comment

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384266 mentions this issue: cmd/doc: use new go/doc APIs

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384268 mentions this issue: all: gofmt main repo

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384264 mentions this issue: go/printer: format doc comments

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384259 mentions this issue: all: remove trailing blank doc comment lines

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384260 mentions this issue: all: separate doc comment from //go: directives

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 8, 2022

Change https://go.dev/cl/384274 mentions this issue: internal/pkgdoc: update go/doc API usage for links

@rsc rsc added this to Incoming in Proposals Feb 8, 2022
@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 8, 2022

The top comment has been filled in now.
See #51082 (comment).

@jimmyfrasche
Copy link
Member

@jimmyfrasche jimmyfrasche commented Feb 8, 2022

This solves 90% of my problems working with docstrings and makes the remaining 10% trivial. I am ecstatic!

I'd be perfectly happy to have this accepted as-is but I do have a couple minor thoughts:

I'd s/Text/Inline/g and then s/Plain/Text/g to align with html. That could just be familiarity but it seems clearer to me.

Allow subheadings with ##, ###, and so on. I've seen a few packages since the original discussion that could use at least one extra level of indentation for grouping subsections (I did not think to record the packages in question, sorry). This could be added later, but if so it may be prudent to accept ## x as a heading now so it doesn't change from a paragraph to a heading later. Though at that point you're most of the way to an implementation.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 8, 2022

For now I think we should keep things simple and not increase scope past the previous discussion. There would really be no problem with having ## render as a paragraph during a later transition. Single # is rendering as a paragraph during this one.

@kortschak
Copy link
Contributor

@kortschak kortschak commented Feb 9, 2022

In numbered point 5 in the goals and non-goals, the absence of in-line image support is noted for a couple of languages and support by one, but no mention of Python (a much more relevant language than for example Perl). Here it does exist in some projects https://numpy.org/doc/stable/reference/generated/numpy.fft.fft.html#numpy.fft.fft and in general, Python doc strings may be valid reStructuredText which allows embedding of images.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

Python supports images in their docs, noted.
All the problems listed for images still exist though, and images remain out of scope for this change.

@ainar-g
Copy link
Contributor

@ainar-g ainar-g commented Feb 9, 2022

I am extremely excited about this! A couple of questions / suggestions.

  1. Currently, the pkg.go.dev renderer automatically converts text like RFC NNNN into a link to the corresponding RFC. From what I can tell, that's a feature of the website's renderer as opposed to one of godoc itself. Will that behaviour of the renderer remain intact?

  2. The point about ASCII double quotes doesn't mention if they are reformatted in the preformatted text. I think, it is worth explicitly mentioning that they won't be replaced with the Unicode ones in that position and adding test cases for that if there isn't one.

    (This may seem obvious, but anyone who ever tried editing a Shell script with the macOS TextEdit program will know that it sometimes isn't, heh.)

  3. In the numbered list section, it's currently not clear, what happens when N is above 9. I assume that it will become “␠NN.␠”?

  4. In the heading section, perhaps there should be an example of a line that begins with a space and then a #. Markdown allows a certain amount of spaces before the heading character, and if the new format doesn't allow that then it's a differrence worth mentioning explicitly, I think.

Again, really looking forward to this in any case.

@kortschak
Copy link
Contributor

@kortschak kortschak commented Feb 9, 2022

Overall this looks good, but I want to restate my concern about the accessibility issues I raised with the approach to marking links.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

Thanks for the questions, @ainar-g.

  1. I dont know for sure, but I think the idea is to drop the special case RFC NNNN support on pkg.go.dev and let people write links. The current support does not handle references to subsections well, for example. It was probably a mistake for pkg.go.dev to diverge this way. There is a bit more prior discussion at #48305 (comment).

  2. Doubled ASCII single quotes (`` and '') are reformatted. This is called out explicitly:

    The ASCII double-single-quote forms that have always been defined to render as “ and ” are replaced with those.

  3. If N is 10, then “␠N.␠” would be “␠10.␠”. This seems clearer than writing “␠NN.␠” - wouldn't that be “␠1010.␠”?

  4. Thanks for the suggestion. Added that as an example.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

@kortschak Noted re link syntax. I don't have anything more to add beyond #48305 (comment).

@kortschak
Copy link
Contributor

@kortschak kortschak commented Feb 9, 2022

It's worth noting that I did follow that up OOB with a detailed response, but did not receive a reply.

@thepudds
Copy link

@thepudds thepudds commented Feb 9, 2022

Overall, this is exciting to see.

A few quick questions about linking to headers and the relationship to the base URL.

  1. For linking to headers in the same package, would something like the following work?
See [Some Heading] for more information.

[Some Heading]: #some-heading
  1. Would that rely on knowing the default header anchor ID algorithm (or doing a copy/paste or similar)? From the API, it looks to be proposed as:
// The default anchor ID is constructed by dropping all characters
// except A-Z, a-z, 0-9, space, and tab, converting to lower case,
// splitting into space-or-tab-separated fields, and then rejoining
// the fields using hyphens. For example, if the heading text is
// “Everybody Says Don't”, the default ID is “everybody-says-dont”.
  1. Would you be able to link to headers in other packages without needing to know the base URL, such as something like:
[Another Heading]: /some/import/path#another-heading
  1. Anchor IDs removing non-ASCII letters certainly simplifies things, but if linking to something like this (one of the examples from #7349) is contemplated for the future, simply changing the anchor ID algorithm might then break other links that are mixed ASCII/non-ASCII headers?
# これは、ヘッダです

In any event, I think the ability to link to headers would be extremely useful, especially for complex packages.

@rsc
Copy link
Contributor Author

@rsc rsc commented Feb 9, 2022

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals Feb 9, 2022
@rsc
Copy link
Contributor Author

@rsc rsc commented Jun 1, 2022

Yes, this is waiting on docs, which was in turn blocked on my analysis of the rewrites on Google's code base, so I know what to call out in the docs. Just finished the analysis this morning. I'm aiming to have the docs sent out this week.

@rsc
Copy link
Contributor Author

@rsc rsc commented Jun 2, 2022

In addition to docs, this is waiting on integration with pkgsite and x/website/cmd/golangorg.
@jba has done the former but won't submit it until beta1 is available.
I can do x/website.

Only the docs and some minor gofmt fixes I've discovered block the beta.
Once they are submitted we can add okay-after-beta1.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 4, 2022

Change https://go.dev/cl/410358 mentions this issue: go/doc/comment: add doc comment

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 4, 2022

Change https://go.dev/cl/410359 mentions this issue: go/doc/comment: do not turn ``` into “`

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 4, 2022

Change https://go.dev/cl/410394 mentions this issue: _content/doc: document doc comments at go.dev/doc/comment

gopherbot pushed a commit that referenced this issue Jun 6, 2022
A CL in the website repo will add go.dev/doc/comment.

One of the final steps for #51082.

Change-Id: I419b4f6dbb424a8a93a8d09db30f7321af9ae976
Reviewed-on: https://go-review.googlesource.com/c/go/+/410358
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit that referenced this issue Jun 6, 2022
``` is Markdown, not Go doc comment, but some small fraction of users get confused.

In a set of 55M Go doc comments drawn from the latest version of
all public Go modules known to the module proxy in spring 2020,
the current Go 1.19 gofmt reformats about 1.57M of them.
Out of those 1.57M comments, 8k of them (about 0.5%) contain ```.

Instead of rewriting ``` to “`, leave it alone.

For #51082.

Change-Id: I1c8c88aac7ef75ec03e1a396b84ffe711c46f941
Reviewed-on: https://go-review.googlesource.com/c/go/+/410359
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
gopherbot pushed a commit to golang/website that referenced this issue Jun 7, 2022
This new page documents how to write doc comments,
which has always been missing, as well as explaining the
new Go 1.19 doc comment syntax and what to do if gofmt
reformats a comment in an unexpected way.

For golang/go#51082.
Fixes golang/go#44194.

Change-Id: I64baed6124e05cbe084c2df14f2a5611c6fccd5d
Reviewed-on: https://go-review.googlesource.com/c/website/+/410394
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Chris Hines <chris.cs.guy@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
@rsc rsc added the okay-after-beta1 label Jun 8, 2022
@rsc
Copy link
Contributor Author

@rsc rsc commented Jun 8, 2022

All that remains is updating x/website and pkgsite; marked okay-after-beta1.

@cherrymui cherrymui mentioned this issue Jun 9, 2022
288 tasks
@gopherbot gopherbot removed the okay-after-beta1 label Jun 10, 2022
@heschi
Copy link
Contributor

@heschi heschi commented Jun 15, 2022

Ping: beta1 is out.

@mattn
Copy link
Member

@mattn mattn commented Jun 18, 2022

I leave this comment with a my real experience. I created the following pull-request with a Go tip. It's my fault for using Go tip, so it's not someone else's problem to blame. Just my fault. It should be noted that this change will not be undone if you run the new gofmt and then run it again with the old gofmt.

charmbracelet/bubbletea#343

I am not against this pull-request because in my personal opinion this change is reasonable. This is an issue that everyone should be aware of.

dteh pushed a commit to dteh/fhttp that referenced this issue Jun 22, 2022
Gofmt to update doc comments to the new formatting.

For golang/go#51082.

Change-Id: Iae68a9cd600060577271575e893ecb23bed1e509
Reviewed-on: https://go-review.googlesource.com/c/net/+/399599
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 24, 2022

@jba what is the status of supporting the new doc comment code on pkgsite? That is the only work remaining for this 1.19 issue. Thanks.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 27, 2022

Change https://go.dev/cl/414694 mentions this issue: _content/doc/comment: add note about conflicting package imports

gopherbot pushed a commit to golang/website that referenced this issue Jun 28, 2022
When multiple source files import different packages to the same name,
that creates an ambiguity, disallowing the use of the short package name
in doc comment links.

For golang/go#51082.

Change-Id: Id46ca1e2db2c681e02ed34fc4dd57e3e43030994
Reviewed-on: https://go-review.googlesource.com/c/website/+/414694
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@jba
Copy link
Contributor

@jba jba commented Jun 28, 2022

The pkgsite code has been updated to use the new doc comment syntax. It has not been deployed yet.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 29, 2022

Change https://go.dev/cl/415174 mentions this issue: go/doc/comment: support links in lists in comments

@rsc
Copy link
Contributor Author

@rsc rsc commented Jun 29, 2022

I think I still need to update x/website as well.

gopherbot pushed a commit that referenced this issue Jun 30, 2022
The proposed (#51082) new go doc comment additions supports lists,
links, and doc links, but does not support links and doc links inside
lists, so implemnent this.

Fixes #53610

Change-Id: I4fa17d204fc9efa8f3633133e4a49e56cf1aa9bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/415174
Reviewed-by: Ben Golightly <golightly.ben@googlemail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Jun 30, 2022

Change https://go.dev/cl/415554 mentions this issue: go/doc/comment: support doc links to builtins

@thomaspeugeot
Copy link

@thomaspeugeot thomaspeugeot commented Jul 1, 2022

Naive question : is there a way to reference Note in a link ?

consider the following example

// NOTE(boiled egg cooking time experiment): following
// cooking time have been tested
//
// 6' : too hard
// 5' : fine
// 4' : too soft

// BoiledEggCookingTime is the desired cooking duration
//
// see [NOTE(boiled egg cooking time experiment)]
const BoiledEggCookingTime = time.Duration(5 * time.Minute)

would // see [NOTE(boiled egg cooking time experiment)] works ?
If "boiling egg cooking time experiment" is not typed correctly, would gopls spots the broken link ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Accepted
Status: No status
Development

No branches or pull requests