cmd/go: add go doc -json

[paulvollmer]

To build better language support for editors and ide's it could be helpful to have the docs as json.

For example https://golang.org/pkg/strings.json to get the strings package documentation.
Or via terminal by a flag like go doc fmt -format json

Then the docs can be used to generate snippets based on the documentation.

[joshuabezaleel]

Hi @odeke-em , do you have any recommendation on where should I start looking if I am interested in this issue?
Thank you very much! 🙂

[ianlancetaylor]

@joshuabezaleel Note that this is a proposal. It may be accepted, but so far it has not been accepted. You are welcome to look into the issue, but if the proposal is not accepted your work will not be accepted either. Thanks.

[rsc]

There is a go/doc package that lets you get package docs in Go data structures. It seems like that should be enough.

Failing that, gopls clearly has some way to give docs to the editor for helper popups. What does it use? Also, more generally, gopls is the way to "better language support for editors and ide's". So if more is needed, it should be in gopls, not cmd/go or a URL on a web site.

/cc @stamblerre

[stamblerre]

LSP only supports plaintext or markdown for hover (see https://microsoft.github.io/language-server-protocol/specifications/specification-3-15/#textDocument_hover), but gopls has an experimental structured hover format that uses JSON. It could be exposed on the command line via the gopls definition command, which already has a -json flag. The code for this command can be found here.

[rsc]

@stamblerre, how does gopls get the docs in a data structure? Does it just use go/doc for that parsing?

@paulvollmer, have you tried using go/doc? Is there anything missing from what it does?

[paulvollmer]

@rsc i'm using go doc to get a fast way to the docs.
what i'm thinking about is using go doc also in a machine readable format.
for example go doc net/http -json to get the docs of the net/http package as a json output.

But maybe there is already a tool doing that. how are the html pages of golang.org/pkg generated?

[stamblerre]

@rsc: gopls borrows a lot of code from go/doc to convert the documentation to markdown (comment.go) and uses go/doc directly to get the synopsis. The rest of the data structure is constructed using type information.

[rsc]

go/doc already defines a data structure, doc.Package.
And I believe 'go doc' creates one of those using go/doc and then formats it to text.
We could say that -json dumps the doc.Package itself to JSON.
That would let us reuse the API we have already defined in go/doc instead of having to define something new.

@paulvollmer, if you got a doc.Package from go doc -json would that work for your use case? (We may need to create some parallel structures with the go/token and go/ast pieces properly converted, which we could do.) At the least that seems like a plausible path forward and is consistent with the other various -json flags in cmd/go.

Thoughts?

[paulvollmer]

@rsc yes, i think this should provide the machine readable data i had in mind. let me know if i can help working on that task.