go/ast: doc insufficient for inserting comments #18593

Open
willfaught opened this Issue Jan 10, 2017 · 2 comments

Projects

None yet

2 participants

@willfaught
willfaught commented Jan 10, 2017 edited

I don't see a way to insert documentation comments for file declarations such that they appear just above their decls after printing with go/printer.Fprint or go/format.Node (or both, as explained below). Is it impossible? If so, it should be noted in the doc; if not, it should be noted how to do it, since a straightforward mutation of the comment fields and/or creation/mutation/saving of a CommentMap for File.Comments doesn't work. Also note the related problems pointed out in the code below.

My scenario is writing a tool that inserts // TODO doc comments for undocumented exported decls.

var fs = token.NewFileSet()
var path = ... // path to .go file with exported decls with no doc
var af, _ = parser.ParseFile(fs, path, nil, parser.DeclarationErrors|parser.ParseComments)
var cm = ast.NewCommentMap(fs, af, af.Comments)

if cm == nil {
	cm = ast.CommentMap{} // problem: doc doesn't say cm can be nil! bug?
}

for _, d := range af.Decls {
	switch d := d.(type) {
	case *ast.FuncDecl:
		d.Doc = ... // insert doc

	case *ast.GenDecl:
		for _, s := range d.Specs {
			switch s := s.(type) {
			case *ast.TypeSpec:
				s.Doc = ... // insert doc

			case *ast.ValueSpec:
				s.Doc = ... // insert doc
			}
		}
	}
}

af.Comments = cm.Comments()

var b bytes.Buffer

format.Node(&b, fs, af) // problem: doesn't format new func doc comments! bug?

// func doc comments appear above their funcs correctly but replace blank line that was there before

bs, _ := format.Source(b.Bytes()) // format again for func doc comments

// var/const decl doc comments are put at the top of the file in one bunch
@griesemer griesemer was assigned by bradfitz Jan 10, 2017
@griesemer
Contributor
griesemer commented Jan 10, 2017 edited

@willfaught It's possible but it's complicated. In fact it's a major reason for an eventual rewrite of the AST (a more modern and simplified version is already in use in cmd/compile/internal/syntax).

Not sure this helps a lot for the moment but there are two mechanisms in place:

  1. "Doc string" comments are attached to nodes directly via the respective Node's Doc fields.
  2. All comments (incl. Doc strings) are collected in a sorted (in source order) list of comments found in the File node. When source code is printed, comments are interspersed with the rest of the code based on position information.

Thus, if you are inserting comments, you need to insert them in the right place in the general comment list in the ast.File.Comments list, and you need to give them the correct position relative to the nodes where they are inserted. Unfortunately that's a bit of a trial and error process.

(go/ast is one of the very earliest Go packages in existence, and the design decision made for handling comments - while it works for gofmt - is unfortunate when it comes to manipulating comments. Mea culpa.)

@griesemer
Contributor

Related: #14629

@griesemer griesemer modified the milestone: Unplanned, Go1.9 Jan 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment