don't keep trailing newline inside line comment txt in AST #48
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is in a similar vein as #46:
protoc
considers the trailing newline for line comments as part of the comment text in its source code info.But that is confusing for users of the AST because the AST ostensibly provides other accessors for querying whitespace between elements:
LeadingWhitespace()
method onNodeInfo
andComment
. By keeping the newline in the comment text, that means that code has to look at both theLeadingWhitespace()
and also inspect the preceding comment'sRawText()
to determine if there was a newline between elements.That's icky and is the source of some confusing and brittle code in the formatter. We can simplify by keeping the trailing/separating newline out of the comment text, so it is instead part of leading whitespace.
However, we have to remember to then add the newline on the end when generating source code info, since
protoc
includes it.While making this change, I refactored a decent bit inside the AST package's file info. It now uses generics to model a sequence of items or tokens. An item is a token or a comment. One neat thing this provides is a simple API for navigating the parsed file as a stream of tokens (or a stream of all items, for consumers that want to inspect the comments, too), instead of needing to use a recursive approach over the tree structure.