[lex.phase] Move normative handling of comments] #8594
Conversation
The specification of phase 3 of translation says too much about how things are done rather than what is done. Per suggestions on PR cplusplus#8414, move the precise handling of comments to [lex.comment]. As this move relies on the specificatin for whitespace, move the definition of whitespace from [lex.pptoken] to [lex.comment] too.]
hubert-reinterpretcast
left a comment
hubert-reinterpretcast
left a comment
There was a problem hiding this comment.
With the proposed changes, the subclause title for [lex.comment] should probably be expanded.
| Each comment is replaced by one \unicode{0020}{space} character; | ||
| new-line characters are retained. |
There was a problem hiding this comment.
There should be no retention of new-line characters within C-style comments. Additionally, retention of new-line characters following the termination of a // comment does not require mention. The termination point of a comment of either form is made clear above. My understanding of the corresponding sentence in the original text is that it is meant to be bound to the next sentence in the original text (not the previous sentence as this rendition assumes).
| Each comment is replaced by one \unicode{0020}{space} character; | |
| new-line characters are retained. | |
| Each comment is replaced by one \unicode{0020}{space} character. |
| Whitespace can appear within a preprocessing token only as part of | ||
| a \grammarterm{header-name} or | ||
| between the quotation characters in a character literal or string literal. |
There was a problem hiding this comment.
My hope is that we would strike this from the note in the future. I think we want to move to a model, like with "UCNs inside raw strings", where "whitespace" (as we define it) does not appear except between preprocessing tokens.
| Whether each nonempty sequence of whitespace characters other than new-line | ||
| is retained or replaced by one \unicode{0020}{space} character is unspecified. |
There was a problem hiding this comment.
For me, this is too far out of context to be worded this way. I would like a statement that such replacement may be done by an implementation before stating that whether it is done (for any specific instance) is unspecified.
Additionally, with this presentation, it is unclear to me when this replacement occurs (which affects whether comments act as separate U+0020 characters and whether both ends of a phase 1 line splice can remain separate U+0020 characters that replaced other whitespace characters). The behaviour can be observed if whitespace is retained by an implementation when forming a header-name token from < h-pp-tokens >.
3 participants
The specification of phase 3 of translation says too much about how things are done rather than what is done. Per suggestions on PR #8414, move the precise handling of comments to [lex.comment]. As this move relies on the specification for whitespace, move the definition of whitespace from [lex.pptoken] to [lex.comment] too.
Note that this PR would obviate #8416, which turns the text moved from [lex.pptoken] into a comment, precisely as the moved text in this PR.