US 7-019 5.2 [lex.phases] Convert pp-tokens to tokens before phase 7 #593
Description
As we are updating the phases of translation in C++26, take advantage to split phase to convert preproccessor tokens to tokens as a distinct phase before translating the TU. This lines up better with modules and header units being an input into the phase after tokenization is complete.
Proposed change:
Assuming phase 6 merges into phase 5 above, create a new phase 6 (preserving phase numbering above) to perform the pp-token to token conversion, prior to phase 7 starting with a full set of (converted) tokens.