[ClangFormat] Space removed in `g< ::f<int>>()` except for `Standard: c++03`

[reventlov]

In all versions of C++ supported by ClangFormat, the alternative operator spelling ("digraph") <: is treated as [. However, ClangFormat will remove a space between < and : (except when Standard: c++03 is specified, or Standard: Auto is specified and no double-closing >> is found in the source file):

g< ::T>();       // before
g<::T>();        // after, compiler error
g [ : T > ( ) ;  // C++ equivalent tokenization

This causes problems when:

• the first template parameter must use an absolute namespace (e.g., the code is in f::g, namespace f::e exists, and the parameter is ::e::T), and
• the code is not compiled with -fno-digraphs (or its equivalent in other compilers)

As far as I am aware, this is the only place where the "digraph" alternative spellings can change the meaning of otherwise-valid C++ code (and I believe there is no such case in C, where the digraph operators originated).

There is a (moderately annoying) workaround of adding a dummy comment:

g</**/ ::T>();  // safe

[llvmbot]

@llvm/issue-subscribers-clang-format

Author: Benjamin (reventlov)

In all versions of C++ supported by ClangFormat, [the alternative operator spelling ("digraph") `<:` is treated as `[`](https://en.cppreference.com/w/cpp/language/operator_alternative). However, ClangFormat will remove a space between `<` and `:` (except when `Standard: c++03` is specified, or `Standard: Auto` is specified and no double-closing `>>` is found in the source file):

g< ::T>();       // before
g<::T>();        // after, compiler error
g [ : T > ( ) ;  // C++ equivalent tokenization

This causes problems when:

• the first template parameter must use an absolute namespace (e.g., the code is in f::g, namespace f::e exists, and the parameter is ::e::T), and
• the code is not compiled with -fno-digraphs (or its equivalent in other compilers)

As far as I am aware, this is the only place where the "digraph" alternative spellings can change the meaning of otherwise-valid C++ code (and I believe there is no such case in C, where the digraph operators originated).

There is a (moderately annoying) workaround of adding a dummy comment:

g</**/ ::T>();  // safe

[reventlov]

I believe the bug is on line 5482 of TokenAnnotator.cpp. The history is a little odd -- it looks like it was originally added for the specific case of < followed by ::, but for whatever reason the original author limited that fix to C++03. Subsequent revisions of the code have not fixed the issue.

It looks easy enough to add an unconditional space by removing the (Style.Standard < FormatStyle::LS_Cpp11) || clause, but this may annoy some users who compile with -fno-digraphs -- I don't know if a proper fix should include a new style option.

[owenca]

g<::T>(); // after, compiler error
g [ : T > ( ) ; // C++ equivalent tokenization

Which compiler would lex <: in g<::T>(); as [? Can you show an example with Compiler Explorer? It seems that clang doesn't do this with or without -fno-diagraphs.

[reventlov]

Ah, drat. We had a GCC 4.7 series compiler running in our build (with --std=c++11, because we're a cutting-edge embedded systems group) and I missed § 2.5 ¶ 3 in the C++11 standard, so it looks like I'm just wrong on this. All the newer compilers I can find on Compiler Explorer seem to tokenize <:: as < ::.