[ClangFormat] Future-proof Standard option, allow floating or pinning…

… to arbitrary lang version

Summary:
The historical context:
- clang-format was written when C++11 was current,
  and the main language-version concern was >> vs > > template-closers.
  An option was added to allow selection of the 03/11 behavior, or auto-detection.
- there was no option to choose simply "latest standard" so anyone who didn't
  ever want 03 behavior or auto-detection specified Cpp11.
- In r185149 this option started to affect lexer mode.
- no options were added to cover c++14, as parsing/formatting
  didn't change that much. The usage of Cpp11 to mean "latest" became
  codified e.g. in r206263
- c++17 added some new constructs. These were mostly backwards-compatible and so
  not used in old programs, so having no way to turn them off was OK.
- c++20 added some new constructs and keywords (e.g. co_*) that changed the
  meaning of existing programs, and people started to complain that
  the c++20 parsing couldn't be turned off.

New plan:
 - Default ('Auto') behavior remains unchanged: parse as latest, format
   template-closers based on input.
 - Add new 'Latest' option that more clearly expresses the intent "use
   modern features" that many projects have chosen for their .clang-format files.
 - Allow pinning to *any* language version, using the same name as clang -std:
   c++03, c++11, c++14 etc. These set precise lexer options, and any
   clang-format code depending on these can use a >= check.
 - For backwards compatibility, `Cpp11` is an alias for `Latest`, not `c++11`.
   This matches the historical documented semantics of this option.
   This spelling (and `Cpp03`) are deprecated.

Reviewers: klimek, modocache

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67541

llvm-svn: 373439

clang/docs/ClangFormatStyleOptions.rst

@@ -2288,22 +2288,38 @@ the configuration (without a prefix: ``Auto``).
      std::unique_ptr<int[]> foo() {} // Won't be affected
 
 **Standard** (``LanguageStandard``)
-  Format compatible with this standard, e.g. use ``A<A<int> >``
-  instead of ``A<A<int>>`` for ``LS_Cpp03``.
+  .. code-block:: c++
+
+     c++03:                                 latest:
+     vector<set<int> > x;           vs.     vector<set<int>> x;
+  Parse and format C++ constructs compatible with this standard.
 
   Possible values:
 
-  * ``LS_Cpp03`` (in configuration: ``Cpp03``)
+  * ``LS_Cpp03`` (in configuration: ``c++03``)
     Use C++03-compatible syntax.
 
-  * ``LS_Cpp11`` (in configuration: ``Cpp11``)
-    Use features of C++11, C++14 and C++1z (e.g. ``A<A<int>>`` instead of
-    ``A<A<int> >``).
+  * ``LS_Cpp11`` (in configuration: ``c++11``)
+    Use C++11-compatible syntax.
+
+  * ``LS_Cpp14`` (in configuration: ``c++14``)
+    Use C++14-compatible syntax.
+
+  * ``LS_Cpp17`` (in configuration: ``c++17``)
+    Use C++17-compatible syntax.
+
+  * ``LS_Cpp20`` (in configuration: ``c++20``)
+    Use C++20-compatible syntax.
+
+  * ``LS_Latest`` (in configuration: ``Latest``)
+    Parse and format using the latest supported language version.
 
   * ``LS_Auto`` (in configuration: ``Auto``)
     Automatic detection based on the input.
 
+  * ``Cpp03``: deprecated alias for ``c++03``
 
+  * ``Cpp11``: deprecated alias for ``Latest``
 
 **StatementMacros** (``std::vector<std::string>``)
   A vector of macros that should be interpreted as complete

clang/include/clang/Format/Format.h

@@ -1945,15 +1945,32 @@ struct FormatStyle {
   /// \endcode
   bool SpacesInSquareBrackets;
 
-  /// Supported language standards.
+  /// Supported language standards for parsing and formatting C++ constructs.
+  /// \code
+  ///    Latest:                                vector<set<int>>
+  ///    c++03                          vs.     vector<set<int> >
+  /// \endcode
+  ///
+  /// The correct way to spell a specific language version is e.g. ``c++11``.
+  /// The historical aliases ``Cpp03`` and ``Cpp11`` are deprecated.
   enum LanguageStandard {
-    /// Use C++03-compatible syntax.
+    /// c++03: Parse and format as C++03.
     LS_Cpp03,
-    /// Use features of C++11, C++14 and C++1z (e.g. ``A<A<int>>`` instead of
-    /// ``A<A<int> >``).
+    /// c++11: Parse and format as C++11.
     LS_Cpp11,
-    /// Automatic detection based on the input.
-    LS_Auto
+    /// c++14: Parse and format as C++14.
+    LS_Cpp14,
+    /// c++17: Parse and format as C++17.
+    LS_Cpp17,
+    /// c++20: Parse and format as C++20.
+    LS_Cpp20,
+    /// Latest: Parse and format using the latest supported language version.
+    /// 'Cpp11' is an alias for LS_Latest for historical reasons.
+    LS_Latest,
+
+    /// Auto: Automatic detection based on the input.
+    /// Parse using the latest language version. Format based on detected input.
+    LS_Auto,
   };
 
   /// Format compatible with this standard, e.g. use ``A<A<int> >``

clang/lib/Format/Format.cpp

@@ -67,10 +67,19 @@ template <> struct ScalarEnumerationTraits<FormatStyle::LanguageKind> {
 
 template <> struct ScalarEnumerationTraits<FormatStyle::LanguageStandard> {
   static void enumeration(IO &IO, FormatStyle::LanguageStandard &Value) {
-    IO.enumCase(Value, "Cpp03", FormatStyle::LS_Cpp03);
-    IO.enumCase(Value, "C++03", FormatStyle::LS_Cpp03);
-    IO.enumCase(Value, "Cpp11", FormatStyle::LS_Cpp11);
-    IO.enumCase(Value, "C++11", FormatStyle::LS_Cpp11);
+    IO.enumCase(Value, "c++03", FormatStyle::LS_Cpp03);
+    IO.enumCase(Value, "C++03", FormatStyle::LS_Cpp03); // Legacy alias
+    IO.enumCase(Value, "Cpp03", FormatStyle::LS_Cpp03); // Legacy alias
+
+    IO.enumCase(Value, "c++11", FormatStyle::LS_Cpp11);
+    IO.enumCase(Value, "C++11", FormatStyle::LS_Cpp11); // Legacy alias
+
+    IO.enumCase(Value, "c++14", FormatStyle::LS_Cpp14);
+    IO.enumCase(Value, "c++17", FormatStyle::LS_Cpp17);
+    IO.enumCase(Value, "c++20", FormatStyle::LS_Cpp20);
+
+    IO.enumCase(Value, "Latest", FormatStyle::LS_Latest);
+    IO.enumCase(Value, "Cpp11", FormatStyle::LS_Latest); // Legacy alias
     IO.enumCase(Value, "Auto", FormatStyle::LS_Auto);
   }
 };
@@ -756,7 +765,7 @@ FormatStyle getLLVMStyle(FormatStyle::LanguageKind Language) {
   LLVMStyle.ObjCSpaceBeforeProtocolList = true;
   LLVMStyle.PointerAlignment = FormatStyle::PAS_Right;
   LLVMStyle.SpacesBeforeTrailingComments = 1;
-  LLVMStyle.Standard = FormatStyle::LS_Cpp11;
+  LLVMStyle.Standard = FormatStyle::LS_Latest;
   LLVMStyle.UseTab = FormatStyle::UT_Never;
   LLVMStyle.ReflowComments = true;
   LLVMStyle.SpacesInParentheses = false;
@@ -1399,7 +1408,7 @@ class Formatter : public TokenAnalyzer {
                                    : FormatStyle::PAS_Right;
     if (Style.Standard == FormatStyle::LS_Auto)
       Style.Standard = hasCpp03IncompatibleFormat(AnnotatedLines)
-                           ? FormatStyle::LS_Cpp11
+                           ? FormatStyle::LS_Latest
                            : FormatStyle::LS_Cpp03;
     BinPackInconclusiveFunctions =
         HasBinPackedFunction || !HasOnePerLineFunction;
@@ -2455,14 +2464,18 @@ tooling::Replacements sortUsingDeclarations(const FormatStyle &Style,
 
 LangOptions getFormattingLangOpts(const FormatStyle &Style) {
   LangOptions LangOpts;
-  FormatStyle::LanguageStandard LexingStd =
-      Style.Standard == FormatStyle::LS_Auto ? FormatStyle::LS_Cpp11
-                                             : Style.Standard;
+
+  FormatStyle::LanguageStandard LexingStd = Style.Standard;
+  if (LexingStd == FormatStyle::LS_Auto)
+    LexingStd = FormatStyle::LS_Latest;
+  if (LexingStd == FormatStyle::LS_Latest)
+    LexingStd = FormatStyle::LS_Cpp20;
   LangOpts.CPlusPlus = 1;
   LangOpts.CPlusPlus11 = LexingStd >= FormatStyle::LS_Cpp11;
-  LangOpts.CPlusPlus14 = LexingStd >= FormatStyle::LS_Cpp11;
-  LangOpts.CPlusPlus17 = LexingStd >= FormatStyle::LS_Cpp11;
-  LangOpts.CPlusPlus2a = LexingStd >= FormatStyle::LS_Cpp11;
+  LangOpts.CPlusPlus14 = LexingStd >= FormatStyle::LS_Cpp14;
+  LangOpts.CPlusPlus17 = LexingStd >= FormatStyle::LS_Cpp17;
+  LangOpts.CPlusPlus2a = LexingStd >= FormatStyle::LS_Cpp20;
+
   LangOpts.LineComment = 1;
   bool AlternativeOperators = Style.isCpp();
   LangOpts.CXXOperatorNames = AlternativeOperators ? 1 : 0;

clang/unittests/Format/FormatTest.cpp

@@ -9388,6 +9388,19 @@ TEST_F(FormatTest, DoesNotTryToParseUDLiteralsInPreCpp11Code) {
             format("#define x(_a) printf(\"foo\"_a);", Style));
 }
 
+TEST_F(FormatTest, CppLexVersion) {
+  FormatStyle Style = getLLVMStyle();
+  // Formatting of x * y differs if x is a type.
+  verifyFormat("void foo() { MACRO(a * b); }", Style);
+  verifyFormat("void foo() { MACRO(int *b); }", Style);
+
+  // LLVM style uses latest lexer.
+  verifyFormat("void foo() { MACRO(char8_t *b); }", Style);
+  Style.Standard = FormatStyle::LS_Cpp17;
+  // But in c++17, char8_t isn't a keyword.
+  verifyFormat("void foo() { MACRO(char8_t * b); }", Style);
+}
+
 TEST_F(FormatTest, UnderstandsCpp1y) { verifyFormat("int bi{1'000'000};"); }
 
 TEST_F(FormatTest, BreakStringLiteralsBeforeUnbreakableTokenSequence) {
@@ -12305,11 +12318,18 @@ TEST_F(FormatTest, ParsesConfiguration) {
               FormatStyle::PAS_Middle);
 
   Style.Standard = FormatStyle::LS_Auto;
+  CHECK_PARSE("Standard: c++03", Standard, FormatStyle::LS_Cpp03);
+  CHECK_PARSE("Standard: c++11", Standard, FormatStyle::LS_Cpp11);
+  CHECK_PARSE("Standard: c++14", Standard, FormatStyle::LS_Cpp14);
+  CHECK_PARSE("Standard: c++17", Standard, FormatStyle::LS_Cpp17);
+  CHECK_PARSE("Standard: c++20", Standard, FormatStyle::LS_Cpp20);
+  CHECK_PARSE("Standard: Auto", Standard, FormatStyle::LS_Auto);
+  CHECK_PARSE("Standard: Latest", Standard, FormatStyle::LS_Latest);
+  // Legacy aliases:
   CHECK_PARSE("Standard: Cpp03", Standard, FormatStyle::LS_Cpp03);
-  CHECK_PARSE("Standard: Cpp11", Standard, FormatStyle::LS_Cpp11);
+  CHECK_PARSE("Standard: Cpp11", Standard, FormatStyle::LS_Latest);
   CHECK_PARSE("Standard: C++03", Standard, FormatStyle::LS_Cpp03);
   CHECK_PARSE("Standard: C++11", Standard, FormatStyle::LS_Cpp11);
-  CHECK_PARSE("Standard: Auto", Standard, FormatStyle::LS_Auto);
 
   Style.BreakBeforeBinaryOperators = FormatStyle::BOS_All;
   CHECK_PARSE("BreakBeforeBinaryOperators: NonAssignment",