-
Notifications
You must be signed in to change notification settings - Fork 15.1k
[clang][clang-cl] Add -fms-preprocessor-compat and support /Zc:preprocessor #167200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
This adds a new compiler flag called -fms-preprocessor-compat which may be used to control whether the preprocessor enables full compatibility mode, which was previously rolled into -fms-compatibility. This means that /Zc:preprocessor for clang-cl can be implemented as an alias for -fno-preprocessor-compat. This diverges from MSVC in that using C11 or newer will not automatically use the conforming preprocessor. Additionally, there is currently not a way to opt out of the charize operator and the /##/ trick when using just the light -fms-extensions. Opting out of the non-conforming preprocessor while using -fms-compatibility will match MSVC in that those two extensions are disabled.
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
|
@llvm/pr-subscribers-clang-driver Author: Josh Dowell (Slartibarty) ChangesThis adds a new compiler flag called This can be merged as-is, but I would greatly appreciate feedback on this from those more experienced with working with the Clang frontend, specifically regarding the following points: Under MSVC, There is currently not a way to opt-out of the two preprocessor extensions enabled by Fixes #147304 Full diff: https://github.com/llvm/llvm-project/pull/167200.diff 14 Files Affected:
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 3a4e1fce2511e..a0db2a8794e14 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -306,12 +306,17 @@ Non-comprehensive list of changes in this release
- Clang now rejects the invalid use of ``constexpr`` with ``auto`` and an explicit type in C. (#GH163090)
+- ``_MSVC_TRADITIONAL`` is now defined to ``0`` when using ``-fms-extensions``. The charize operator and ``/##/`` paste trick remain as supported extensions, which is a divergence from MSVC.
+
New Compiler Flags
------------------
- New option ``-fno-sanitize-debug-trap-reasons`` added to disable emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
- New option ``-fsanitize-debug-trap-reasons=`` added to control emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
- New options for enabling allocation token instrumentation: ``-fsanitize=alloc-token``, ``-falloc-token-max=``, ``-fsanitize-alloc-token-fast-abi``, ``-fsanitize-alloc-token-extended``.
- The ``-resource-dir`` option is now displayed in the list of options shown by ``--help``.
+- The ``-fms-preprocessor-compat`` flag has been added to control whether full emulation of the MSVC traditional preprocessor is performed.
+ Using ``-fms-preprocessor-compat`` will define ``_MSVC_TRADITIONAL`` to ``1``, and enables the full set of functionality required to emulate
+ the old MSVC character buffer based preprocessor as closely as possible. It is enabled by default when using ``-fms-compatibility``, which is the case for clang-cl.
Lanai Support
^^^^^^^^^^^^^^
@@ -581,6 +586,8 @@ Android Support
Windows Support
^^^^^^^^^^^^^^^
+- clang-cl now supports ``/Zc:preprocessor``, which is an alias for ``-fno-ms-preprocessor-compat``.
+
LoongArch Support
^^^^^^^^^^^^^^^^^
- Enable linker relaxation by default for loongarch64.
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index fb22ad3c90af4..2d5208ed317e9 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -5055,6 +5055,8 @@ Execute ``clang-cl /?`` to see a list of supported options:
/Zc:trigraphs Enable trigraphs
/Zc:twoPhase- Disable two-phase name lookup in templates
/Zc:twoPhase Enable two-phase name lookup in templates
+ /Zc:preprocessor- Use the traditional (non-conforming) preprocessor (default)
+ /Zc:preprocessor Use the standard conforming preprocessor
/Zi Alias for /Z7. Does not produce PDBs.
/Zl Don't mention any default libraries in the object file
/Zp Set the default maximum struct packing alignment to 1
diff --git a/clang/include/clang/Basic/LangOptions.def b/clang/include/clang/Basic/LangOptions.def
index 40fc66ea12e34..33dc642f7a280 100644
--- a/clang/include/clang/Basic/LangOptions.def
+++ b/clang/include/clang/Basic/LangOptions.def
@@ -269,6 +269,7 @@ LANGOPT(HIPStdParInterposeAlloc, 1, 0, NotCompatible, "Replace allocations / dea
LANGOPT(OpenACC , 1, 0, NotCompatible, "OpenACC Enabled")
+LANGOPT(MSVCPreprocessor , 1, 0, NotCompatible, "Fully emulate the Microsoft Visual C++ traditional (non-conforming) preprocessor")
LANGOPT(MSVCEnableStdcMacro , 1, 0, NotCompatible, "Define __STDC__ with '-fms-compatibility'")
LANGOPT(SizedDeallocation , 1, 0, NotCompatible, "sized deallocation")
LANGOPT(AlignedAllocation , 1, 0, NotCompatible, "aligned allocation")
diff --git a/clang/include/clang/Basic/LangOptions.h b/clang/include/clang/Basic/LangOptions.h
index 8aa89d8c8c807..02d958878088f 100644
--- a/clang/include/clang/Basic/LangOptions.h
+++ b/clang/include/clang/Basic/LangOptions.h
@@ -627,6 +627,26 @@ class LangOptions : public LangOptionsBase {
return MSCompatibilityVersion >= MajorVersion * 100000U;
}
+ /// Returns true if we want to enable lightweight Microsoft preprocessor
+ /// extensions.
+ /// Historically, MicrosoftExt and MSVCCompat enabled different extensions
+ /// to the preprocessor, but we want to disable a majority of them when
+ /// MSVCPreprocessor is set to false.
+ bool wantsMSVCPreprocessorExtensions() const {
+ // If we want full compatibility then we need the extensions
+ if (MSVCPreprocessor)
+ return true;
+ // If MS extensions are turned off, we don't want the extensions
+ if (!MicrosoftExt)
+ return false;
+ // If we're in full MSVC compat mode, we know MSVCPreprocessor is false,
+ // thus, we don't want the extensions
+ if (MSVCCompat)
+ return false;
+ // We just want lightweight extensions
+ return true;
+ }
+
bool isOverflowPatternExcluded(OverflowPatternExclusionKind Kind) const {
if (OverflowPatternExclusionMask & OverflowPatternExclusionKind::None)
return false;
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 11e81e032d5fc..795b1d4cbf9f1 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3238,6 +3238,10 @@ def fms_compatibility : Flag<["-"], "fms-compatibility">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Enable full Microsoft Visual C++ compatibility">,
MarshallingInfoFlag<LangOpts<"MSVCCompat">>;
+def fms_preprocessor_compat : Flag<["-"], "fms-preprocessor-compat">, Group<f_Group>,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Enable full emulation of the traditional MSVC preprocesor">,
+ MarshallingInfoFlag<LangOpts<"MSVCPreprocessor">>;
def fms_define_stdc : Flag<["-"], "fms-define-stdc">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Define '__STDC__' to '1' in MSVC Compatibility mode">,
@@ -3614,6 +3618,8 @@ def fno_ms_extensions : Flag<["-"], "fno-ms-extensions">, Group<f_Group>,
Visibility<[ClangOption, CLOption]>;
def fno_ms_compatibility : Flag<["-"], "fno-ms-compatibility">, Group<f_Group>,
Visibility<[ClangOption, CLOption]>;
+def fno_ms_preprocessor_compat : Flag<["-"], "fno-ms-preprocessor-compat">, Group<f_Group>,
+ Visibility<[ClangOption]>;
def fno_objc_legacy_dispatch : Flag<["-"], "fno-objc-legacy-dispatch">, Group<f_Group>;
def fno_objc_weak : Flag<["-"], "fno-objc-weak">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>;
@@ -9123,6 +9129,12 @@ def _SLASH_vd : CLJoined<"vd">, HelpText<"Control vtordisp placement">,
Alias<vtordisp_mode_EQ>;
def _SLASH_X : CLFlag<"X">,
HelpText<"Do not add %INCLUDE% to include search path">, Alias<nostdlibinc>;
+def _SLASH_Zc_preprocessor : CLFlag<"Zc:preprocessor">,
+ HelpText<"Use the standard conforming preprocessor">,
+ Alias<fno_ms_preprocessor_compat>;
+def _SLASH_Zc_preprocessor_ : CLFlag<"Zc:preprocessor-">,
+ HelpText<"Use the traditional (non-conforming) preprocessor (default)">,
+ Alias<fms_preprocessor_compat>;
def _SLASH_Zc___STDC__ : CLFlag<"Zc:__STDC__">,
HelpText<"Define __STDC__">,
Alias<fms_define_stdc>;
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 30d3e5293a31b..dd32c0bd92b7e 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -7056,6 +7056,10 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
if (!types::isCXX(Input.getType()) &&
Args.hasArg(options::OPT_fms_define_stdc))
CmdArgs.push_back("-fms-define-stdc");
+ // FIXME: This should be disabled by default in C11 and newer.
+ if (Args.hasFlag(options::OPT_fms_preprocessor_compat,
+ options::OPT_fno_ms_preprocessor_compat, true))
+ CmdArgs.push_back("-fms-preprocessor-compat");
}
if (Triple.isWindowsMSVCEnvironment() && !D.IsCLMode() &&
diff --git a/clang/lib/Frontend/InitPreprocessor.cpp b/clang/lib/Frontend/InitPreprocessor.cpp
index b88d9f89c5f71..d9571c57cfe86 100644
--- a/clang/lib/Frontend/InitPreprocessor.cpp
+++ b/clang/lib/Frontend/InitPreprocessor.cpp
@@ -1022,6 +1022,17 @@ static void InitializePredefinedMacros(const TargetInfo &TI,
}
}
+ if (LangOpts.MSVCPreprocessor) {
+ Builder.defineMacro("_MSVC_TRADITIONAL", "1");
+ } else if (LangOpts.MicrosoftExt) {
+ // Note: when set to 0 this macro implies, under cl.exe, that the
+ // charize operator and /##/ no longer function as they used to.
+ // However, we have special code that cleanly deals with these patterns,
+ // so we'll allow them to exist as extensions, even though we are stating
+ // that we do not support the "traditional" preprocessor.
+ Builder.defineMacro("_MSVC_TRADITIONAL", "0");
+ }
+
// Macros to help identify the narrow and wide character sets
// FIXME: clang currently ignores -fexec-charset=. If this changes,
// then this may need to be updated.
diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index b282a600c0e56..94bb5a8446e55 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -2181,7 +2181,7 @@ const char *Lexer::LexUDSuffix(Token &Result, const char *CurPtr,
if (!IsUDSuffix) {
if (!isLexingRawMode())
- Diag(CurPtr, LangOpts.MSVCCompat
+ Diag(CurPtr, LangOpts.MSVCPreprocessor
? diag::ext_ms_reserved_user_defined_literal
: diag::ext_reserved_user_defined_literal)
<< FixItHint::CreateInsertion(getSourceLocation(CurPtr), " ");
@@ -4217,7 +4217,7 @@ bool Lexer::LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine) {
Kind = tok::hashhash; // '%:%:' -> '##'
CurPtr = ConsumeChar(ConsumeChar(CurPtr, SizeTmp, Result),
SizeTmp2, Result);
- } else if (Char == '@' && LangOpts.MicrosoftExt) {// %:@ -> #@ -> Charize
+ } else if (Char == '@' && LangOpts.wantsMSVCPreprocessorExtensions()) {// %:@ -> #@ -> Charize
CurPtr = ConsumeChar(CurPtr, SizeTmp, Result);
if (!isLexingRawMode())
Diag(BufferPtr, diag::ext_charize_microsoft);
@@ -4405,7 +4405,7 @@ bool Lexer::LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine) {
if (Char == '#') {
Kind = tok::hashhash;
CurPtr = ConsumeChar(CurPtr, SizeTmp, Result);
- } else if (Char == '@' && LangOpts.MicrosoftExt) { // #@ -> Charize
+ } else if (Char == '@' && LangOpts.wantsMSVCPreprocessorExtensions()) { // #@ -> Charize
Kind = tok::hashat;
if (!isLexingRawMode())
Diag(BufferPtr, diag::ext_charize_microsoft);
diff --git a/clang/lib/Lex/TokenLexer.cpp b/clang/lib/Lex/TokenLexer.cpp
index 47f4134fb1465..cbe6ba1d56e1f 100644
--- a/clang/lib/Lex/TokenLexer.cpp
+++ b/clang/lib/Lex/TokenLexer.cpp
@@ -145,7 +145,7 @@ bool TokenLexer::MaybeRemoveCommaBeforeVaArgs(
// In Microsoft-compatibility mode, a comma is removed in the expansion
// of " ... , __VA_ARGS__ " if __VA_ARGS__ is empty. This extension is
// not supported by gcc.
- if (!HasPasteOperator && !PP.getLangOpts().MSVCCompat)
+ if (!HasPasteOperator && !PP.getLangOpts().MSVCPreprocessor)
return false;
// GCC removes the comma in the expansion of " ... , ## __VA_ARGS__ " if
@@ -467,7 +467,7 @@ void TokenLexer::ExpandFunctionArguments() {
// behavior by not considering single commas from nested macro
// expansions as argument separators. Set a flag on the token so we can
// test for this later when the macro expansion is processed.
- if (PP.getLangOpts().MSVCCompat && NumToks == 1 &&
+ if (PP.getLangOpts().MSVCPreprocessor && NumToks == 1 &&
ResultToks.back().is(tok::comma))
ResultToks.back().setFlag(Token::IgnoredComma);
@@ -653,7 +653,7 @@ bool TokenLexer::Lex(Token &Tok) {
// Special processing of L#x macros in -fms-compatibility mode.
// Microsoft compiler is able to form a wide string literal from
// 'L#macro_arg' construct in a function-like macro.
- (PP.getLangOpts().MSVCCompat &&
+ (PP.getLangOpts().MSVCPreprocessor &&
isWideStringLiteralFromMacro(Tok, Tokens[CurTokenIdx])))) {
// When handling the microsoft /##/ extension, the final token is
// returned by pasteTokens, not the pasted token.
@@ -732,7 +732,7 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream,
unsigned int &CurIdx) {
assert(CurIdx > 0 && "## can not be the first token within tokens");
assert((TokenStream[CurIdx].is(tok::hashhash) ||
- (PP.getLangOpts().MSVCCompat &&
+ (PP.getLangOpts().MSVCPreprocessor &&
isWideStringLiteralFromMacro(LHSTok, TokenStream[CurIdx]))) &&
"Token at this Index must be ## or part of the MSVC 'L "
"#macro-arg' pasting pair");
@@ -740,7 +740,7 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream,
// MSVC: If previous token was pasted, this must be a recovery from an invalid
// paste operation. Ignore spaces before this token to mimic MSVC output.
// Required for generating valid UUID strings in some MS headers.
- if (PP.getLangOpts().MicrosoftExt && (CurIdx >= 2) &&
+ if (PP.getLangOpts().wantsMSVCPreprocessorExtensions() && (CurIdx >= 2) &&
TokenStream[CurIdx - 2].is(tok::hashhash))
LHSTok.clearFlag(Token::LeadingSpace);
@@ -854,8 +854,8 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream,
// Test for the Microsoft extension of /##/ turning into // here on the
// error path.
- if (PP.getLangOpts().MicrosoftExt && LHSTok.is(tok::slash) &&
- RHS.is(tok::slash)) {
+ if (PP.getLangOpts().wantsMSVCPreprocessorExtensions() &&
+ LHSTok.is(tok::slash) && RHS.is(tok::slash)) {
HandleMicrosoftCommentPaste(LHSTok, Loc);
return true;
}
diff --git a/clang/test/Driver/cl-options.c b/clang/test/Driver/cl-options.c
index 1b1169b71554a..fb137d48c0fe8 100644
--- a/clang/test/Driver/cl-options.c
+++ b/clang/test/Driver/cl-options.c
@@ -375,6 +375,12 @@
// RUN: %clang_cl -c -### /std:clatest -- %s 2>&1 | FileCheck -check-prefix CHECK-CLATEST %s
// CHECK-CLATEST: -std=c23
+// RUN: %clang_cl -c -### /Zc:preprocessor- -- %s 2>&1 | FileCheck -check-prefix CHECK-ZC-PREPROCESSOR-NO %s
+// CHECK-ZC-PREPROCESSOR-NO: -fms-preprocessor-compat
+
+// RUN: %clang_cl -c -### /Zc:preprocessor -- %s 2>&1 | FileCheck -check-prefix CHECK-ZC-PREPROCESSOR %s
+// CHECK-ZC-PREPROCESSOR-NOT: -fms-preprocessor-compat
+
// For some warning ids, we can map from MSVC warning to Clang warning.
// RUN: %clang_cl -wd4005 -wd4100 -wd4910 -wd4996 -wd12345678 -### -- %s 2>&1 | FileCheck -check-prefix=Wno %s
// Wno: "-cc1"
diff --git a/clang/test/Lexer/ms-compatibility.c b/clang/test/Lexer/ms-compatibility.c
index 2981b8e062e93..ebaf89a7c4738 100644
--- a/clang/test/Lexer/ms-compatibility.c
+++ b/clang/test/Lexer/ms-compatibility.c
@@ -1,4 +1,4 @@
-// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s
// RUN: %clang_cc1 -E %s | FileCheck --check-prefix=CHECK-NO-MS-COMPAT %s
#define FN(x) L#x
diff --git a/clang/test/Preprocessor/macro_fn_comma_swallow2.c b/clang/test/Preprocessor/macro_fn_comma_swallow2.c
index 93ab2b83664a1..f594e4139bb33 100644
--- a/clang/test/Preprocessor/macro_fn_comma_swallow2.c
+++ b/clang/test/Preprocessor/macro_fn_comma_swallow2.c
@@ -5,7 +5,7 @@
// RUN: %clang_cc1 -E -std=c11 %s | FileCheck -check-prefix=C99 -strict-whitespace %s
// RUN: %clang_cc1 -E -x c++ %s | FileCheck -check-prefix=GCC -strict-whitespace %s
// RUN: %clang_cc1 -E -std=gnu99 %s | FileCheck -check-prefix=GCC -strict-whitespace %s
-// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck -check-prefix=MS -strict-whitespace %s
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck -check-prefix=MS -strict-whitespace %s
// RUN: %clang_cc1 -E -DNAMED %s | FileCheck -check-prefix=GCC -strict-whitespace %s
// RUN: %clang_cc1 -E -std=c99 -DNAMED %s | FileCheck -check-prefix=C99 -strict-whitespace %s
diff --git a/clang/test/Preprocessor/microsoft-ext.c b/clang/test/Preprocessor/microsoft-ext.c
index aae52a84b62f1..51e2f9aff234d 100644
--- a/clang/test/Preprocessor/microsoft-ext.c
+++ b/clang/test/Preprocessor/microsoft-ext.c
@@ -1,4 +1,4 @@
-// RUN: %clang_cc1 -E -fms-compatibility %s -o %t
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s -o %t
// RUN: FileCheck %s < %t
# define M2(x, y) x + y
diff --git a/clang/test/Preprocessor/microsoft-preprocessor-compat.c b/clang/test/Preprocessor/microsoft-preprocessor-compat.c
new file mode 100644
index 0000000000000..29883409a85b8
--- /dev/null
+++ b/clang/test/Preprocessor/microsoft-preprocessor-compat.c
@@ -0,0 +1,36 @@
+// This test verifies that _MSVC_TRADITIONAL is defined under the right
+// circumstances, microsoft-ext.c is responsible for testing the implementation
+// details of the traditional preprocessor.
+// FIXME: C11 should eventually use the conforming preprocessor by default.
+//
+// RUN: %clang_cc1 -E %s | FileCheck --check-prefix=CHECK-DEFAULT %s
+// RUN: %clang_cc1 -E -fms-extensions %s | FileCheck --check-prefix=CHECK-MS-EXT %s
+// RUN: %clang -E -fms-compatibility %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-MS-PREPRO-COMPAT %s
+// RUN: %clang_cl -E %s | FileCheck --check-prefix=CHECK-CL %s
+// RUN: %clang_cl -E /std:c11 %s | FileCheck --check-prefix=CHECK-C11 %s
+// RUN: %clang_cl -E /Zc:preprocessor %s | FileCheck --check-prefix=CHECK-ZC-PREPRO %s
+// RUN: %clang_cl -E /clang:-fno-ms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-NO-PREPRO-COMPAT %s
+
+typedef enum {
+ NOT_DEFINED,
+ IS_ZERO,
+ IS_ONE
+} State;
+
+#if !defined(_MSVC_TRADITIONAL)
+State state = NOT_DEFINED;
+#elif _MSVC_TRADITIONAL == 0
+State state = IS_ZERO;
+#elif _MSVC_TRADITIONAL == 1
+State state = IS_ONE;
+#endif
+
+// CHECK-DEFAULT: State state = NOT_DEFINED;
+// CHECK-MS-EXT: State state = IS_ZERO;
+// CHECK-MS-COMPAT: State state = IS_ONE;
+// CHECK-MS-PREPRO-COMPAT: State state = IS_ONE;
+// CHECK-CL: State state = IS_ONE;
+// CHECK-C11: State state = IS_ONE;
+// CHECK-ZC-PREPRO: State state = IS_ZERO;
+// CHECK-NO-PREPRO-COMPAT: State state = IS_ZERO;
|
|
@llvm/pr-subscribers-clang Author: Josh Dowell (Slartibarty) ChangesThis adds a new compiler flag called This can be merged as-is, but I would greatly appreciate feedback on this from those more experienced with working with the Clang frontend, specifically regarding the following points: Under MSVC, There is currently not a way to opt-out of the two preprocessor extensions enabled by Fixes #147304 Full diff: https://github.com/llvm/llvm-project/pull/167200.diff 14 Files Affected:
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 3a4e1fce2511e..a0db2a8794e14 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -306,12 +306,17 @@ Non-comprehensive list of changes in this release
- Clang now rejects the invalid use of ``constexpr`` with ``auto`` and an explicit type in C. (#GH163090)
+- ``_MSVC_TRADITIONAL`` is now defined to ``0`` when using ``-fms-extensions``. The charize operator and ``/##/`` paste trick remain as supported extensions, which is a divergence from MSVC.
+
New Compiler Flags
------------------
- New option ``-fno-sanitize-debug-trap-reasons`` added to disable emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
- New option ``-fsanitize-debug-trap-reasons=`` added to control emitting trap reasons into the debug info when compiling with trapping UBSan (e.g. ``-fsanitize-trap=undefined``).
- New options for enabling allocation token instrumentation: ``-fsanitize=alloc-token``, ``-falloc-token-max=``, ``-fsanitize-alloc-token-fast-abi``, ``-fsanitize-alloc-token-extended``.
- The ``-resource-dir`` option is now displayed in the list of options shown by ``--help``.
+- The ``-fms-preprocessor-compat`` flag has been added to control whether full emulation of the MSVC traditional preprocessor is performed.
+ Using ``-fms-preprocessor-compat`` will define ``_MSVC_TRADITIONAL`` to ``1``, and enables the full set of functionality required to emulate
+ the old MSVC character buffer based preprocessor as closely as possible. It is enabled by default when using ``-fms-compatibility``, which is the case for clang-cl.
Lanai Support
^^^^^^^^^^^^^^
@@ -581,6 +586,8 @@ Android Support
Windows Support
^^^^^^^^^^^^^^^
+- clang-cl now supports ``/Zc:preprocessor``, which is an alias for ``-fno-ms-preprocessor-compat``.
+
LoongArch Support
^^^^^^^^^^^^^^^^^
- Enable linker relaxation by default for loongarch64.
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index fb22ad3c90af4..2d5208ed317e9 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -5055,6 +5055,8 @@ Execute ``clang-cl /?`` to see a list of supported options:
/Zc:trigraphs Enable trigraphs
/Zc:twoPhase- Disable two-phase name lookup in templates
/Zc:twoPhase Enable two-phase name lookup in templates
+ /Zc:preprocessor- Use the traditional (non-conforming) preprocessor (default)
+ /Zc:preprocessor Use the standard conforming preprocessor
/Zi Alias for /Z7. Does not produce PDBs.
/Zl Don't mention any default libraries in the object file
/Zp Set the default maximum struct packing alignment to 1
diff --git a/clang/include/clang/Basic/LangOptions.def b/clang/include/clang/Basic/LangOptions.def
index 40fc66ea12e34..33dc642f7a280 100644
--- a/clang/include/clang/Basic/LangOptions.def
+++ b/clang/include/clang/Basic/LangOptions.def
@@ -269,6 +269,7 @@ LANGOPT(HIPStdParInterposeAlloc, 1, 0, NotCompatible, "Replace allocations / dea
LANGOPT(OpenACC , 1, 0, NotCompatible, "OpenACC Enabled")
+LANGOPT(MSVCPreprocessor , 1, 0, NotCompatible, "Fully emulate the Microsoft Visual C++ traditional (non-conforming) preprocessor")
LANGOPT(MSVCEnableStdcMacro , 1, 0, NotCompatible, "Define __STDC__ with '-fms-compatibility'")
LANGOPT(SizedDeallocation , 1, 0, NotCompatible, "sized deallocation")
LANGOPT(AlignedAllocation , 1, 0, NotCompatible, "aligned allocation")
diff --git a/clang/include/clang/Basic/LangOptions.h b/clang/include/clang/Basic/LangOptions.h
index 8aa89d8c8c807..02d958878088f 100644
--- a/clang/include/clang/Basic/LangOptions.h
+++ b/clang/include/clang/Basic/LangOptions.h
@@ -627,6 +627,26 @@ class LangOptions : public LangOptionsBase {
return MSCompatibilityVersion >= MajorVersion * 100000U;
}
+ /// Returns true if we want to enable lightweight Microsoft preprocessor
+ /// extensions.
+ /// Historically, MicrosoftExt and MSVCCompat enabled different extensions
+ /// to the preprocessor, but we want to disable a majority of them when
+ /// MSVCPreprocessor is set to false.
+ bool wantsMSVCPreprocessorExtensions() const {
+ // If we want full compatibility then we need the extensions
+ if (MSVCPreprocessor)
+ return true;
+ // If MS extensions are turned off, we don't want the extensions
+ if (!MicrosoftExt)
+ return false;
+ // If we're in full MSVC compat mode, we know MSVCPreprocessor is false,
+ // thus, we don't want the extensions
+ if (MSVCCompat)
+ return false;
+ // We just want lightweight extensions
+ return true;
+ }
+
bool isOverflowPatternExcluded(OverflowPatternExclusionKind Kind) const {
if (OverflowPatternExclusionMask & OverflowPatternExclusionKind::None)
return false;
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 11e81e032d5fc..795b1d4cbf9f1 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3238,6 +3238,10 @@ def fms_compatibility : Flag<["-"], "fms-compatibility">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Enable full Microsoft Visual C++ compatibility">,
MarshallingInfoFlag<LangOpts<"MSVCCompat">>;
+def fms_preprocessor_compat : Flag<["-"], "fms-preprocessor-compat">, Group<f_Group>,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Enable full emulation of the traditional MSVC preprocesor">,
+ MarshallingInfoFlag<LangOpts<"MSVCPreprocessor">>;
def fms_define_stdc : Flag<["-"], "fms-define-stdc">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Define '__STDC__' to '1' in MSVC Compatibility mode">,
@@ -3614,6 +3618,8 @@ def fno_ms_extensions : Flag<["-"], "fno-ms-extensions">, Group<f_Group>,
Visibility<[ClangOption, CLOption]>;
def fno_ms_compatibility : Flag<["-"], "fno-ms-compatibility">, Group<f_Group>,
Visibility<[ClangOption, CLOption]>;
+def fno_ms_preprocessor_compat : Flag<["-"], "fno-ms-preprocessor-compat">, Group<f_Group>,
+ Visibility<[ClangOption]>;
def fno_objc_legacy_dispatch : Flag<["-"], "fno-objc-legacy-dispatch">, Group<f_Group>;
def fno_objc_weak : Flag<["-"], "fno-objc-weak">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>;
@@ -9123,6 +9129,12 @@ def _SLASH_vd : CLJoined<"vd">, HelpText<"Control vtordisp placement">,
Alias<vtordisp_mode_EQ>;
def _SLASH_X : CLFlag<"X">,
HelpText<"Do not add %INCLUDE% to include search path">, Alias<nostdlibinc>;
+def _SLASH_Zc_preprocessor : CLFlag<"Zc:preprocessor">,
+ HelpText<"Use the standard conforming preprocessor">,
+ Alias<fno_ms_preprocessor_compat>;
+def _SLASH_Zc_preprocessor_ : CLFlag<"Zc:preprocessor-">,
+ HelpText<"Use the traditional (non-conforming) preprocessor (default)">,
+ Alias<fms_preprocessor_compat>;
def _SLASH_Zc___STDC__ : CLFlag<"Zc:__STDC__">,
HelpText<"Define __STDC__">,
Alias<fms_define_stdc>;
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 30d3e5293a31b..dd32c0bd92b7e 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -7056,6 +7056,10 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
if (!types::isCXX(Input.getType()) &&
Args.hasArg(options::OPT_fms_define_stdc))
CmdArgs.push_back("-fms-define-stdc");
+ // FIXME: This should be disabled by default in C11 and newer.
+ if (Args.hasFlag(options::OPT_fms_preprocessor_compat,
+ options::OPT_fno_ms_preprocessor_compat, true))
+ CmdArgs.push_back("-fms-preprocessor-compat");
}
if (Triple.isWindowsMSVCEnvironment() && !D.IsCLMode() &&
diff --git a/clang/lib/Frontend/InitPreprocessor.cpp b/clang/lib/Frontend/InitPreprocessor.cpp
index b88d9f89c5f71..d9571c57cfe86 100644
--- a/clang/lib/Frontend/InitPreprocessor.cpp
+++ b/clang/lib/Frontend/InitPreprocessor.cpp
@@ -1022,6 +1022,17 @@ static void InitializePredefinedMacros(const TargetInfo &TI,
}
}
+ if (LangOpts.MSVCPreprocessor) {
+ Builder.defineMacro("_MSVC_TRADITIONAL", "1");
+ } else if (LangOpts.MicrosoftExt) {
+ // Note: when set to 0 this macro implies, under cl.exe, that the
+ // charize operator and /##/ no longer function as they used to.
+ // However, we have special code that cleanly deals with these patterns,
+ // so we'll allow them to exist as extensions, even though we are stating
+ // that we do not support the "traditional" preprocessor.
+ Builder.defineMacro("_MSVC_TRADITIONAL", "0");
+ }
+
// Macros to help identify the narrow and wide character sets
// FIXME: clang currently ignores -fexec-charset=. If this changes,
// then this may need to be updated.
diff --git a/clang/lib/Lex/Lexer.cpp b/clang/lib/Lex/Lexer.cpp
index b282a600c0e56..94bb5a8446e55 100644
--- a/clang/lib/Lex/Lexer.cpp
+++ b/clang/lib/Lex/Lexer.cpp
@@ -2181,7 +2181,7 @@ const char *Lexer::LexUDSuffix(Token &Result, const char *CurPtr,
if (!IsUDSuffix) {
if (!isLexingRawMode())
- Diag(CurPtr, LangOpts.MSVCCompat
+ Diag(CurPtr, LangOpts.MSVCPreprocessor
? diag::ext_ms_reserved_user_defined_literal
: diag::ext_reserved_user_defined_literal)
<< FixItHint::CreateInsertion(getSourceLocation(CurPtr), " ");
@@ -4217,7 +4217,7 @@ bool Lexer::LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine) {
Kind = tok::hashhash; // '%:%:' -> '##'
CurPtr = ConsumeChar(ConsumeChar(CurPtr, SizeTmp, Result),
SizeTmp2, Result);
- } else if (Char == '@' && LangOpts.MicrosoftExt) {// %:@ -> #@ -> Charize
+ } else if (Char == '@' && LangOpts.wantsMSVCPreprocessorExtensions()) {// %:@ -> #@ -> Charize
CurPtr = ConsumeChar(CurPtr, SizeTmp, Result);
if (!isLexingRawMode())
Diag(BufferPtr, diag::ext_charize_microsoft);
@@ -4405,7 +4405,7 @@ bool Lexer::LexTokenInternal(Token &Result, bool TokAtPhysicalStartOfLine) {
if (Char == '#') {
Kind = tok::hashhash;
CurPtr = ConsumeChar(CurPtr, SizeTmp, Result);
- } else if (Char == '@' && LangOpts.MicrosoftExt) { // #@ -> Charize
+ } else if (Char == '@' && LangOpts.wantsMSVCPreprocessorExtensions()) { // #@ -> Charize
Kind = tok::hashat;
if (!isLexingRawMode())
Diag(BufferPtr, diag::ext_charize_microsoft);
diff --git a/clang/lib/Lex/TokenLexer.cpp b/clang/lib/Lex/TokenLexer.cpp
index 47f4134fb1465..cbe6ba1d56e1f 100644
--- a/clang/lib/Lex/TokenLexer.cpp
+++ b/clang/lib/Lex/TokenLexer.cpp
@@ -145,7 +145,7 @@ bool TokenLexer::MaybeRemoveCommaBeforeVaArgs(
// In Microsoft-compatibility mode, a comma is removed in the expansion
// of " ... , __VA_ARGS__ " if __VA_ARGS__ is empty. This extension is
// not supported by gcc.
- if (!HasPasteOperator && !PP.getLangOpts().MSVCCompat)
+ if (!HasPasteOperator && !PP.getLangOpts().MSVCPreprocessor)
return false;
// GCC removes the comma in the expansion of " ... , ## __VA_ARGS__ " if
@@ -467,7 +467,7 @@ void TokenLexer::ExpandFunctionArguments() {
// behavior by not considering single commas from nested macro
// expansions as argument separators. Set a flag on the token so we can
// test for this later when the macro expansion is processed.
- if (PP.getLangOpts().MSVCCompat && NumToks == 1 &&
+ if (PP.getLangOpts().MSVCPreprocessor && NumToks == 1 &&
ResultToks.back().is(tok::comma))
ResultToks.back().setFlag(Token::IgnoredComma);
@@ -653,7 +653,7 @@ bool TokenLexer::Lex(Token &Tok) {
// Special processing of L#x macros in -fms-compatibility mode.
// Microsoft compiler is able to form a wide string literal from
// 'L#macro_arg' construct in a function-like macro.
- (PP.getLangOpts().MSVCCompat &&
+ (PP.getLangOpts().MSVCPreprocessor &&
isWideStringLiteralFromMacro(Tok, Tokens[CurTokenIdx])))) {
// When handling the microsoft /##/ extension, the final token is
// returned by pasteTokens, not the pasted token.
@@ -732,7 +732,7 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream,
unsigned int &CurIdx) {
assert(CurIdx > 0 && "## can not be the first token within tokens");
assert((TokenStream[CurIdx].is(tok::hashhash) ||
- (PP.getLangOpts().MSVCCompat &&
+ (PP.getLangOpts().MSVCPreprocessor &&
isWideStringLiteralFromMacro(LHSTok, TokenStream[CurIdx]))) &&
"Token at this Index must be ## or part of the MSVC 'L "
"#macro-arg' pasting pair");
@@ -740,7 +740,7 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream,
// MSVC: If previous token was pasted, this must be a recovery from an invalid
// paste operation. Ignore spaces before this token to mimic MSVC output.
// Required for generating valid UUID strings in some MS headers.
- if (PP.getLangOpts().MicrosoftExt && (CurIdx >= 2) &&
+ if (PP.getLangOpts().wantsMSVCPreprocessorExtensions() && (CurIdx >= 2) &&
TokenStream[CurIdx - 2].is(tok::hashhash))
LHSTok.clearFlag(Token::LeadingSpace);
@@ -854,8 +854,8 @@ bool TokenLexer::pasteTokens(Token &LHSTok, ArrayRef<Token> TokenStream,
// Test for the Microsoft extension of /##/ turning into // here on the
// error path.
- if (PP.getLangOpts().MicrosoftExt && LHSTok.is(tok::slash) &&
- RHS.is(tok::slash)) {
+ if (PP.getLangOpts().wantsMSVCPreprocessorExtensions() &&
+ LHSTok.is(tok::slash) && RHS.is(tok::slash)) {
HandleMicrosoftCommentPaste(LHSTok, Loc);
return true;
}
diff --git a/clang/test/Driver/cl-options.c b/clang/test/Driver/cl-options.c
index 1b1169b71554a..fb137d48c0fe8 100644
--- a/clang/test/Driver/cl-options.c
+++ b/clang/test/Driver/cl-options.c
@@ -375,6 +375,12 @@
// RUN: %clang_cl -c -### /std:clatest -- %s 2>&1 | FileCheck -check-prefix CHECK-CLATEST %s
// CHECK-CLATEST: -std=c23
+// RUN: %clang_cl -c -### /Zc:preprocessor- -- %s 2>&1 | FileCheck -check-prefix CHECK-ZC-PREPROCESSOR-NO %s
+// CHECK-ZC-PREPROCESSOR-NO: -fms-preprocessor-compat
+
+// RUN: %clang_cl -c -### /Zc:preprocessor -- %s 2>&1 | FileCheck -check-prefix CHECK-ZC-PREPROCESSOR %s
+// CHECK-ZC-PREPROCESSOR-NOT: -fms-preprocessor-compat
+
// For some warning ids, we can map from MSVC warning to Clang warning.
// RUN: %clang_cl -wd4005 -wd4100 -wd4910 -wd4996 -wd12345678 -### -- %s 2>&1 | FileCheck -check-prefix=Wno %s
// Wno: "-cc1"
diff --git a/clang/test/Lexer/ms-compatibility.c b/clang/test/Lexer/ms-compatibility.c
index 2981b8e062e93..ebaf89a7c4738 100644
--- a/clang/test/Lexer/ms-compatibility.c
+++ b/clang/test/Lexer/ms-compatibility.c
@@ -1,4 +1,4 @@
-// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s
// RUN: %clang_cc1 -E %s | FileCheck --check-prefix=CHECK-NO-MS-COMPAT %s
#define FN(x) L#x
diff --git a/clang/test/Preprocessor/macro_fn_comma_swallow2.c b/clang/test/Preprocessor/macro_fn_comma_swallow2.c
index 93ab2b83664a1..f594e4139bb33 100644
--- a/clang/test/Preprocessor/macro_fn_comma_swallow2.c
+++ b/clang/test/Preprocessor/macro_fn_comma_swallow2.c
@@ -5,7 +5,7 @@
// RUN: %clang_cc1 -E -std=c11 %s | FileCheck -check-prefix=C99 -strict-whitespace %s
// RUN: %clang_cc1 -E -x c++ %s | FileCheck -check-prefix=GCC -strict-whitespace %s
// RUN: %clang_cc1 -E -std=gnu99 %s | FileCheck -check-prefix=GCC -strict-whitespace %s
-// RUN: %clang_cc1 -E -fms-compatibility %s | FileCheck -check-prefix=MS -strict-whitespace %s
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck -check-prefix=MS -strict-whitespace %s
// RUN: %clang_cc1 -E -DNAMED %s | FileCheck -check-prefix=GCC -strict-whitespace %s
// RUN: %clang_cc1 -E -std=c99 -DNAMED %s | FileCheck -check-prefix=C99 -strict-whitespace %s
diff --git a/clang/test/Preprocessor/microsoft-ext.c b/clang/test/Preprocessor/microsoft-ext.c
index aae52a84b62f1..51e2f9aff234d 100644
--- a/clang/test/Preprocessor/microsoft-ext.c
+++ b/clang/test/Preprocessor/microsoft-ext.c
@@ -1,4 +1,4 @@
-// RUN: %clang_cc1 -E -fms-compatibility %s -o %t
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s -o %t
// RUN: FileCheck %s < %t
# define M2(x, y) x + y
diff --git a/clang/test/Preprocessor/microsoft-preprocessor-compat.c b/clang/test/Preprocessor/microsoft-preprocessor-compat.c
new file mode 100644
index 0000000000000..29883409a85b8
--- /dev/null
+++ b/clang/test/Preprocessor/microsoft-preprocessor-compat.c
@@ -0,0 +1,36 @@
+// This test verifies that _MSVC_TRADITIONAL is defined under the right
+// circumstances, microsoft-ext.c is responsible for testing the implementation
+// details of the traditional preprocessor.
+// FIXME: C11 should eventually use the conforming preprocessor by default.
+//
+// RUN: %clang_cc1 -E %s | FileCheck --check-prefix=CHECK-DEFAULT %s
+// RUN: %clang_cc1 -E -fms-extensions %s | FileCheck --check-prefix=CHECK-MS-EXT %s
+// RUN: %clang -E -fms-compatibility %s | FileCheck --check-prefix=CHECK-MS-COMPAT %s
+// RUN: %clang_cc1 -E -fms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-MS-PREPRO-COMPAT %s
+// RUN: %clang_cl -E %s | FileCheck --check-prefix=CHECK-CL %s
+// RUN: %clang_cl -E /std:c11 %s | FileCheck --check-prefix=CHECK-C11 %s
+// RUN: %clang_cl -E /Zc:preprocessor %s | FileCheck --check-prefix=CHECK-ZC-PREPRO %s
+// RUN: %clang_cl -E /clang:-fno-ms-preprocessor-compat %s | FileCheck --check-prefix=CHECK-NO-PREPRO-COMPAT %s
+
+typedef enum {
+ NOT_DEFINED,
+ IS_ZERO,
+ IS_ONE
+} State;
+
+#if !defined(_MSVC_TRADITIONAL)
+State state = NOT_DEFINED;
+#elif _MSVC_TRADITIONAL == 0
+State state = IS_ZERO;
+#elif _MSVC_TRADITIONAL == 1
+State state = IS_ONE;
+#endif
+
+// CHECK-DEFAULT: State state = NOT_DEFINED;
+// CHECK-MS-EXT: State state = IS_ZERO;
+// CHECK-MS-COMPAT: State state = IS_ONE;
+// CHECK-MS-PREPRO-COMPAT: State state = IS_ONE;
+// CHECK-CL: State state = IS_ONE;
+// CHECK-C11: State state = IS_ONE;
+// CHECK-ZC-PREPRO: State state = IS_ZERO;
+// CHECK-NO-PREPRO-COMPAT: State state = IS_ZERO;
|
|
CC'ing @AaronBallman since he was active in the original issue report. I assume the auto-assigned labels will help other maintainers see this too. |
This adds a new compiler flag called
-fms-preprocessor-compatwhich may be used to control whether the preprocessor enables full Microsoft compatibility mode, which was previously rolled into-fms-compatibility.This means that
/Zc:preprocessorfor clang-cl can be implemented as an alias for-fno-ms-preprocessor-compat.It also sets the predefined macro
_MSVC_TRADITIONALunder appropriate circumstances.This can be merged as-is, but I would greatly appreciate feedback on this from those more experienced with working with the Clang frontend, specifically regarding the following points:
Under MSVC,
/Zc:preprocessoris implied when using/std:c11or newer, however I did not implement this due to there not being a straightforward or future proof way to get the current C standard in use when determining arguments for the Clang driver, there is already code in place which checks for C++20 or newer in the same function, for something else, but it is hacky and checks for every possible combination of C++20 arguments, of which there are many more for C11 and newer. Is this an okay divergence?There is currently not a way to opt-out of the two preprocessor extensions enabled by
-fms-extensionsunless you have also specified-fms-compatibilityas well as-fno-ms-preprocessor-compat. I implemented it this way so that using clang-cl like MSVC will indeed match MSVC exactly, where the charize operator and /##/ trick are disabled. I could percieve a situtation where it would be desirable to continue using these extensions under clang-cl, but I preferred matching MSVC. Is this okay?Fixes #147304