Skip to content

v1.5 - Explanation of MSVC Compilation Fixes for PyTorch 2.13.0+cu13.2

Choose a tag to compare

View all tags
@ussoewwin ussoewwin released this 09 Jul 18:44
· 82 commits to main since this release
v1.5
fab466e

1. Error Details

During the build process, two distinct sets of compilation errors were encountered:

① Overload Resolution Ambiguity in c10::HeaderOnlyArrayRef<int64_t>::operator == (C2666)

When comparing tensor sizes (e.g., IntArrayRef equality checks) in PyTorch headers such as torch/nn/functional/activation.h and loss.h, the compiler failed to resolve which operator== to invoke.

D:\USERFILES\fp8e4m3\venv\Lib\site-packages\torch\include\torch\csrc\api\include\torch/nn/functional/activation.h(674): error C2666: 'c10::HeaderOnlyArrayRef<int64_t>::operator ==': overloaded functions have similar conversions
D:\USERFILES\fp8e4m3\venv\Lib\site-packages\torch\include\torch/headeronly/util/HeaderOnlyArrayRef.h(251): note: could be 'bool c10::HeaderOnlyArrayRef<int64_t>::operator ==(c10::HeaderOnlyArrayRef<int64_t>,c10::HeaderOnlyArrayRef<int64_t>)' [found using argument-dependent lookup]
D:\USERFILES\fp8e4m3\venv\Lib\site-packages\torch\include\c10/util/OptionalArrayRef.h(223): note: or 'bool c10::OptionalArrayRef<int64_t>::operator ==(c10::OptionalArrayRef<int64_t>,c10::ArrayRef<int64_t>)' [found using argument-dependent lookup]
D:\USERFILES\fp8e4m3\venv\Lib\site-packages\torch\include\ATen/core/ivalue.h(284): note: or 'bool c10::operator ==(const c10::IValue &,const c10::IValue &)' [found using argument-dependent lookup]

② nvcc (CUDA Compiler) Rejection of C++20 Syntax

The CUDA compiler rejected modern C++20 syntax elements used in PyTorch's headers, such as designated initializers and default member initializers for bitfields.

D:/USERFILES/fp8e4m3/venv/Lib/site-packages/torch/include\c10/util/StringUtil.h(169): error: expected an expression
      return {.function = function, .file = file, .line = line};
              ^

D:/USERFILES/fp8e4m3/venv/Lib/site-packages/torch/include\c10/core/AutogradState.h(89): error: data member initializer is not allowed
    bool view_replay_enabled_ : 1 = false;
                                  ^

2. Root Cause Analysis

① Root Cause of the C2666 Error

Starting with PyTorch 2.13.0, a refactoring was introduced where ArrayRef<T> inherits from HeaderOnlyArrayRef<T>.
However, operator== was only defined in the base class HeaderOnlyArrayRef, leaving the derived class ArrayRef without its own explicit operator== definition.

When comparing two ArrayRef<int64_t> objects, MSVC's ADL (Argument-Dependent Lookup) detects multiple potential overloads:

  1. HeaderOnlyArrayRef's operator== (requiring implicit conversion of both sides to their base class).
  2. OptionalArrayRef's operator== (requiring implicit construction of one side).
  3. IValue's operator== (requiring implicit construction of both sides to IValue).

While compilers like GCC and Clang silently resolve this overload, MSVC strictly enforces C++ standard overload resolution rules. Since all candidates require some form of implicit user-defined conversion of equal rank, MSVC flags the call as ambiguous (C2666).

② Root Cause of the C++20 Syntax Error

PyTorch 2.13.0's C++ headers internally require C++20 features. However, FlashAttention's setup.py explicitly appended -std=c++17 (and /std:c++17 on Windows) to the compiler flags.

Because the -std=c++17 flag was appended after PyTorch's internal -std=c++20 flag, the compiler fallback to C++17 mode. This caused compiling errors when parsing C++20 features:

  • Designated initializers: {.function = ...}
  • Default bitfield member initializers: bool view_replay_enabled_ : 1 = false;

3. Resolution

① Overview

  1. Patching PyTorch Headers (ArrayRef.h):
    Add explicit exact-match operator== and operator!= function templates for c10::ArrayRef<T> inside namespace c10. This eliminates the need for any implicit conversions, resolving MSVC's overload ambiguity.
  2. Updating FlashAttention Build Configuration (setup.py):
    Change the default C++ standard flag from C++17 to C++20 to align with PyTorch's requirements.

② Patched Code (Full Diff)

1) Patch applied to ArrayRef.h

Modified c10/util/ArrayRef.h:L166-L188:

       std::initializer_list<U>) = delete;
 
   /// @}
 };
 
+/// MSVC C2666 workaround: namespace-scope exact-match operator== for ArrayRef.
+/// MSVC fails to find hidden friends in derived class templates via ADL,
+/// causing ambiguity with HeaderOnlyArrayRef, OptionalArrayRef, IValue, and
+/// SymbolicShape overloads. Namespace-scope functions are reliably found via
+/// ADL and, as exact matches, unambiguously win overload resolution.
+template <typename T>
+inline bool operator==(ArrayRef<T> lhs, ArrayRef<T> rhs) {
+  return lhs.equals(rhs);
+}
+template <typename T>
+inline bool operator!=(ArrayRef<T> lhs, ArrayRef<T> rhs) {
+  return !lhs.equals(rhs);
+}
+
 /// Deduction guides for ArrayRef to support CTAD with inherited constructors
 /// These mirror the constructors inherited from HeaderOnlyArrayRef
 /// @{

2) Patch applied to setup.py

Modified setup.py:L320-L355:

     nvcc_flags = [
     "-O3",
-    "-std=c++17",
+    "-std=c++20",
     "-U__CUDA_NO_HALF_OPERATORS__",
     "-U__CUDA_NO_HALF_CONVERSIONS__",
     "-U__CUDA_NO_HALF2_OPERATORS__",
     "-U__CUDA_NO_BFLOAT16_CONVERSIONS__",
     "--expt-relaxed-constexpr",
     "--expt-extended-lambda",
     "--use_fast_math",
     # "--ptxas-options=-v",
     # "--ptxas-options=-O2",
     # "-lineinfo",
     # "-DFLASHATTENTION_DISABLE_BACKWARD",
     # "-DFLASHATTENTION_DISABLE_DROPOUT",
     # "-DFLASHATTENTION_DISABLE_ALIBI",
     # "-DFLASHATTENTION_DISABLE_SOFTCAP",
     # "-DFLASHATTENTION_DISABLE_UNEVEN_K",
     # "-DFLASHATTENTION_DISABLE_LOCAL",
     ]
 
-    compiler_c17_flag=["-O3", "-std=c++17"]
+    compiler_c17_flag=["-O3", "-std=c++20"]
     # Add Windows-specific flags
     if sys.platform == "win32" and os.getenv('DISTUTILS_USE_SDK') == '1':
         # CUDA 13.2 CCCL headers require MSVC conforming preprocessor (see md/CUDA_13.0_TO_13.2_BUILD_FIX.md).
         nvcc_flags.extend(
             ["-Xcompiler", "/Zc:__cplusplus", "-Xcompiler", "/Zc:preprocessor"]
         )
         compiler_c17_flag = [
             "-O2",
-            "/std:c++17",
+            "/std:c++20",
             "/Zc:__cplusplus",
             "/Zc:preprocessor",
         ]

③ Technical Explanation

Significance of the ArrayRef.h Patch

In C++ overload resolution rules, an Exact Match (requiring no implicit conversions) always takes absolute precedence over candidates that require implicit user-defined conversions.
By defining the operator== template in the c10 namespace scope, the compiler can directly match comparisons of two ArrayRef<T> objects (such as IntArrayRef) without attempting to cast them to their base class (HeaderOnlyArrayRef) or wrapping them into another type. This prevents MSVC's ADL from flagging the comparison as ambiguous.

Significance of the setup.py Patch

Configuring the host compiler and nvcc to use c++20 standard options ensures that the entire compilation unit runs in C++20 mode. This enables native support for modern features used by PyTorch's backend, such as designated initializers ({.function = ...}) and in-class bitfield initializers, allowing compilation to proceed without syntax errors.

Assets 2
Loading