From 12b8e7ae64f809c8ce4a6f1d3bff0109990a75f2 Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Wed, 8 Oct 2025 00:25:57 +0000 Subject: [PATCH 01/11] [libc++] Optimize `std::exception_ptr` This commit optimizes the performance for `std::exception_ptr` for an empty exception objects. To do so, we use 3 high-level approaches: 1. Moving the implementation from the libc++ library into the libc++ headers, thereby allowing the compiler to inline the function bodies. 2. Adding fast paths to the (now inlineable) bodies, checking for the empty case. 3. Adding move constuctor, assignment and `swap` Those optimizations were implemented for the libc++abi, libsupc++ and libstdc++ ABIs. Fixes #XXX Performance ----------- With this change, the compiler can now completely constant-fold https://godbolt.org/z/NaNKe5. Also in cases where the compiler cannot statically prove that exception_ptr is empty, it can at least generate a fast-path check if the exception_ptr is empty, without calling into the library. ABI compatibility ----------------- We use a new visibility macro `_LIBCPP_EXPORTED_FROM_ABI_INLINEABLE` to ensure that the functions which are now declared in the header are still exported by the library. See the description in `VisibilityMacros.rst` for details. This approach was originally proposed by Nikolas Klauser in https://reviews.llvm.org/D122536 Moving implementations to the header ------------------------------------ To move the implementation to the header, we must know the selected LIBCXX_CXX_ABI also in the headers. For that purpose, the LIBCXX_CXX_ABI configuration is now exposed via `__config_site`. While the Microsoft ABI and the "unimplemented" APIs do not benefit from this optimizations, I also moved their implementation to the headers for more uniformity. Mid-term, we probably have to expose all implementations in the header, anyway, because P3068R6 mandates all methods of `exception_ptr` to be constexpr. Unifying libc++abi, libsupc++ and `none` ----------------------------------------------- Both libc++abi and libsupc++ are reference-counted. The primary difference is the function called for ref-counting: * libc++api uses `__cxa_{in,de}crement_exception_refcount` * libsupc++ uses `__exception_ptr::_M_{addref,release}` This commit factors out the common reference-counting logic into a shared header. Our libsupc++ implementation so far did not take advantage of `_M_addref`/`_M_release`. For the performance benefits of this PR it was necessary to start using them. The same reference-counted approach is also reused for the `none`/`unimplemented` implementation. This has the side effect, that users can now use empty `exception_ptr`s even in the `none` ABI. The abort is only triggered for non-empty `exception_ptr`s. Given this simplifies the code, I think change is acceptable. Unifying nested_exception ------------------------- The implementation for `nested_exception` was effectively identical across all ABIs. For most ABIs, the source code was literally identical. There were only two deviations: * For libsupc++ or libstdc++, we did not use define the `~nested_exception` destructor. * The abort in `nested_exception::rethrow_nested` was unnecessary as the same abort is also present in the `rethrow_exception` implementation. As such, we were able to simply remove that special casing. The implementation is now unified directly in the `__nested_exception.h` header. Standard conformance -------------------- The available constructors, operators and methods of `exception_ptr` is not specified by the standard. As such, adding the move constructor and the assignment operator are standard conformant. libstdc++ made the same decision and also provides those headers. --- libcxx/CMakeLists.txt | 15 ++ libcxx/cmake/Modules/HandleLibCXXABI.cmake | 4 - libcxx/docs/DesignDocs/VisibilityMacros.rst | 19 +++ libcxx/include/CMakeLists.txt | 4 + libcxx/include/__config_site.in | 8 ++ libcxx/include/__exception/exception_ptr.h | 129 +++++++++++++++--- .../__exception/exception_ptr_cxxabi.ipp | 50 +++++++ .../__exception/exception_ptr_glibcxx.ipp | 47 +++++++ .../__exception/exception_ptr_msvc.ipp} | 35 ++--- .../exception_ptr_unimplemented.ipp | 36 +++++ libcxx/include/__exception/nested_exception.h | 2 + libcxx/include/__exception/operations.h | 5 - libcxx/include/exception | 1 + libcxx/src/CMakeLists.txt | 4 - libcxx/src/exception.cpp | 44 +++--- libcxx/src/new_handler.cpp | 12 +- .../support/runtime/exception_libcxxabi.ipp | 9 +- .../support/runtime/exception_libcxxrt.ipp | 5 - .../runtime/exception_pointer_cxxabi.ipp | 64 --------- .../runtime/exception_pointer_glibcxx.ipp | 68 --------- .../exception_pointer_unimplemented.ipp | 62 --------- .../src/support/runtime/stdexcept_default.ipp | 2 +- libcxx/src/typeinfo.cpp | 6 +- .../test/benchmarks/exception_ptr.bench.cpp | 17 +++ libcxx/utils/libcxx/header_information.py | 4 + 25 files changed, 360 insertions(+), 292 deletions(-) create mode 100644 libcxx/include/__exception/exception_ptr_cxxabi.ipp create mode 100644 libcxx/include/__exception/exception_ptr_glibcxx.ipp rename libcxx/{src/support/runtime/exception_pointer_msvc.ipp => include/__exception/exception_ptr_msvc.ipp} (52%) create mode 100644 libcxx/include/__exception/exception_ptr_unimplemented.ipp delete mode 100644 libcxx/src/support/runtime/exception_pointer_cxxabi.ipp delete mode 100644 libcxx/src/support/runtime/exception_pointer_glibcxx.ipp delete mode 100644 libcxx/src/support/runtime/exception_pointer_unimplemented.ipp diff --git a/libcxx/CMakeLists.txt b/libcxx/CMakeLists.txt index a119850cd808e..248c83dd3b3b0 100644 --- a/libcxx/CMakeLists.txt +++ b/libcxx/CMakeLists.txt @@ -731,6 +731,7 @@ config_define(${LIBCXX_ABI_VERSION} _LIBCPP_ABI_VERSION) config_define(${LIBCXX_ABI_NAMESPACE} _LIBCPP_ABI_NAMESPACE) config_define(${LIBCXX_ABI_FORCE_ITANIUM} _LIBCPP_ABI_FORCE_ITANIUM) config_define(${LIBCXX_ABI_FORCE_MICROSOFT} _LIBCPP_ABI_FORCE_MICROSOFT) +config_define(${LIBCXX_CXX_ABI} _LIBCXX_CXX_ABI) config_define(${LIBCXX_ENABLE_THREADS} _LIBCPP_HAS_THREADS) config_define(${LIBCXX_ENABLE_MONOTONIC_CLOCK} _LIBCPP_HAS_MONOTONIC_CLOCK) config_define(${LIBCXX_HAS_TERMINAL_AVAILABLE} _LIBCPP_HAS_TERMINAL) @@ -750,6 +751,20 @@ config_define(${LIBCXX_ENABLE_WIDE_CHARACTERS} _LIBCPP_HAS_WIDE_CHARACTERS) config_define(${LIBCXX_ENABLE_TIME_ZONE_DATABASE} _LIBCPP_HAS_TIME_ZONE_DATABASE) config_define(${LIBCXX_ENABLE_VENDOR_AVAILABILITY_ANNOTATIONS} _LIBCPP_HAS_VENDOR_AVAILABILITY_ANNOTATIONS) +if (LIBCXX_CXX_ABI STREQUAL "none") + config_define(1 _LIBCPP_CXX_ABI_NONE) +elseif (LIBCXX_CXX_ABI STREQUAL "libcxxabi" OR LIBCXX_CXX_ABI STREQUAL "system-libcxxabi") + config_define(1 _LIBCPP_CXX_ABI_LIBCXXABI) +elseif (LIBCXX_CXX_ABI STREQUAL "libcxxrt") + config_define(1 _LIBCPP_CXX_ABI_LIBCXXRT) +elseif (LIBCXX_CXX_ABI STREQUAL "libstdc++") + config_define(1 _LIBCPP_CXX_ABI_LIBSTDCXX) +elseif (LIBCXX_CXX_ABI STREQUAL "libsupc++") + config_define(1 _LIBCPP_CXX_ABI_LIBSUPCXX) +elseif (LIBCXX_CXX_ABI STREQUAL "vcruntime") + config_define(1 _LIBCPP_CXX_ABI_VCRUNTIME) +endif() + # TODO: Remove in LLVM 21. We're leaving an error to make this fail explicitly. if (LIBCXX_ENABLE_ASSERTIONS) message(FATAL_ERROR "LIBCXX_ENABLE_ASSERTIONS has been removed. Please use LIBCXX_HARDENING_MODE instead.") diff --git a/libcxx/cmake/Modules/HandleLibCXXABI.cmake b/libcxx/cmake/Modules/HandleLibCXXABI.cmake index 52236f473f35d..35287cf380da6 100644 --- a/libcxx/cmake/Modules/HandleLibCXXABI.cmake +++ b/libcxx/cmake/Modules/HandleLibCXXABI.cmake @@ -119,7 +119,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "libsupc++") elseif ("${LIBCXX_CXX_ABI}" STREQUAL "libcxxabi") add_library(libcxx-abi-headers INTERFACE) target_link_libraries(libcxx-abi-headers INTERFACE cxxabi-headers) - target_compile_definitions(libcxx-abi-headers INTERFACE "-DLIBCXX_BUILDING_LIBCXXABI") if (TARGET cxxabi_shared) add_library(libcxx-abi-shared INTERFACE) @@ -156,7 +155,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "system-libcxxabi") add_library(libcxx-abi-headers INTERFACE) import_private_headers(libcxx-abi-headers "${LIBCXX_CXX_ABI_INCLUDE_PATHS}" "cxxabi.h;__cxxabi_config.h") - target_compile_definitions(libcxx-abi-headers INTERFACE "-DLIBCXX_BUILDING_LIBCXXABI") import_shared_library(libcxx-abi-shared c++abi) target_link_libraries(libcxx-abi-shared INTERFACE libcxx-abi-headers) @@ -173,7 +171,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "libcxxrt") add_library(libcxx-abi-headers INTERFACE) import_private_headers(libcxx-abi-headers "${LIBCXX_CXX_ABI_INCLUDE_PATHS}" "cxxabi.h;unwind.h;unwind-arm.h;unwind-itanium.h") - target_compile_definitions(libcxx-abi-headers INTERFACE "-DLIBCXXRT") import_shared_library(libcxx-abi-shared cxxrt) target_link_libraries(libcxx-abi-shared INTERFACE libcxx-abi-headers) @@ -191,7 +188,6 @@ elseif ("${LIBCXX_CXX_ABI}" STREQUAL "vcruntime") # Don't link against any ABI library elseif ("${LIBCXX_CXX_ABI}" STREQUAL "none") add_library(libcxx-abi-headers INTERFACE) - target_compile_definitions(libcxx-abi-headers INTERFACE "-D_LIBCPP_BUILDING_HAS_NO_ABI_LIBRARY") add_library(libcxx-abi-shared INTERFACE) target_link_libraries(libcxx-abi-shared INTERFACE libcxx-abi-headers) diff --git a/libcxx/docs/DesignDocs/VisibilityMacros.rst b/libcxx/docs/DesignDocs/VisibilityMacros.rst index db54b35386b19..79d26cd0c550d 100644 --- a/libcxx/docs/DesignDocs/VisibilityMacros.rst +++ b/libcxx/docs/DesignDocs/VisibilityMacros.rst @@ -35,6 +35,25 @@ Visibility Macros used on class templates. On classes it should only be used if the vtable lives in the built library. +**_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE** + Mark a symbol as exported from the libc++ library, while still providing an + inlineable definition that can be used by the compiler for optimization + purposes. + + To use this macro on a class method, define the method body + *outside* of the class definition and annotate that definition with + `_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE`. Make sure to include the + header in at least one translation unit linked into the libc++ library. + + This macro works by applying `[[gnu::gnu_inline]] inline` to the funciton + in the header, thereby suppressing code generation while still allowing the + compiler to use the function for optimization purposes. + During the build of libc++, we trigger code generation by not expanding the + macro to `_LIBCPP_EXPORTED_FROM_ABI`. Since the function is no longer marked + as `inline`, it will be emitted even if not called. (For this reason its + paramount to not define methods in the class definition, since those definitions + would be implicitly `inline`.) + **_LIBCPP_OVERRIDABLE_FUNC_VIS** Mark a symbol as being exported by the libc++ library, but allow it to be overridden locally. On non-Windows, this is equivalent to `_LIBCPP_FUNC_VIS`. diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt index ddace8bf8c728..2baf6fca70a5f 100644 --- a/libcxx/include/CMakeLists.txt +++ b/libcxx/include/CMakeLists.txt @@ -343,6 +343,10 @@ set(files __debug_utils/strict_weak_ordering_check.h __exception/exception.h __exception/exception_ptr.h + __exception/exception_ptr_cxxabi.ipp + __exception/exception_ptr_glibcxx.ipp + __exception/exception_ptr_msvc.ipp + __exception/exception_ptr_unimplemented.ipp __exception/nested_exception.h __exception/operations.h __exception/terminate.h diff --git a/libcxx/include/__config_site.in b/libcxx/include/__config_site.in index b68c0c8258366..58714ed231b88 100644 --- a/libcxx/include/__config_site.in +++ b/libcxx/include/__config_site.in @@ -33,6 +33,14 @@ #cmakedefine01 _LIBCPP_HAS_TIME_ZONE_DATABASE #cmakedefine01 _LIBCPP_INSTRUMENTED_WITH_ASAN +// LIBCXX_CXX_ABI backends +#cmakedefine _LIBCPP_CXX_ABI_NONE +#cmakedefine _LIBCPP_CXX_ABI_LIBCXXABI +#cmakedefine _LIBCPP_CXX_ABI_LIBCXXRT +#cmakedefine _LIBCPP_CXX_ABI_LIBSTDCXX +#cmakedefine _LIBCPP_CXX_ABI_LIBSUPCXX +#cmakedefine _LIBCPP_CXX_ABI_VCRUNTIME + // PSTL backends #cmakedefine _LIBCPP_PSTL_BACKEND_SERIAL #cmakedefine _LIBCPP_PSTL_BACKEND_STD_THREAD diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index 796fa924be121..e442e198aa0e3 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -23,9 +23,29 @@ # pragma GCC system_header #endif -#ifndef _LIBCPP_ABI_MICROSOFT +// Previously, parts of exception_ptr were defined out-of-line, which prevented +// useful compiler optimizations. Changing the out-of-line definitions to inline +// definitions is an ABI break, however. To prevent this, we have to make sure +// the symbols remain available in the libc++ library, in addition to being +// defined inline here in this header. +// To this end, we use _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE macro: +// The macro is defined as empty for src/exception.cpp, forcing the definitions of +// the functions to be emitted and included in the library. When users of libc++ +// compile their code, the __gnu_inline__ attribute will suppress generation of +// these functions while making their definitions available for inlining. +# ifdef _LIBCPP_EMIT_CODE_FOR_EXCEPTION_PTR +# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE _LIBCPP_EXPORTED_FROM_ABI +# else +# if !__has_cpp_attribute(gnu::gnu_inline) +# error "GNU inline attribute is not supported" +# endif +# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[gnu::gnu_inline]] inline +# endif -# if _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION +_LIBCPP_DIAGNOSTIC_PUSH +_LIBCPP_CLANG_DIAGNOSTIC_IGNORED("-Wgnu-inline-cpp-without-extern") + +#ifdef _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION namespace __cxxabiv1 { @@ -49,18 +69,25 @@ _LIBCPP_OVERRIDABLE_FUNC_VIS __cxa_exception* __cxa_init_primary_exception( } // namespace __cxxabiv1 -# endif - #endif _LIBCPP_BEGIN_UNVERSIONED_NAMESPACE_STD -#ifndef _LIBCPP_ABI_MICROSOFT +class _LIBCPP_EXPORTED_FROM_ABI exception_ptr; +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT; +[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr); + +#ifndef _LIBCPP_CXX_ABI_VCRUNTIME class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { void* __ptr_; - static exception_ptr __from_native_exception_pointer(void*) _NOEXCEPT; + // Customization points to adjust the reference counting for cxxabi or + // libsupc++/libstdc++ + _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE static void __increment_refcount(void* __ptr) _NOEXCEPT; + _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE static void __decrement_refcount(void* __ptr) _NOEXCEPT; + + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE static exception_ptr __from_native_exception_pointer(void*) _NOEXCEPT; template friend _LIBCPP_HIDE_FROM_ABI exception_ptr __make_exception_ptr_explicit(_Ep&) _NOEXCEPT; @@ -74,9 +101,11 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { _LIBCPP_HIDE_FROM_ABI exception_ptr() _NOEXCEPT : __ptr_() {} _LIBCPP_HIDE_FROM_ABI exception_ptr(nullptr_t) _NOEXCEPT : __ptr_() {} - exception_ptr(const exception_ptr&) _NOEXCEPT; - exception_ptr& operator=(const exception_ptr&) _NOEXCEPT; - ~exception_ptr() _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr(const exception_ptr&) _NOEXCEPT; + _LIBCPP_HIDE_FROM_ABI exception_ptr(exception_ptr&&) _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& operator=(const exception_ptr&) _NOEXCEPT; + _LIBCPP_HIDE_FROM_ABI exception_ptr& operator=(exception_ptr&&) _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE ~exception_ptr() _NOEXCEPT; _LIBCPP_HIDE_FROM_ABI explicit operator bool() const _NOEXCEPT { return __ptr_ != nullptr; } @@ -88,10 +117,53 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { return !(__x == __y); } + friend _LIBCPP_HIDE_FROM_ABI void swap(exception_ptr& __x, exception_ptr& __y) { + void* __tmp = __x.__ptr_; + __x.__ptr_ = __y.__ptr_; + __y.__ptr_ = __tmp; + } + friend _LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT; friend _LIBCPP_EXPORTED_FROM_ABI void rethrow_exception(exception_ptr); }; +// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr exception_ptr::__from_native_exception_pointer(void* __e) noexcept { + __increment_refcount(__e); + exception_ptr __ptr; + __ptr.__ptr_ = __e; + return __ptr; +} + +// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) noexcept : __ptr_(__other.__ptr_) { + __increment_refcount(__ptr_); +} + +_LIBCPP_HIDE_FROM_ABI inline exception_ptr::exception_ptr(exception_ptr&& __other) _NOEXCEPT : __ptr_(__other.__ptr_) { + __other.__ptr_ = nullptr; +} + +// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(const exception_ptr& __other) noexcept { + if (__ptr_ != __other.__ptr_) { + __increment_refcount(__other.__ptr_); + __decrement_refcount(__ptr_); + __ptr_ = __other.__ptr_; + } + return *this; +} + +_LIBCPP_HIDE_FROM_ABI inline exception_ptr& exception_ptr::operator=(exception_ptr&& __other) noexcept { + __decrement_refcount(__ptr_); + __ptr_ = __other.__ptr_; + __other.__ptr_ = nullptr; + return *this; +} + +// Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::~exception_ptr() noexcept { __decrement_refcount(__ptr_); } + # if _LIBCPP_HAS_EXCEPTIONS # if _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION template @@ -167,26 +239,26 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { _LIBCPP_DIAGNOSTIC_POP public: - exception_ptr() _NOEXCEPT; - exception_ptr(nullptr_t) _NOEXCEPT; - exception_ptr(const exception_ptr& __other) _NOEXCEPT; - exception_ptr& operator=(const exception_ptr& __other) _NOEXCEPT; - exception_ptr& operator=(nullptr_t) _NOEXCEPT; - ~exception_ptr() _NOEXCEPT; - explicit operator bool() const _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr() _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr(nullptr_t) _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr(const exception_ptr& __other) _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& operator=(const exception_ptr& __other) _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& operator=(nullptr_t) _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE ~exception_ptr() _NOEXCEPT; + _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE explicit operator bool() const _NOEXCEPT; }; -_LIBCPP_EXPORTED_FROM_ABI bool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT; +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE bool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT; inline _LIBCPP_HIDE_FROM_ABI bool operator!=(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT { return !(__x == __y); } -_LIBCPP_EXPORTED_FROM_ABI void swap(exception_ptr&, exception_ptr&) _NOEXCEPT; +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void swap(exception_ptr&, exception_ptr&) _NOEXCEPT; -_LIBCPP_EXPORTED_FROM_ABI exception_ptr __copy_exception_ptr(void* __except, const void* __ptr); -_LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT; -[[__noreturn__]] _LIBCPP_EXPORTED_FROM_ABI void rethrow_exception(exception_ptr); +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr __copy_exception_ptr(void* __except, const void* __ptr); +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT; +[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr); // This is a built-in template function which automagically extracts the required // information. @@ -199,6 +271,21 @@ _LIBCPP_HIDE_FROM_ABI exception_ptr make_exception_ptr(_Ep __e) _NOEXCEPT { } #endif // _LIBCPP_ABI_MICROSOFT + _LIBCPP_END_UNVERSIONED_NAMESPACE_STD +# if defined(_LIBCPP_CXX_ABI_NONE) +# include <__exception/exception_ptr_unimplemented.ipp> +# elif defined(_LIBCPP_CXX_ABI_LIBCXXABI) || defined(_LIBCPP_CXX_ABI_LIBCXXRT) +# include <__exception/exception_ptr_cxxabi.ipp> +# elif defined(_LIBCPP_CXX_ABI_LIBSTDCXX) || defined(_LIBCPP_CXX_ABI_LIBSUPCXX) +# include <__exception/exception_ptr_glibcxx.ipp> +# elif defined(_LIBCPP_CXX_ABI_VCRUNTIME) +# include <__exception/exception_ptr_msvc.ipp> +# else +# error "Unsupported C++ ABI library" +# endif + +_LIBCPP_DIAGNOSTIC_POP + #endif // _LIBCPP___EXCEPTION_EXCEPTION_PTR_H diff --git a/libcxx/include/__exception/exception_ptr_cxxabi.ipp b/libcxx/include/__exception/exception_ptr_cxxabi.ipp new file mode 100644 index 0000000000000..0ad94e3a6bddf --- /dev/null +++ b/libcxx/include/__exception/exception_ptr_cxxabi.ipp @@ -0,0 +1,50 @@ +// -*- C++ -*- +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include <__exception/terminate.h> + +namespace __cxxabiv1 { + +extern "C" { +_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_increment_exception_refcount(void*) noexcept; +_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_decrement_exception_refcount(void*) noexcept; +_LIBCPP_OVERRIDABLE_FUNC_VIS void* __cxa_current_primary_exception() noexcept; +_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_rethrow_primary_exception(void*); +} + +} // namespace __cxxabiv1 + +namespace std { + +_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__increment_refcount(void* __ptr) _NOEXCEPT { + if (__ptr) + __cxxabiv1::__cxa_increment_exception_refcount(__ptr); +} + +_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decrement_refcount(void* __ptr) _NOEXCEPT { + if (__ptr) + __cxxabiv1::__cxa_decrement_exception_refcount(__ptr); +} + +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT { + // It would be nicer if there was a constructor that took a ptr, then + // this whole function would be just: + // return exception_ptr(__cxa_current_primary_exception()); + exception_ptr __ptr; + __ptr.__ptr_ = __cxxabiv1::__cxa_current_primary_exception(); + return __ptr; +} + +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr __ptr) { + __cxxabiv1::__cxa_rethrow_primary_exception(__ptr.__ptr_); + // if __ptr.__ptr_ is NULL, above returns so we terminate. + terminate(); +} + +} // namespace std diff --git a/libcxx/include/__exception/exception_ptr_glibcxx.ipp b/libcxx/include/__exception/exception_ptr_glibcxx.ipp new file mode 100644 index 0000000000000..74e12f71587c3 --- /dev/null +++ b/libcxx/include/__exception/exception_ptr_glibcxx.ipp @@ -0,0 +1,47 @@ +// -*- C++ -*- +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +namespace std { + +// libsupc++ does not implement the dependent EH ABI and the functionality +// it uses to implement std::exception_ptr (which it declares as an alias of +// std::__exception_ptr::exception_ptr) is not directly exported to clients. So +// we have little choice but to hijack std::__exception_ptr::exception_ptr's +// (which fortunately has the same layout as our std::exception_ptr) copy +// constructor, assignment operator and destructor (which are part of its +// stable ABI), and its rethrow_exception(std::__exception_ptr::exception_ptr) +// function. +namespace __exception_ptr { + +struct exception_ptr { + void* __ptr_; + + void _M_addref() _GLIBCXX_USE_NOEXCEPT; + void _M_release() _GLIBCXX_USE_NOEXCEPT; +}; + +} // namespace __exception_ptr + +_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__increment_refcount(void* __ptr) _NOEXCEPT { + if (__ptr) + reinterpret_cast<__exception_ptr::exception_ptr*>(this)->_M_addref(); +} + +_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decrement_refcount(void* __ptr) _NOEXCEPT { + if (__ptr) + reinterpret_cast<__exception_ptr::exception_ptr*>(this)->_M_release(); +} + +[[noreturn]] void rethrow_exception(__exception_ptr::exception_ptr); + +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[noreturn]] void rethrow_exception(exception_ptr __ptr) { + rethrow_exception(reinterpret_cast<__exception_ptr::exception_ptr&>(__ptr)); +} + +} // namespace std diff --git a/libcxx/src/support/runtime/exception_pointer_msvc.ipp b/libcxx/include/__exception/exception_ptr_msvc.ipp similarity index 52% rename from libcxx/src/support/runtime/exception_pointer_msvc.ipp rename to libcxx/include/__exception/exception_ptr_msvc.ipp index 2be5136176e32..fcd426e7abd0a 100644 --- a/libcxx/src/support/runtime/exception_pointer_msvc.ipp +++ b/libcxx/include/__exception/exception_ptr_msvc.ipp @@ -7,7 +7,6 @@ // //===----------------------------------------------------------------------===// -#include #include _LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrCreate(void*); @@ -23,54 +22,44 @@ _LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrCopyException(void*, const void*, co namespace std { -exception_ptr::exception_ptr() noexcept { __ExceptionPtrCreate(this); } -exception_ptr::exception_ptr(nullptr_t) noexcept { __ExceptionPtrCreate(this); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr() _NOEXCEPT { __ExceptionPtrCreate(this); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(nullptr_t) _NOEXCEPT { __ExceptionPtrCreate(this); } -exception_ptr::exception_ptr(const exception_ptr& __other) noexcept { __ExceptionPtrCopy(this, &__other); } -exception_ptr& exception_ptr::operator=(const exception_ptr& __other) noexcept { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) _NOEXCEPT { __ExceptionPtrCopy(this, &__other); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(const exception_ptr& __other) _NOEXCEPT { __ExceptionPtrAssign(this, &__other); return *this; } -exception_ptr& exception_ptr::operator=(nullptr_t) noexcept { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLEexception_ptr& exception_ptr::operator=(nullptr_t) _NOEXCEPT { exception_ptr dummy; __ExceptionPtrAssign(this, &dummy); return *this; } -exception_ptr::~exception_ptr() noexcept { __ExceptionPtrDestroy(this); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::~exception_ptr() _NOEXCEPT { __ExceptionPtrDestroy(this); } -exception_ptr::operator bool() const noexcept { return __ExceptionPtrToBool(this); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::operator bool() const _NOEXCEPT { return __ExceptionPtrToBool(this); } -bool operator==(const exception_ptr& __x, const exception_ptr& __y) noexcept { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLEbool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT { return __ExceptionPtrCompare(&__x, &__y); } -void swap(exception_ptr& lhs, exception_ptr& rhs) noexcept { __ExceptionPtrSwap(&rhs, &lhs); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void swap(exception_ptr& lhs, exception_ptr& rhs) _NOEXCEPT { __ExceptionPtrSwap(&rhs, &lhs); } -exception_ptr __copy_exception_ptr(void* __except, const void* __ptr) { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr __copy_exception_ptr(void* __except, const void* __ptr) { exception_ptr __ret = nullptr; if (__ptr) __ExceptionPtrCopyException(&__ret, __except, __ptr); return __ret; } -exception_ptr current_exception() noexcept { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLEexception_ptr current_exception() _NOEXCEPT { exception_ptr __ret; __ExceptionPtrCurrentException(&__ret); return __ret; } -[[noreturn]] void rethrow_exception(exception_ptr p) { __ExceptionPtrRethrow(&p); } - -nested_exception::nested_exception() noexcept : __ptr_(current_exception()) {} - -nested_exception::~nested_exception() noexcept {} - -[[noreturn]] void nested_exception::rethrow_nested() const { - if (__ptr_ == nullptr) - terminate(); - rethrow_exception(__ptr_); -} +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[noreturn]] void rethrow_exception(exception_ptr __ptr) { __ExceptionPtrRethrow(&__ptr); } } // namespace std diff --git a/libcxx/include/__exception/exception_ptr_unimplemented.ipp b/libcxx/include/__exception/exception_ptr_unimplemented.ipp new file mode 100644 index 0000000000000..716e5143f374a --- /dev/null +++ b/libcxx/include/__exception/exception_ptr_unimplemented.ipp @@ -0,0 +1,36 @@ +// -*- C++ -*- +//===----------------------------------------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include <__verbose_abort> + +namespace std { + +#ifdef _LIBCPP_BUILDING_LIBCXXABI +# warning exception_ptr not yet implemented +#endif + +_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__increment_refcount(void* __ptr) _NOEXCEPT { + if (__ptr) + __libcpp_verbose_abort("exception_ptr not yet implemented\n"); +} + +_LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decrement_refcount(void* __ptr) _NOEXCEPT { + if (__ptr) + __libcpp_verbose_abort("exception_ptr not yet implemented\n"); +} + +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT { + __libcpp_verbose_abort("exception_ptr not yet implemented\n"); +} + +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[noreturn]] void rethrow_exception(exception_ptr p) { + __libcpp_verbose_abort("exception_ptr not yet implemented\n"); +} + +} // namespace std diff --git a/libcxx/include/__exception/nested_exception.h b/libcxx/include/__exception/nested_exception.h index 90b14158d57a2..38689b810bb15 100644 --- a/libcxx/include/__exception/nested_exception.h +++ b/libcxx/include/__exception/nested_exception.h @@ -35,7 +35,9 @@ class _LIBCPP_EXPORTED_FROM_ABI nested_exception { public: nested_exception() _NOEXCEPT; _LIBCPP_HIDE_FROM_ABI nested_exception(const nested_exception&) _NOEXCEPT = default; + _LIBCPP_HIDE_FROM_ABI nested_exception(nested_exception&&) _NOEXCEPT = default; _LIBCPP_HIDE_FROM_ABI nested_exception& operator=(const nested_exception&) _NOEXCEPT = default; + _LIBCPP_HIDE_FROM_ABI nested_exception& operator=(nested_exception&&) _NOEXCEPT = default; virtual ~nested_exception() _NOEXCEPT; // access functions diff --git a/libcxx/include/__exception/operations.h b/libcxx/include/__exception/operations.h index 29d5c698a96db..9b17ab4373bcc 100644 --- a/libcxx/include/__exception/operations.h +++ b/libcxx/include/__exception/operations.h @@ -32,11 +32,6 @@ _LIBCPP_EXPORTED_FROM_ABI terminate_handler get_terminate() _NOEXCEPT; _LIBCPP_EXPORTED_FROM_ABI _LIBCPP_DEPRECATED_IN_CXX17 bool uncaught_exception() _NOEXCEPT; #endif // _LIBCPP_STD_VER <= 17 || defined(_LIBCPP_ENABLE_CXX20_REMOVED_UNCAUGHT_EXCEPTION) _LIBCPP_EXPORTED_FROM_ABI int uncaught_exceptions() _NOEXCEPT; - -class _LIBCPP_EXPORTED_FROM_ABI exception_ptr; - -_LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT; -[[__noreturn__]] _LIBCPP_EXPORTED_FROM_ABI void rethrow_exception(exception_ptr); _LIBCPP_END_UNVERSIONED_NAMESPACE_STD #endif // _LIBCPP___EXCEPTION_OPERATIONS_H diff --git a/libcxx/include/exception b/libcxx/include/exception index 74229cd16c006..c3e35583f1b3d 100644 --- a/libcxx/include/exception +++ b/libcxx/include/exception @@ -97,6 +97,7 @@ template void rethrow_if_nested(const E& e); # include # include # endif + #endif // __cplusplus < 201103L && defined(_LIBCPP_USE_FROZEN_CXX03_HEADERS) #endif // _LIBCPP_EXCEPTION diff --git a/libcxx/src/CMakeLists.txt b/libcxx/src/CMakeLists.txt index f59fe0e08fccb..5f7a534e43794 100644 --- a/libcxx/src/CMakeLists.txt +++ b/libcxx/src/CMakeLists.txt @@ -47,10 +47,6 @@ set(LIBCXX_SOURCES support/runtime/exception_libcxxabi.ipp support/runtime/exception_libcxxrt.ipp support/runtime/exception_msvc.ipp - support/runtime/exception_pointer_cxxabi.ipp - support/runtime/exception_pointer_glibcxx.ipp - support/runtime/exception_pointer_msvc.ipp - support/runtime/exception_pointer_unimplemented.ipp support/runtime/stdexcept_default.ipp support/runtime/stdexcept_vcruntime.ipp system_error.cpp diff --git a/libcxx/src/exception.cpp b/libcxx/src/exception.cpp index ac6324cd9fe35..3ab6c1c2b42f3 100644 --- a/libcxx/src/exception.cpp +++ b/libcxx/src/exception.cpp @@ -8,31 +8,39 @@ #define _LIBCPP_ENABLE_CXX20_REMOVED_UNCAUGHT_EXCEPTION #define _LIBCPP_DISABLE_DEPRECATION_WARNINGS +#define _LIBCPP_EMIT_CODE_FOR_EXCEPTION_PTR #include #include #include -#if defined(LIBCXXRT) || defined(LIBCXX_BUILDING_LIBCXXABI) -# include -using namespace __cxxabiv1; -# define HAVE_DEPENDENT_EH_ABI 1 -#endif - -#if defined(_LIBCPP_ABI_MICROSOFT) -# include "support/runtime/exception_msvc.ipp" -# include "support/runtime/exception_pointer_msvc.ipp" -#elif defined(_LIBCPPABI_VERSION) +#if defined(_LIBCPP_CXX_ABI_NONE) +# include "include/atomic_support.h" +# include "support/runtime/exception_fallback.ipp" +#elif defined(_LIBCPP_CXX_ABI_LIBCXXABI) # include "support/runtime/exception_libcxxabi.ipp" -# include "support/runtime/exception_pointer_cxxabi.ipp" -#elif defined(LIBCXXRT) +#elif defined(_LIBCPP_CXX_ABI_LIBCXXRT) # include "support/runtime/exception_libcxxrt.ipp" -# include "support/runtime/exception_pointer_cxxabi.ipp" -#elif defined(__GLIBCXX__) +#elif defined(_LIBCPP_CXX_ABI_LIBSTDCXX) || defined(_LIBCPP_CXX_ABI_LIBSUPCXX) # include "support/runtime/exception_glibcxx.ipp" -# include "support/runtime/exception_pointer_glibcxx.ipp" +#elif defined(_LIBCPP_CXX_ABI_VCRUNTIME) +# include "support/runtime/exception_msvc.ipp" #else -# include "include/atomic_support.h" -# include "support/runtime/exception_fallback.ipp" -# include "support/runtime/exception_pointer_unimplemented.ipp" +# error "Unsupported C++ ABI library" #endif + +namespace std { + +nested_exception::nested_exception() noexcept : __ptr_(current_exception()) {} + +#if !defined(_LIBCPP_CXX_ABI_LIBSTDCXX) && !defined(_LIBCPP_CXX_ABI_LIBSUPCXX) +nested_exception::~nested_exception() noexcept {} +#endif + +void nested_exception::rethrow_nested() const { + if (__ptr_ == nullptr) + terminate(); + rethrow_exception(__ptr_); +} + +} diff --git a/libcxx/src/new_handler.cpp b/libcxx/src/new_handler.cpp index 49c21d85f5908..b803521545137 100644 --- a/libcxx/src/new_handler.cpp +++ b/libcxx/src/new_handler.cpp @@ -10,15 +10,13 @@ #include "include/atomic_support.h" -#if defined(_LIBCPP_ABI_MICROSOFT) -# if !defined(_LIBCPP_ABI_VCRUNTIME) -# define _LIBPCPP_DEFINE_NEW_HANDLER -# endif -#elif defined(LIBCXX_BUILDING_LIBCXXABI) +#if defined(_LIBCPP_CXX_ABI_VCRUNTIME) +// nothing to do, we use the one from the VCRuntime +#elif defined(_LIBCPP_CXX_ABI_LIBCXXABI) // nothing to do, we use the one from libc++abi -#elif defined(LIBCXXRT) +#elif defined(_LIBCPP_CXX_ABI_CXXRT) # define _LIBPCPP_DEFINE_NEW_HANDLER -#elif defined(__GLIBCXX__) +#elif defined(_LIBCPP_CXX_ABI_LIBSTDCXX) || defined(_LIBCPP_CXX_ABI_LIBSUPCXX) // nothing to do, we use the one from libstdc++/libsupc++ #else # define _LIBPCPP_DEFINE_NEW_HANDLER diff --git a/libcxx/src/support/runtime/exception_libcxxabi.ipp b/libcxx/src/support/runtime/exception_libcxxabi.ipp index df6bd6574bde2..49d6057be2cc4 100644 --- a/libcxx/src/support/runtime/exception_libcxxabi.ipp +++ b/libcxx/src/support/runtime/exception_libcxxabi.ipp @@ -7,19 +7,16 @@ // //===----------------------------------------------------------------------===// -#ifndef _LIBCPPABI_VERSION -# error this header can only be used with libc++abi -#endif - +#include namespace std { bool uncaught_exception() noexcept { return uncaught_exceptions() > 0; } int uncaught_exceptions() noexcept { #if _LIBCPPABI_VERSION > 1001 - return __cxa_uncaught_exceptions(); + return __cxxabiv1::__cxa_uncaught_exceptions(); #else - return __cxa_uncaught_exception() ? 1 : 0; + return __cxxabiv1::__cxa_uncaught_exception() ? 1 : 0; #endif } diff --git a/libcxx/src/support/runtime/exception_libcxxrt.ipp b/libcxx/src/support/runtime/exception_libcxxrt.ipp index f17fecc71e34b..3629a456a78f4 100644 --- a/libcxx/src/support/runtime/exception_libcxxrt.ipp +++ b/libcxx/src/support/runtime/exception_libcxxrt.ipp @@ -6,11 +6,6 @@ // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // //===----------------------------------------------------------------------===// - -#ifndef LIBCXXRT -# error this header may only be used when targeting libcxxrt -#endif - namespace std { bad_exception::~bad_exception() noexcept {} diff --git a/libcxx/src/support/runtime/exception_pointer_cxxabi.ipp b/libcxx/src/support/runtime/exception_pointer_cxxabi.ipp deleted file mode 100644 index 8f5c2060bb06c..0000000000000 --- a/libcxx/src/support/runtime/exception_pointer_cxxabi.ipp +++ /dev/null @@ -1,64 +0,0 @@ -// -*- C++ -*- -//===----------------------------------------------------------------------===// -// -// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. -// See https://llvm.org/LICENSE.txt for license information. -// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception -// -//===----------------------------------------------------------------------===// - -#ifndef HAVE_DEPENDENT_EH_ABI -# error this header may only be used with libc++abi or libcxxrt -#endif - -namespace std { - -exception_ptr::~exception_ptr() noexcept { __cxa_decrement_exception_refcount(__ptr_); } - -exception_ptr::exception_ptr(const exception_ptr& other) noexcept : __ptr_(other.__ptr_) { - __cxa_increment_exception_refcount(__ptr_); -} - -exception_ptr& exception_ptr::operator=(const exception_ptr& other) noexcept { - if (__ptr_ != other.__ptr_) { - __cxa_increment_exception_refcount(other.__ptr_); - __cxa_decrement_exception_refcount(__ptr_); - __ptr_ = other.__ptr_; - } - return *this; -} - -exception_ptr exception_ptr::__from_native_exception_pointer(void* __e) noexcept { - exception_ptr ptr; - ptr.__ptr_ = __e; - __cxa_increment_exception_refcount(ptr.__ptr_); - - return ptr; -} - -nested_exception::nested_exception() noexcept : __ptr_(current_exception()) {} - -nested_exception::~nested_exception() noexcept {} - -void nested_exception::rethrow_nested() const { - if (__ptr_ == nullptr) - terminate(); - rethrow_exception(__ptr_); -} - -exception_ptr current_exception() noexcept { - // be nicer if there was a constructor that took a ptr, then - // this whole function would be just: - // return exception_ptr(__cxa_current_primary_exception()); - exception_ptr ptr; - ptr.__ptr_ = __cxa_current_primary_exception(); - return ptr; -} - -void rethrow_exception(exception_ptr p) { - __cxa_rethrow_primary_exception(p.__ptr_); - // if p.__ptr_ is NULL, above returns so we terminate - terminate(); -} - -} // namespace std diff --git a/libcxx/src/support/runtime/exception_pointer_glibcxx.ipp b/libcxx/src/support/runtime/exception_pointer_glibcxx.ipp deleted file mode 100644 index 174b44ce0e6f7..0000000000000 --- a/libcxx/src/support/runtime/exception_pointer_glibcxx.ipp +++ /dev/null @@ -1,68 +0,0 @@ -// -*- C++ -*- -//===----------------------------------------------------------------------===// -// -// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. -// See https://llvm.org/LICENSE.txt for license information. -// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception -// -//===----------------------------------------------------------------------===// - -// libsupc++ does not implement the dependent EH ABI and the functionality -// it uses to implement std::exception_ptr (which it declares as an alias of -// std::__exception_ptr::exception_ptr) is not directly exported to clients. So -// we have little choice but to hijack std::__exception_ptr::exception_ptr's -// (which fortunately has the same layout as our std::exception_ptr) copy -// constructor, assignment operator and destructor (which are part of its -// stable ABI), and its rethrow_exception(std::__exception_ptr::exception_ptr) -// function. - -namespace std { - -namespace __exception_ptr { - -struct exception_ptr { - void* __ptr_; - - explicit exception_ptr(void*) noexcept; - exception_ptr(const exception_ptr&) noexcept; - exception_ptr& operator=(const exception_ptr&) noexcept; - ~exception_ptr() noexcept; -}; - -} // namespace __exception_ptr - -[[noreturn]] void rethrow_exception(__exception_ptr::exception_ptr); - -exception_ptr::~exception_ptr() noexcept { reinterpret_cast<__exception_ptr::exception_ptr*>(this)->~exception_ptr(); } - -exception_ptr::exception_ptr(const exception_ptr& other) noexcept : __ptr_(other.__ptr_) { - new (reinterpret_cast(this)) - __exception_ptr::exception_ptr(reinterpret_cast(other)); -} - -exception_ptr& exception_ptr::operator=(const exception_ptr& other) noexcept { - *reinterpret_cast<__exception_ptr::exception_ptr*>(this) = - reinterpret_cast(other); - return *this; -} - -exception_ptr exception_ptr::__from_native_exception_pointer(void* __e) noexcept { - exception_ptr ptr{}; - new (reinterpret_cast(&ptr)) __exception_ptr::exception_ptr(__e); - - return ptr; -} - -nested_exception::nested_exception() noexcept : __ptr_(current_exception()) {} - -[[noreturn]] void nested_exception::rethrow_nested() const { - if (__ptr_ == nullptr) - terminate(); - rethrow_exception(__ptr_); -} - -[[noreturn]] void rethrow_exception(exception_ptr p) { - rethrow_exception(reinterpret_cast<__exception_ptr::exception_ptr&>(p)); -} - -} // namespace std diff --git a/libcxx/src/support/runtime/exception_pointer_unimplemented.ipp b/libcxx/src/support/runtime/exception_pointer_unimplemented.ipp deleted file mode 100644 index 05a71ce34e5ac..0000000000000 --- a/libcxx/src/support/runtime/exception_pointer_unimplemented.ipp +++ /dev/null @@ -1,62 +0,0 @@ -// -*- C++ -*- -//===----------------------------------------------------------------------===// -// -// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. -// See https://llvm.org/LICENSE.txt for license information. -// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception -// -//===----------------------------------------------------------------------===// - -#include <__verbose_abort> - -namespace std { - -exception_ptr::~exception_ptr() noexcept { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -} - -exception_ptr::exception_ptr(const exception_ptr& other) noexcept : __ptr_(other.__ptr_) { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -} - -exception_ptr& exception_ptr::operator=(const exception_ptr& other) noexcept { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -} - -exception_ptr exception_ptr::__from_native_exception_pointer(void *__e) noexcept { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -} - -nested_exception::nested_exception() noexcept : __ptr_(current_exception()) {} - -#if !defined(__GLIBCXX__) - -nested_exception::~nested_exception() noexcept {} - -#endif - -[[noreturn]] void nested_exception::rethrow_nested() const { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -#if 0 - if (__ptr_ == nullptr) - terminate(); - rethrow_exception(__ptr_); -#endif // FIXME -} - -exception_ptr current_exception() noexcept { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -} - -[[noreturn]] void rethrow_exception(exception_ptr p) { -#warning exception_ptr not yet implemented - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); -} - -} // namespace std diff --git a/libcxx/src/support/runtime/stdexcept_default.ipp b/libcxx/src/support/runtime/stdexcept_default.ipp index 1f47a0325d76b..0786ac532d8e4 100644 --- a/libcxx/src/support/runtime/stdexcept_default.ipp +++ b/libcxx/src/support/runtime/stdexcept_default.ipp @@ -9,7 +9,7 @@ #include "../../include/refstring.h" /* For _LIBCPPABI_VERSION */ -#if !defined(_LIBCPP_BUILDING_HAS_NO_ABI_LIBRARY) && (defined(LIBCXX_BUILDING_LIBCXXABI) || defined(LIBCXXRT)) +#if !defined(_LIBCPP_CXX_ABI_NONE) && (defined(LIBCPP_CXX_ABI_LIBCXXABI) || defined(LIBCPP_CXX_ABI_LIBCXXRT)) # include #endif diff --git a/libcxx/src/typeinfo.cpp b/libcxx/src/typeinfo.cpp index e5f59da31cffa..9c1a8ed389884 100644 --- a/libcxx/src/typeinfo.cpp +++ b/libcxx/src/typeinfo.cpp @@ -46,9 +46,7 @@ size_t std::type_info::hash_code() const noexcept { } #endif // _LIBCPP_ABI_MICROSOFT -// FIXME: Remove the _LIBCPP_BUILDING_HAS_NO_ABI_LIBRARY configuration. -#if (!defined(LIBCXX_BUILDING_LIBCXXABI) && !defined(LIBCXXRT) && !defined(__GLIBCXX__) && \ - !defined(_LIBCPP_ABI_VCRUNTIME)) || \ - defined(_LIBCPP_BUILDING_HAS_NO_ABI_LIBRARY) +// FIXME: Remove the _LIBCPP_CXX_ABI_NONE configuration. +#ifdef _LIBCPP_CXX_ABI_NONE std::type_info::~type_info() {} #endif diff --git a/libcxx/test/benchmarks/exception_ptr.bench.cpp b/libcxx/test/benchmarks/exception_ptr.bench.cpp index 7791c510b1eb6..6a4210bf65f57 100644 --- a/libcxx/test/benchmarks/exception_ptr.bench.cpp +++ b/libcxx/test/benchmarks/exception_ptr.bench.cpp @@ -18,4 +18,21 @@ void bm_make_exception_ptr(benchmark::State& state) { } BENCHMARK(bm_make_exception_ptr)->ThreadRange(1, 8); + +void bm_empty_exception_ptr(benchmark::State& state) { + for (auto _ : state) { + // All of the following operations are no-ops because + // the exception_ptr is empty. Hence, the compiler should + // be able to optimize them very aggressively. + std::exception_ptr p1; + std::exception_ptr p2 (p1); + std::exception_ptr p3 (std::move(p2)); + p2 = std::move(p1); + p1 = p2; + swap(p1, p2); + benchmark::DoNotOptimize(p1 == nullptr && nullptr == p2 && p1 == p2); + } +} +BENCHMARK(bm_empty_exception_ptr); + BENCHMARK_MAIN(); diff --git a/libcxx/utils/libcxx/header_information.py b/libcxx/utils/libcxx/header_information.py index d06271a7908cc..7daa399420d3b 100644 --- a/libcxx/utils/libcxx/header_information.py +++ b/libcxx/utils/libcxx/header_information.py @@ -113,6 +113,10 @@ def is_in_modulemap(self) -> bool: if self._name in ["cxxabi.h", "__cxxabi_config.h"]: return False + # exclude .ipp files - these are implementation details included by other headers + if self._name.endswith(".ipp"): + return False + # exclude headers in __support/ - these aren't supposed to work everywhere, # so they shouldn't be included in general if self._name.startswith("__support/"): From 02ab3cb054bb4fcfc9a39e012cd42f5a69439b72 Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 04:49:34 +0000 Subject: [PATCH 02/11] Format code --- libcxx/include/__exception/exception_ptr.h | 45 ++++++++++--------- libcxx/src/exception.cpp | 2 +- .../test/benchmarks/exception_ptr.bench.cpp | 5 +-- 3 files changed, 26 insertions(+), 26 deletions(-) diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index e442e198aa0e3..e7a1c7278005a 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -33,14 +33,14 @@ // the functions to be emitted and included in the library. When users of libc++ // compile their code, the __gnu_inline__ attribute will suppress generation of // these functions while making their definitions available for inlining. -# ifdef _LIBCPP_EMIT_CODE_FOR_EXCEPTION_PTR -# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE _LIBCPP_EXPORTED_FROM_ABI -# else -# if !__has_cpp_attribute(gnu::gnu_inline) -# error "GNU inline attribute is not supported" -# endif -# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[gnu::gnu_inline]] inline +#ifdef _LIBCPP_EMIT_CODE_FOR_EXCEPTION_PTR +# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE _LIBCPP_EXPORTED_FROM_ABI +#else +# if !__has_cpp_attribute(gnu::gnu_inline) +# error "GNU inline attribute is not supported" # endif +# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[gnu::gnu_inline]] inline +#endif _LIBCPP_DIAGNOSTIC_PUSH _LIBCPP_CLANG_DIAGNOSTIC_IGNORED("-Wgnu-inline-cpp-without-extern") @@ -119,8 +119,8 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { friend _LIBCPP_HIDE_FROM_ABI void swap(exception_ptr& __x, exception_ptr& __y) { void* __tmp = __x.__ptr_; - __x.__ptr_ = __y.__ptr_; - __y.__ptr_ = __tmp; + __x.__ptr_ = __y.__ptr_; + __y.__ptr_ = __tmp; } friend _LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT; @@ -136,7 +136,8 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr exception_ptr::__from_native_ } // Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) noexcept : __ptr_(__other.__ptr_) { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) noexcept + : __ptr_(__other.__ptr_) { __increment_refcount(__ptr_); } @@ -156,7 +157,7 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(con _LIBCPP_HIDE_FROM_ABI inline exception_ptr& exception_ptr::operator=(exception_ptr&& __other) noexcept { __decrement_refcount(__ptr_); - __ptr_ = __other.__ptr_; + __ptr_ = __other.__ptr_; __other.__ptr_ = nullptr; return *this; } @@ -274,17 +275,17 @@ _LIBCPP_HIDE_FROM_ABI exception_ptr make_exception_ptr(_Ep __e) _NOEXCEPT { _LIBCPP_END_UNVERSIONED_NAMESPACE_STD -# if defined(_LIBCPP_CXX_ABI_NONE) -# include <__exception/exception_ptr_unimplemented.ipp> -# elif defined(_LIBCPP_CXX_ABI_LIBCXXABI) || defined(_LIBCPP_CXX_ABI_LIBCXXRT) -# include <__exception/exception_ptr_cxxabi.ipp> -# elif defined(_LIBCPP_CXX_ABI_LIBSTDCXX) || defined(_LIBCPP_CXX_ABI_LIBSUPCXX) -# include <__exception/exception_ptr_glibcxx.ipp> -# elif defined(_LIBCPP_CXX_ABI_VCRUNTIME) -# include <__exception/exception_ptr_msvc.ipp> -# else -# error "Unsupported C++ ABI library" -# endif +#if defined(_LIBCPP_CXX_ABI_NONE) +# include <__exception/exception_ptr_unimplemented.ipp> +#elif defined(_LIBCPP_CXX_ABI_LIBCXXABI) || defined(_LIBCPP_CXX_ABI_LIBCXXRT) +# include <__exception/exception_ptr_cxxabi.ipp> +#elif defined(_LIBCPP_CXX_ABI_LIBSTDCXX) || defined(_LIBCPP_CXX_ABI_LIBSUPCXX) +# include <__exception/exception_ptr_glibcxx.ipp> +#elif defined(_LIBCPP_CXX_ABI_VCRUNTIME) +# include <__exception/exception_ptr_msvc.ipp> +#else +# error "Unsupported C++ ABI library" +#endif _LIBCPP_DIAGNOSTIC_POP diff --git a/libcxx/src/exception.cpp b/libcxx/src/exception.cpp index 3ab6c1c2b42f3..b0e4e7557f235 100644 --- a/libcxx/src/exception.cpp +++ b/libcxx/src/exception.cpp @@ -43,4 +43,4 @@ void nested_exception::rethrow_nested() const { rethrow_exception(__ptr_); } -} +} // namespace std diff --git a/libcxx/test/benchmarks/exception_ptr.bench.cpp b/libcxx/test/benchmarks/exception_ptr.bench.cpp index 6a4210bf65f57..b3fdf474d5cef 100644 --- a/libcxx/test/benchmarks/exception_ptr.bench.cpp +++ b/libcxx/test/benchmarks/exception_ptr.bench.cpp @@ -18,15 +18,14 @@ void bm_make_exception_ptr(benchmark::State& state) { } BENCHMARK(bm_make_exception_ptr)->ThreadRange(1, 8); - void bm_empty_exception_ptr(benchmark::State& state) { for (auto _ : state) { // All of the following operations are no-ops because // the exception_ptr is empty. Hence, the compiler should // be able to optimize them very aggressively. std::exception_ptr p1; - std::exception_ptr p2 (p1); - std::exception_ptr p3 (std::move(p2)); + std::exception_ptr p2(p1); + std::exception_ptr p3(std::move(p2)); p2 = std::move(p1); p1 = p2; swap(p1, p2); From 4661785bef428438dae4aed2a8582456045c8209 Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 04:49:34 +0000 Subject: [PATCH 03/11] Fix accidentally exposed symbols --- libcxx/src/support/runtime/stdexcept_default.ipp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libcxx/src/support/runtime/stdexcept_default.ipp b/libcxx/src/support/runtime/stdexcept_default.ipp index 0786ac532d8e4..c51be0b60c79f 100644 --- a/libcxx/src/support/runtime/stdexcept_default.ipp +++ b/libcxx/src/support/runtime/stdexcept_default.ipp @@ -9,7 +9,7 @@ #include "../../include/refstring.h" /* For _LIBCPPABI_VERSION */ -#if !defined(_LIBCPP_CXX_ABI_NONE) && (defined(LIBCPP_CXX_ABI_LIBCXXABI) || defined(LIBCPP_CXX_ABI_LIBCXXRT)) +#if defined(_LIBCPP_CXX_ABI_LIBCXXABI) || defined(_LIBCPP_CXX_ABI_LIBCXXRT) # include #endif From 3c8f64d271983081f185c3ded40d0fc2c42a4bf1 Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 05:37:54 +0000 Subject: [PATCH 04/11] Uglify attribute --- libcxx/include/__exception/exception_ptr.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index e7a1c7278005a..61b5e7fc607cf 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -36,10 +36,10 @@ #ifdef _LIBCPP_EMIT_CODE_FOR_EXCEPTION_PTR # define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE _LIBCPP_EXPORTED_FROM_ABI #else -# if !__has_cpp_attribute(gnu::gnu_inline) +# if !__has_cpp_attribute(__gnu__::__gnu_inline__) # error "GNU inline attribute is not supported" # endif -# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[gnu::gnu_inline]] inline +# define _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[__gnu__::__gnu_inline__]] inline #endif _LIBCPP_DIAGNOSTIC_PUSH From f79298326b556ea8d70b8c95d18353a06532adcc Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 05:51:08 +0000 Subject: [PATCH 05/11] Adjust clang-tidy check --- libcxx/test/tools/clang_tidy_checks/hide_from_abi.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libcxx/test/tools/clang_tidy_checks/hide_from_abi.cpp b/libcxx/test/tools/clang_tidy_checks/hide_from_abi.cpp index 38bf62019599e..3740ffb7f7149 100644 --- a/libcxx/test/tools/clang_tidy_checks/hide_from_abi.cpp +++ b/libcxx/test/tools/clang_tidy_checks/hide_from_abi.cpp @@ -23,7 +23,8 @@ hide_from_abi::hide_from_abi(llvm::StringRef name, clang::tidy::ClangTidyContext void hide_from_abi::registerMatchers(clang::ast_matchers::MatchFinder* finder) { using namespace clang::ast_matchers; - auto has_hide_from_abi_attr = anyOf(hasAttr(clang::attr::Visibility), hasAttr(clang::attr::AbiTag)); + auto has_hide_from_abi_attr = + anyOf(hasAttr(clang::attr::Visibility), hasAttr(clang::attr::AbiTag), hasAttr(clang::attr::GNUInline)); finder->addMatcher( functionDecl( From 186bbd1d8c50d8f404a5dc708d8f9d3ebd13e5c8 Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 07:00:13 +0000 Subject: [PATCH 06/11] Uglify noexcept --- libcxx/include/__exception/exception_ptr.h | 10 +++++----- libcxx/include/__exception/exception_ptr_cxxabi.ipp | 6 +++--- libcxx/include/exception | 1 - 3 files changed, 8 insertions(+), 9 deletions(-) diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index 61b5e7fc607cf..003182a6f02b6 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -128,7 +128,7 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { }; // Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr exception_ptr::__from_native_exception_pointer(void* __e) noexcept { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr exception_ptr::__from_native_exception_pointer(void* __e) _NOEXCEPT { __increment_refcount(__e); exception_ptr __ptr; __ptr.__ptr_ = __e; @@ -136,7 +136,7 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr exception_ptr::__from_native_ } // Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) noexcept +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::exception_ptr(const exception_ptr& __other) _NOEXCEPT : __ptr_(__other.__ptr_) { __increment_refcount(__ptr_); } @@ -146,7 +146,7 @@ _LIBCPP_HIDE_FROM_ABI inline exception_ptr::exception_ptr(exception_ptr&& __othe } // Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(const exception_ptr& __other) noexcept { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(const exception_ptr& __other) _NOEXCEPT { if (__ptr_ != __other.__ptr_) { __increment_refcount(__other.__ptr_); __decrement_refcount(__ptr_); @@ -155,7 +155,7 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(con return *this; } -_LIBCPP_HIDE_FROM_ABI inline exception_ptr& exception_ptr::operator=(exception_ptr&& __other) noexcept { +_LIBCPP_HIDE_FROM_ABI inline exception_ptr& exception_ptr::operator=(exception_ptr&& __other) _NOEXCEPT { __decrement_refcount(__ptr_); __ptr_ = __other.__ptr_; __other.__ptr_ = nullptr; @@ -163,7 +163,7 @@ _LIBCPP_HIDE_FROM_ABI inline exception_ptr& exception_ptr::operator=(exception_p } // Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::~exception_ptr() noexcept { __decrement_refcount(__ptr_); } +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::~exception_ptr() _NOEXCEPT { __decrement_refcount(__ptr_); } # if _LIBCPP_HAS_EXCEPTIONS # if _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION diff --git a/libcxx/include/__exception/exception_ptr_cxxabi.ipp b/libcxx/include/__exception/exception_ptr_cxxabi.ipp index 0ad94e3a6bddf..187a6e0ad94c9 100644 --- a/libcxx/include/__exception/exception_ptr_cxxabi.ipp +++ b/libcxx/include/__exception/exception_ptr_cxxabi.ipp @@ -12,9 +12,9 @@ namespace __cxxabiv1 { extern "C" { -_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_increment_exception_refcount(void*) noexcept; -_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_decrement_exception_refcount(void*) noexcept; -_LIBCPP_OVERRIDABLE_FUNC_VIS void* __cxa_current_primary_exception() noexcept; +_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_increment_exception_refcount(void*) _NOEXCEPT; +_LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_decrement_exception_refcount(void*) _NOEXCEPT; +_LIBCPP_OVERRIDABLE_FUNC_VIS void* __cxa_current_primary_exception() _NOEXCEPT; _LIBCPP_OVERRIDABLE_FUNC_VIS void __cxa_rethrow_primary_exception(void*); } diff --git a/libcxx/include/exception b/libcxx/include/exception index c3e35583f1b3d..74229cd16c006 100644 --- a/libcxx/include/exception +++ b/libcxx/include/exception @@ -97,7 +97,6 @@ template void rethrow_if_nested(const E& e); # include # include # endif - #endif // __cplusplus < 201103L && defined(_LIBCPP_USE_FROZEN_CXX03_HEADERS) #endif // _LIBCPP_EXCEPTION From 8466f5891ed967a981360b798a8f1d91eca86dcc Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 07:48:22 +0000 Subject: [PATCH 07/11] Comment & whitespace fixes --- libcxx/docs/DesignDocs/VisibilityMacros.rst | 10 +++++----- libcxx/include/__exception/exception_ptr_glibcxx.ipp | 8 ++++---- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/libcxx/docs/DesignDocs/VisibilityMacros.rst b/libcxx/docs/DesignDocs/VisibilityMacros.rst index 79d26cd0c550d..949401012ae36 100644 --- a/libcxx/docs/DesignDocs/VisibilityMacros.rst +++ b/libcxx/docs/DesignDocs/VisibilityMacros.rst @@ -48,11 +48,11 @@ Visibility Macros This macro works by applying `[[gnu::gnu_inline]] inline` to the funciton in the header, thereby suppressing code generation while still allowing the compiler to use the function for optimization purposes. - During the build of libc++, we trigger code generation by not expanding the - macro to `_LIBCPP_EXPORTED_FROM_ABI`. Since the function is no longer marked - as `inline`, it will be emitted even if not called. (For this reason its - paramount to not define methods in the class definition, since those definitions - would be implicitly `inline`.) + During the build of libc++, we trigger code generation by expanding the + macro to `/*empty*/`. Since the function is no longer marked as `inline`, + it will be emitted even if not called. (For this reason its paramount to + not define methods in the class definition, since those definitions would + be implicitly `inline`.) **_LIBCPP_OVERRIDABLE_FUNC_VIS** Mark a symbol as being exported by the libc++ library, but allow it to be diff --git a/libcxx/include/__exception/exception_ptr_glibcxx.ipp b/libcxx/include/__exception/exception_ptr_glibcxx.ipp index 74e12f71587c3..2d4a0b8d6ba67 100644 --- a/libcxx/include/__exception/exception_ptr_glibcxx.ipp +++ b/libcxx/include/__exception/exception_ptr_glibcxx.ipp @@ -13,10 +13,10 @@ namespace std { // it uses to implement std::exception_ptr (which it declares as an alias of // std::__exception_ptr::exception_ptr) is not directly exported to clients. So // we have little choice but to hijack std::__exception_ptr::exception_ptr's -// (which fortunately has the same layout as our std::exception_ptr) copy -// constructor, assignment operator and destructor (which are part of its -// stable ABI), and its rethrow_exception(std::__exception_ptr::exception_ptr) -// function. +// _M_addref and _M_release (which are part of its ABI), and its +// rethrow_exception(std::__exception_ptr::exception_ptr) function. Fortunately, +// glibcxx's exception_ptr has the same layout as our exception_ptr and we can +// reinterpret_cast between the two. namespace __exception_ptr { struct exception_ptr { From ed1d67b81670b97ea39b716506242a4f0ee4625e Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 08:36:02 +0000 Subject: [PATCH 08/11] Remove minor unrelated changes --- libcxx/CMakeLists.txt | 1 - libcxx/include/__exception/exception_ptr.h | 1 - libcxx/include/__exception/nested_exception.h | 2 -- libcxx/src/support/runtime/exception_libcxxabi.ipp | 1 + 4 files changed, 1 insertion(+), 4 deletions(-) diff --git a/libcxx/CMakeLists.txt b/libcxx/CMakeLists.txt index 248c83dd3b3b0..4ff23f0038d0a 100644 --- a/libcxx/CMakeLists.txt +++ b/libcxx/CMakeLists.txt @@ -731,7 +731,6 @@ config_define(${LIBCXX_ABI_VERSION} _LIBCPP_ABI_VERSION) config_define(${LIBCXX_ABI_NAMESPACE} _LIBCPP_ABI_NAMESPACE) config_define(${LIBCXX_ABI_FORCE_ITANIUM} _LIBCPP_ABI_FORCE_ITANIUM) config_define(${LIBCXX_ABI_FORCE_MICROSOFT} _LIBCPP_ABI_FORCE_MICROSOFT) -config_define(${LIBCXX_CXX_ABI} _LIBCXX_CXX_ABI) config_define(${LIBCXX_ENABLE_THREADS} _LIBCPP_HAS_THREADS) config_define(${LIBCXX_ENABLE_MONOTONIC_CLOCK} _LIBCPP_HAS_MONOTONIC_CLOCK) config_define(${LIBCXX_HAS_TERMINAL_AVAILABLE} _LIBCPP_HAS_TERMINAL) diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index 003182a6f02b6..65da99618a08a 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -272,7 +272,6 @@ _LIBCPP_HIDE_FROM_ABI exception_ptr make_exception_ptr(_Ep __e) _NOEXCEPT { } #endif // _LIBCPP_ABI_MICROSOFT - _LIBCPP_END_UNVERSIONED_NAMESPACE_STD #if defined(_LIBCPP_CXX_ABI_NONE) diff --git a/libcxx/include/__exception/nested_exception.h b/libcxx/include/__exception/nested_exception.h index 38689b810bb15..90b14158d57a2 100644 --- a/libcxx/include/__exception/nested_exception.h +++ b/libcxx/include/__exception/nested_exception.h @@ -35,9 +35,7 @@ class _LIBCPP_EXPORTED_FROM_ABI nested_exception { public: nested_exception() _NOEXCEPT; _LIBCPP_HIDE_FROM_ABI nested_exception(const nested_exception&) _NOEXCEPT = default; - _LIBCPP_HIDE_FROM_ABI nested_exception(nested_exception&&) _NOEXCEPT = default; _LIBCPP_HIDE_FROM_ABI nested_exception& operator=(const nested_exception&) _NOEXCEPT = default; - _LIBCPP_HIDE_FROM_ABI nested_exception& operator=(nested_exception&&) _NOEXCEPT = default; virtual ~nested_exception() _NOEXCEPT; // access functions diff --git a/libcxx/src/support/runtime/exception_libcxxabi.ipp b/libcxx/src/support/runtime/exception_libcxxabi.ipp index 49d6057be2cc4..2bdfc9aa7f898 100644 --- a/libcxx/src/support/runtime/exception_libcxxabi.ipp +++ b/libcxx/src/support/runtime/exception_libcxxabi.ipp @@ -8,6 +8,7 @@ //===----------------------------------------------------------------------===// #include + namespace std { bool uncaught_exception() noexcept { return uncaught_exceptions() > 0; } From cdf5409532a84db0bc4b9db85bf27ad1ad984195 Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Fri, 10 Oct 2025 07:58:35 +0000 Subject: [PATCH 09/11] Fix Windows? --- libcxx/include/__exception/exception_ptr.h | 6 +++--- libcxx/include/__exception/exception_ptr_glibcxx.ipp | 4 ++-- libcxx/include/__exception/exception_ptr_msvc.ipp | 12 ++++++------ 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index 65da99618a08a..5a30fc875bce0 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -123,8 +123,8 @@ class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { __y.__ptr_ = __tmp; } - friend _LIBCPP_EXPORTED_FROM_ABI exception_ptr current_exception() _NOEXCEPT; - friend _LIBCPP_EXPORTED_FROM_ABI void rethrow_exception(exception_ptr); + friend exception_ptr current_exception() _NOEXCEPT; + friend void rethrow_exception(exception_ptr); }; // Must be defined outside the class definition due to _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE @@ -232,7 +232,7 @@ _LIBCPP_HIDE_FROM_ABI exception_ptr make_exception_ptr(_Ep) _NOEXCEPT { #else // _LIBCPP_ABI_MICROSOFT -class _LIBCPP_EXPORTED_FROM_ABI exception_ptr { +class exception_ptr { _LIBCPP_DIAGNOSTIC_PUSH _LIBCPP_CLANG_DIAGNOSTIC_IGNORED("-Wunused-private-field") void* __ptr1_; diff --git a/libcxx/include/__exception/exception_ptr_glibcxx.ipp b/libcxx/include/__exception/exception_ptr_glibcxx.ipp index 2d4a0b8d6ba67..3f054249bca83 100644 --- a/libcxx/include/__exception/exception_ptr_glibcxx.ipp +++ b/libcxx/include/__exception/exception_ptr_glibcxx.ipp @@ -38,9 +38,9 @@ _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decreme reinterpret_cast<__exception_ptr::exception_ptr*>(this)->_M_release(); } -[[noreturn]] void rethrow_exception(__exception_ptr::exception_ptr); +[[__noreturn__]] void rethrow_exception(__exception_ptr::exception_ptr); -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[noreturn]] void rethrow_exception(exception_ptr __ptr) { +[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr __ptr) { rethrow_exception(reinterpret_cast<__exception_ptr::exception_ptr&>(__ptr)); } diff --git a/libcxx/include/__exception/exception_ptr_msvc.ipp b/libcxx/include/__exception/exception_ptr_msvc.ipp index fcd426e7abd0a..d893045ba17b0 100644 --- a/libcxx/include/__exception/exception_ptr_msvc.ipp +++ b/libcxx/include/__exception/exception_ptr_msvc.ipp @@ -31,9 +31,9 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(con return *this; } -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLEexception_ptr& exception_ptr::operator=(nullptr_t) _NOEXCEPT { - exception_ptr dummy; - __ExceptionPtrAssign(this, &dummy); +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr& exception_ptr::operator=(nullptr_t) _NOEXCEPT { + exception_ptr __dummy; + __ExceptionPtrAssign(this, &__dummy); return *this; } @@ -41,7 +41,7 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::~exception_ptr() _NOEXCEPT { _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr::operator bool() const _NOEXCEPT { return __ExceptionPtrToBool(this); } -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLEbool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE bool operator==(const exception_ptr& __x, const exception_ptr& __y) _NOEXCEPT { return __ExceptionPtrCompare(&__x, &__y); } @@ -54,12 +54,12 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr __copy_exception_ptr(void* __ return __ret; } -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLEexception_ptr current_exception() _NOEXCEPT { +_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT { exception_ptr __ret; __ExceptionPtrCurrentException(&__ret); return __ret; } -_LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[noreturn]] void rethrow_exception(exception_ptr __ptr) { __ExceptionPtrRethrow(&__ptr); } +[[__noreturn__]] _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr __ptr) { __ExceptionPtrRethrow(&__ptr); } } // namespace std From 484cfaa0b5c9de0fe576d4c2e8b0cc3cc3dc219e Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Mon, 13 Oct 2025 14:23:56 +0000 Subject: [PATCH 10/11] Fix macos: Cleanup includes --- libcxx/include/__exception/exception_ptr.h | 1 + libcxx/include/__exception/exception_ptr_cxxabi.ipp | 4 +--- libcxx/include/__exception/exception_ptr_msvc.ipp | 2 -- .../include/__exception/exception_ptr_unimplemented.ipp | 8 ++++---- 4 files changed, 6 insertions(+), 9 deletions(-) diff --git a/libcxx/include/__exception/exception_ptr.h b/libcxx/include/__exception/exception_ptr.h index 5a30fc875bce0..3ba84cb50463e 100644 --- a/libcxx/include/__exception/exception_ptr.h +++ b/libcxx/include/__exception/exception_ptr.h @@ -16,6 +16,7 @@ #include <__memory/construct_at.h> #include <__type_traits/decay.h> #include <__type_traits/is_pointer.h> +#include <__verbose_abort> // used by exception_ptr_{unimplemented,cxxabi}.ipp #include #include diff --git a/libcxx/include/__exception/exception_ptr_cxxabi.ipp b/libcxx/include/__exception/exception_ptr_cxxabi.ipp index 187a6e0ad94c9..970127b94d70c 100644 --- a/libcxx/include/__exception/exception_ptr_cxxabi.ipp +++ b/libcxx/include/__exception/exception_ptr_cxxabi.ipp @@ -7,8 +7,6 @@ // //===----------------------------------------------------------------------===// -#include <__exception/terminate.h> - namespace __cxxabiv1 { extern "C" { @@ -44,7 +42,7 @@ _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE void rethrow_exception(exception_ptr __ptr) { __cxxabiv1::__cxa_rethrow_primary_exception(__ptr.__ptr_); // if __ptr.__ptr_ is NULL, above returns so we terminate. - terminate(); + _LIBCPP_VERBOSE_ABORT("tried to rethrow an empty exception_ptr\n"); } } // namespace std diff --git a/libcxx/include/__exception/exception_ptr_msvc.ipp b/libcxx/include/__exception/exception_ptr_msvc.ipp index d893045ba17b0..0cf28bee56ea1 100644 --- a/libcxx/include/__exception/exception_ptr_msvc.ipp +++ b/libcxx/include/__exception/exception_ptr_msvc.ipp @@ -7,8 +7,6 @@ // //===----------------------------------------------------------------------===// -#include - _LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrCreate(void*); _LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrDestroy(void*); _LIBCPP_CRT_FUNC void __cdecl __ExceptionPtrCopy(void*, const void*); diff --git a/libcxx/include/__exception/exception_ptr_unimplemented.ipp b/libcxx/include/__exception/exception_ptr_unimplemented.ipp index 716e5143f374a..2d6df0ea12821 100644 --- a/libcxx/include/__exception/exception_ptr_unimplemented.ipp +++ b/libcxx/include/__exception/exception_ptr_unimplemented.ipp @@ -17,20 +17,20 @@ namespace std { _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__increment_refcount(void* __ptr) _NOEXCEPT { if (__ptr) - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); + _LIBCPP_VERBOSE_ABORT("exception_ptr not yet implemented\n"); } _LIBCPP_HIDE_FROM_ABI _LIBCPP_ALWAYS_INLINE inline void exception_ptr::__decrement_refcount(void* __ptr) _NOEXCEPT { if (__ptr) - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); + _LIBCPP_VERBOSE_ABORT("exception_ptr not yet implemented\n"); } _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE exception_ptr current_exception() _NOEXCEPT { - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); + _LIBCPP_VERBOSE_ABORT("exception_ptr not yet implemented\n"); } _LIBCPP_EXPORTED_FROM_LIB_INLINEABLE [[noreturn]] void rethrow_exception(exception_ptr p) { - __libcpp_verbose_abort("exception_ptr not yet implemented\n"); + _LIBCPP_VERBOSE_ABORT("exception_ptr not yet implemented\n"); } } // namespace std From 7280bd7bbfae3ca8f823445b44eceed6e689f79c Mon Sep 17 00:00:00 2001 From: Adrian Vogelsgesang Date: Mon, 13 Oct 2025 14:26:57 +0000 Subject: [PATCH 11/11] Add release note --- libcxx/docs/ReleaseNotes/22.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libcxx/docs/ReleaseNotes/22.rst b/libcxx/docs/ReleaseNotes/22.rst index 1a450218cb988..7a8dc942a3871 100644 --- a/libcxx/docs/ReleaseNotes/22.rst +++ b/libcxx/docs/ReleaseNotes/22.rst @@ -69,6 +69,9 @@ Improvements and New Features - The performance of ``std::find`` has been improved by up to 2x for integral types +- ``std::exception_ptr`` was optimized, allowing the compiler to generate better code especially for empty + ``std::exception_ptr`` values. + Deprecations and Removals -------------------------