Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libc++] How to override a single setting in __availability? #87012

Closed
aheejin opened this issue Mar 28, 2024 · 2 comments
Closed

[libc++] How to override a single setting in __availability? #87012

aheejin opened this issue Mar 28, 2024 · 2 comments
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. question A question, not bug report. Check out https://llvm.org/docs/GettingInvolved.html instead!

Comments

@aheejin
Copy link
Member

aheejin commented Mar 28, 2024

WebAssembly's emscripten toolchain (https://github.com/emscripten-core/emscripten) has been using _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT not to pay for the increased code size of __libcpp_verbose_abort so far. But after #71002, that macro does not exist. We don't provide our own vendor annotation, and defining _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT as 0 is overridden by this line:

# define _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT 1

I read the discussions in that PR that it is recommended to create our own availability markup like Apple:

#elif defined(__APPLE__)

But to do that it looks we have to copy the whole list of macros here

// These macros control the availability of std::bad_optional_access and
// other exception types. These were put in the shared library to prevent
// code bloat from every user program defining the vtable for these exception
// types.
//
// Note that when exceptions are disabled, the methods that normally throw
// these exceptions can be used even on older deployment targets, but those
// methods will abort instead of throwing.
# define _LIBCPP_AVAILABILITY_HAS_BAD_OPTIONAL_ACCESS 1
# define _LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS
# define _LIBCPP_AVAILABILITY_HAS_BAD_VARIANT_ACCESS 1
# define _LIBCPP_AVAILABILITY_BAD_VARIANT_ACCESS
# define _LIBCPP_AVAILABILITY_HAS_BAD_ANY_CAST 1
# define _LIBCPP_AVAILABILITY_BAD_ANY_CAST
// These macros control the availability of all parts of <filesystem> that
// depend on something in the dylib.
# define _LIBCPP_AVAILABILITY_HAS_FILESYSTEM_LIBRARY 1
# define _LIBCPP_AVAILABILITY_FILESYSTEM_LIBRARY
# define _LIBCPP_AVAILABILITY_FILESYSTEM_LIBRARY_PUSH
# define _LIBCPP_AVAILABILITY_FILESYSTEM_LIBRARY_POP
// This controls the availability of the C++20 synchronization library,
// which requires shared library support for various operations
// (see libcxx/src/atomic.cpp). This includes <barier>, <latch>,
// <semaphore>, and notification functions on std::atomic.
# define _LIBCPP_AVAILABILITY_HAS_SYNC 1
# define _LIBCPP_AVAILABILITY_SYNC
// Enable additional explicit instantiations of iostreams components. This
// reduces the number of weak definitions generated in programs that use
// iostreams by providing a single strong definition in the shared library.
//
// TODO: Enable additional explicit instantiations on GCC once it supports exclude_from_explicit_instantiation,
// or once libc++ doesn't use the attribute anymore.
// TODO: Enable them on Windows once https://llvm.org/PR41018 has been fixed.
# if !defined(_LIBCPP_COMPILER_GCC) && !defined(_WIN32)
# define _LIBCPP_AVAILABILITY_HAS_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1 1
# else
# define _LIBCPP_AVAILABILITY_HAS_ADDITIONAL_IOSTREAM_EXPLICIT_INSTANTIATIONS_1 0
# endif
// This controls the availability of floating-point std::to_chars functions.
// These overloads were added later than the integer overloads.
# define _LIBCPP_AVAILABILITY_HAS_TO_CHARS_FLOATING_POINT 1
# define _LIBCPP_AVAILABILITY_TO_CHARS_FLOATING_POINT
// This controls whether the library claims to provide a default verbose
// termination function, and consequently whether the headers will try
// to use it when the mechanism isn't overriden at compile-time.
# define _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT 1
# define _LIBCPP_AVAILABILITY_VERBOSE_ABORT
// This controls the availability of the C++17 std::pmr library,
// which is implemented in large part in the built library.
# define _LIBCPP_AVAILABILITY_HAS_PMR 1
# define _LIBCPP_AVAILABILITY_PMR
// These macros controls the availability of __cxa_init_primary_exception
// in the built library, which std::make_exception_ptr might use
// (see libcxx/include/__exception/exception_ptr.h).
# define _LIBCPP_AVAILABILITY_HAS_INIT_PRIMARY_EXCEPTION 1
# define _LIBCPP_AVAILABILITY_INIT_PRIMARY_EXCEPTION
// This controls the availability of C++23 <print>, which
// has a dependency on the built library (it needs access to
// the underlying buffer types of std::cout, std::cerr, and std::clog.
# define _LIBCPP_AVAILABILITY_HAS_PRINT 1
# define _LIBCPP_AVAILABILITY_PRINT
// This controls the availability of the C++20 time zone database.
// The parser code is built in the library.
# define _LIBCPP_AVAILABILITY_HAS_TZDB 1
# define _LIBCPP_AVAILABILITY_TZDB

and copy-paste them to our section in order to just change the one line (_LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT). Is this the recommended way of changing one setting? Is there a way we can control this elsewhere?

This doesn't necessarily have to be about _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT, but it can be more general question about what is the recommended way of toggling one setting without copy-pasting dozens of settings to add another ifdef (__SOMEPLATFORM__) in __availability.

A similar concern was posted in #71002 (comment) by @wang-bin.

cc @philnik777 @ldionne

@github-actions github-actions bot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Mar 28, 2024
aheejin added a commit to aheejin/emscripten that referenced this issue Mar 29, 2024
In emscripten-core#20707 we decided to disable `__libcpp_verbose_abort` not to incur
the code size increase that brings (it brings in `vfprintf` and its
family). We disabled it in adding
`#define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT` in `__config_site`.

That `_NO_` macros are gone in LLVM 18.1.2, so I changed it to
`#define _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT 0`
emscripten-core@2bd7c67
But apparently, when including `__availability`, that setting is
overridden by this line:
https://github.com/llvm/llvm-project/blob/38f5596feda3276a8aa64fc14e074334017088ca/libcxx/include/__availability#L138
because we don't define our own vendor availability annotations
https://github.com/emscripten-core/emscripten/blob/193b2bff980529dbf5d067f01761ed8a6e41189f/system/lib/libcxx/include/__config_site#L4

I asked about this in
llvm/llvm-project#87012 and
llvm/llvm-project#71002 (comment)
(and more comments below)
but didn't get an actionable answer yet.

This disables `__libcpp_verbose_abort` in the code directly for now,
hopefully temporarily.
@ldionne
Copy link
Member

ldionne commented Apr 2, 2024

These settings are not intended to be "customizable" in the general sense. Specifically, these settings represent features that are only disabled when the deployment target is too old to support them, such as trying to use <filesystem> on an old OS where the required support had not landed in the shared library yet. In a case like that, it's not a matter of taste or preference: these settings basically prevent the ABI from being broken.

These settings are inherently temporal: whenever we don't support any platform that requires the old behavior, we remove the setting and unconditionally use the new behavior. Hence, we don't want to start allowing for individual settings to be toggled based on preference, since that's not their goal. Many other settings are toggle-able individually and we document them at https://libcxx.llvm.org/UsingLibcxx.html#libc-configuration-macros, but something like _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT doesn't fall in that category.

If you want __libcpp_verbose_abort to be toggle-able individually for code-size reason, I believe what you actually want is something slightly different that gives you a "minimal code size" version of the library. I believe this could make sense, but we'd want to include a lot more than just removing __libcpp_verbose_abort in such a mode. If you want us to investigate this idea, please detail your code size requirements and what kind of things are problematic in libc++ in an issue so we can think about it. This could make sense as part of freestanding support or on its own. For example, we'd definitely want to disable some of the recent vectorization optimizations in algorithms which increase performance but result in more code. That's just one example.

@ldionne ldionne closed this as completed Apr 2, 2024
@EugeneZelenko EugeneZelenko added the question A question, not bug report. Check out https://llvm.org/docs/GettingInvolved.html instead! label Apr 2, 2024
aheejin added a commit to emscripten-core/emscripten that referenced this issue Apr 3, 2024
This updates libcxx and libcxxabi to LLVM 18.1.2. Due to
llvm/llvm-project#65534 we need to update both
at the same time.

Things to note:

- Disable hardening mode:
  LLVM 18.1.2 adds support for "Hardening Modes" to libcxx
  (https://libcxx.llvm.org/Hardening.html). There are four modes: none,
  fast, extensive and debug. Different hardening modes make different
  trade-offs between the amount of checking and runtime performance. We
  for now disable it (i.e., set it to 'none') so that we don't have any
  effects on the performance. We can consider enabling it when we ever
  get to enable the debug version of libcxx.

- Disable C++20 time zone support:
  LLVM 18.1.2 adds C++20 time zone support
  (https://libcxx.llvm.org/DesignDocs/TimeZone.html) to libcxx, which
  requires access IANA Time Zone Database. Currently it seems it only
  supports Linux:
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/src/tz.cpp#L45-L49
  So this excludes the two source files from build (which is done via
  `CMakeLists.txt` in the upstream LLVM) and sets
  `_LIBCPP_HAS_NO_TIME_ZONE_DATABASE` macro in `__config_site`. In
  future maybe we can consider implementing this in JS.

- `__cxa_init_primary_exception` support:
  llvm/llvm-project#65534 introduces
  `__cxa_init_primary_exception` and uses this in libcxx. Like several
  other methods like the below in `cxa_exception.cpp`,
  https://github.com/emscripten-core/emscripten/blob/fbdd9249e939660cb0d20f00d6bc1897c2f3905e/system/lib/libcxxabi/src/cxa_exception.cpp#L264-L269
  this new function takes a pointer to a destructor, and in Wasm
  destructor returns a `this` pointer, which requires `ifdef
  __USING_WASM_EXCEPTION__` on its signature. And that it is also used
  in libcxx means we need to guard some code in libcxx with `ifdef
  __USING_WASM_EXCEPTION__` for the signature difference, and we need to
  build libcxx with `-D__USING_WASM_EXCEPTIONS__` when Wasm EH is used.
  Also we add Emscripte EH's counterpart in
  `cxa_exception_emscripten.cpp` too.

- This version fixes long-running misbehaviors of `operator new`
  llvm/llvm-project#69498 seems to fix some
  long-running misbhaviors of `operator new`, which in emscripten we
  tried to fix in #11079 and #20154. So while resolving the conflicts, I
  effectively reverted #11079 and #20154 in
  `libcxx/src/stdlib_new_delete.cpp` and `libcxx/src/new.cpp`.

- Add default `__assertion_handler` to `libcxx/include`:
  llvm/llvm-project#77883 adds a way for vendors
  to override how we handle assertions. If a vendor doesn't want to
  override it, CMake will copy the provided [default template assertion
  handler
  file](https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/vendor/llvm/default_assertion_handler.in)
  to libcxx's generated include dir, which defaults to
  `SYSROOT/include/c++/v1`:
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/CMakeLists.txt#L72-L73
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/CMakeLists.txt#L439
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/include/CMakeLists.txt#L1024
  We don't use CMake, so this renames the provided
  `vendor/llvm/default_assertion_handler.in` to `__assert_handler` in
  our `libcxx/include` directory, which will be copied to
  `SYSROOT/include/c++/v1`.

- Disable `__libcpp_verbose_abort directly` in code:
  In #20707 we decided to disable `__libcpp_verbose_abort` not to incur
  the code size increase that brings (it brings in `vfprintf` and its
  family). We disabled it in adding
  `#define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT` in
  `__config_site`. But that `_NO_` macros are gone in LLVM 18.1.2, and
  changing it to `#define _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT 0` does
  not work because it is overridden by this line, unless we provide our
  own vendor annotations:
  https://github.com/llvm/llvm-project/blob/38f5596feda3276a8aa64fc14e074334017088ca/libcxx/include/__availability#L138
  I asked about this in
  llvm/llvm-project#87012 and
  llvm/llvm-project#71002 (comment)
  (and more comments below)
  but didn't get an actionable answer yet. So this disables
  `__libcpp_verbose_abort` in the code directly for now, hopefully
  temporarily.

Each fix is described in its own commit, except for the fixes required
to resolve conflicts when merging our changes, which I wasn't able to
commit separately.

Fixes #21603.
@aheejin
Copy link
Member Author

aheejin commented Apr 9, 2024

Thanks for the reply. We don't have a concrete requirement for now, but will post an issue later if we get to work on it.

impact-maker pushed a commit to impact-maker/emscripten that referenced this issue Apr 11, 2024
This updates libcxx and libcxxabi to LLVM 18.1.2. Due to
llvm/llvm-project#65534 we need to update both
at the same time.

Things to note:

- Disable hardening mode:
  LLVM 18.1.2 adds support for "Hardening Modes" to libcxx
  (https://libcxx.llvm.org/Hardening.html). There are four modes: none,
  fast, extensive and debug. Different hardening modes make different
  trade-offs between the amount of checking and runtime performance. We
  for now disable it (i.e., set it to 'none') so that we don't have any
  effects on the performance. We can consider enabling it when we ever
  get to enable the debug version of libcxx.

- Disable C++20 time zone support:
  LLVM 18.1.2 adds C++20 time zone support
  (https://libcxx.llvm.org/DesignDocs/TimeZone.html) to libcxx, which
  requires access IANA Time Zone Database. Currently it seems it only
  supports Linux:
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/src/tz.cpp#L45-L49
  So this excludes the two source files from build (which is done via
  `CMakeLists.txt` in the upstream LLVM) and sets
  `_LIBCPP_HAS_NO_TIME_ZONE_DATABASE` macro in `__config_site`. In
  future maybe we can consider implementing this in JS.

- `__cxa_init_primary_exception` support:
  llvm/llvm-project#65534 introduces
  `__cxa_init_primary_exception` and uses this in libcxx. Like several
  other methods like the below in `cxa_exception.cpp`,
  https://github.com/emscripten-core/emscripten/blob/fbdd9249e939660cb0d20f00d6bc1897c2f3905e/system/lib/libcxxabi/src/cxa_exception.cpp#L264-L269
  this new function takes a pointer to a destructor, and in Wasm
  destructor returns a `this` pointer, which requires `ifdef
  __USING_WASM_EXCEPTION__` on its signature. And that it is also used
  in libcxx means we need to guard some code in libcxx with `ifdef
  __USING_WASM_EXCEPTION__` for the signature difference, and we need to
  build libcxx with `-D__USING_WASM_EXCEPTIONS__` when Wasm EH is used.
  Also we add Emscripte EH's counterpart in
  `cxa_exception_emscripten.cpp` too.

- This version fixes long-running misbehaviors of `operator new`
  llvm/llvm-project#69498 seems to fix some
  long-running misbhaviors of `operator new`, which in emscripten we
  tried to fix in emscripten-core#11079 and emscripten-core#20154. So while resolving the conflicts, I
  effectively reverted emscripten-core#11079 and emscripten-core#20154 in
  `libcxx/src/stdlib_new_delete.cpp` and `libcxx/src/new.cpp`.

- Add default `__assertion_handler` to `libcxx/include`:
  llvm/llvm-project#77883 adds a way for vendors
  to override how we handle assertions. If a vendor doesn't want to
  override it, CMake will copy the provided [default template assertion
  handler
  file](https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/vendor/llvm/default_assertion_handler.in)
  to libcxx's generated include dir, which defaults to
  `SYSROOT/include/c++/v1`:
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/CMakeLists.txt#L72-L73
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/CMakeLists.txt#L439
  https://github.com/llvm/llvm-project/blob/26a1d6601d727a96f4301d0d8647b5a42760ae0c/libcxx/include/CMakeLists.txt#L1024
  We don't use CMake, so this renames the provided
  `vendor/llvm/default_assertion_handler.in` to `__assert_handler` in
  our `libcxx/include` directory, which will be copied to
  `SYSROOT/include/c++/v1`.

- Disable `__libcpp_verbose_abort directly` in code:
  In emscripten-core#20707 we decided to disable `__libcpp_verbose_abort` not to incur
  the code size increase that brings (it brings in `vfprintf` and its
  family). We disabled it in adding
  `#define _LIBCPP_AVAILABILITY_HAS_NO_VERBOSE_ABORT` in
  `__config_site`. But that `_NO_` macros are gone in LLVM 18.1.2, and
  changing it to `#define _LIBCPP_AVAILABILITY_HAS_VERBOSE_ABORT 0` does
  not work because it is overridden by this line, unless we provide our
  own vendor annotations:
  https://github.com/llvm/llvm-project/blob/38f5596feda3276a8aa64fc14e074334017088ca/libcxx/include/__availability#L138
  I asked about this in
  llvm/llvm-project#87012 and
  llvm/llvm-project#71002 (comment)
  (and more comments below)
  but didn't get an actionable answer yet. So this disables
  `__libcpp_verbose_abort` in the code directly for now, hopefully
  temporarily.

Each fix is described in its own commit, except for the fixes required
to resolve conflicts when merging our changes, which I wasn't able to
commit separately.

Fixes emscripten-core#21603.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. question A question, not bug report. Check out https://llvm.org/docs/GettingInvolved.html instead!
Projects
None yet
Development

No branches or pull requests

3 participants