rootcling segmentation fault on macos26

[andresailer]

Check duplicate issues.

• Checked for duplicates

Description

The invocations of rootcling during the compilation of ROOT on macos 26 fails with a segmentation fault. We first observed this on December 1, the build on November 29 was still OK

See https://lcgapp-services.cern.ch/cdash/build/130903/file

e.g.:

[ 54%] Generating G__RooFitJSONInterface.cxx, ../../lib/RooFitJSONInterface.pcm
Segmentation fault

Reproducer

Building ROOT with lcgcmake on macos26. I will come up with a step by step reproducer if warranted

ROOT version

master branch

Installation method

from source

Operating system

macOS 26

Additional context

running rootcling manually through lldb points towards tbb, for which we use an externally built one.

(lldb) run  -rootbuild -v2 -f G__RooFitJSONInterface.cxx -cxxmodule -s /Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/lib/libRooFitJSONInterface.so -m Core.pcm -excludePath /Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD -excludePath /Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/ginclude -excludePath /Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/externals -excludePath /Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/builtins -writeEmptyRootPCM -DNLOHMANN_JSON_PROVIDES_FWD_HPP -DNLOHMANN_JSON_PROVIDES_FWD_HPP -compilerI/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -compilerI/Library/Developer/CommandLineTools/usr/lib/clang/17/include -compilerI/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -compilerI/Library/Developer/CommandLineTools/usr/include -compilerI/Library/Developer/CommandLineTools/usr/lib/clang/17/include -compilerI/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -compilerI/Library/Developer/CommandLineTools/usr/include -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/include -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/roofit/jsoninterface/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/unix/../clib/res -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/unix/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/macosx/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/builtins/zstd -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/zstd/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/lz4/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/lzma/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/zip/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/thread/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/textinput/src -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/textinput/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/rint/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/meta/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/gui/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/foundation/res -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/foundation/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/cont/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/clingutils/res -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/clingutils/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/clib/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/base/inc -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/base/src/ -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/core/base/inc/ -I/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/ginclude -I/Users/sailer/SPI/install/dev3/jsonmcpp/3.12.0/arm64-mac26-clang170-opt/include RooFit/Detail/JSONInterface.h /Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT/HEAD/roofit/jsoninterface/LinkDef.h -DR__DUMMY_CXX_STANDARD_20
Process 70303 launched: '/Users/sailer/SPI/build/projects/ROOT-HEAD/src/ROOT-HEAD-build/bin/rootcling' (arm64)
Process 70303 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x100265860)
    frame #0: 0x0000000100265860
error: memory read failed for 0x100265800
Target 0: (rootcling) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x100265860)
  * frame #0: 0x0000000100265860
    frame #1: 0x000000010020a2d4 libtbb.12.16.dylib`tbb::detail::r1::governor::release_resources() + 344
    frame #2: 0x000000010020ce04 libtbb.12.16.dylib`tbb::detail::r1::__TBB_InitOnce::~__TBB_InitOnce() + 60
    frame #3: 0x000000019bd2942c libsystem_c.dylib`__cxa_finalize_ranges + 480
    frame #4: 0x000000019bd291ec libsystem_c.dylib`exit + 44
    frame #5: 0x000000019ba77fc8 libdyld.dylib`dyld4::LibSystemHelpers::exit(int) const + 20
    frame #6: 0x000000019baadf04 dyld`dyld4::LibSystemHelpersWrapper::exit(int) const + 172
    frame #7: 0x000000019baadd84 dyld`start + 7232

[andresailer]

PS: since the backtrace mentions tbb, and #20564 was merged in the timeframe in question, pinging @guitargeek

[guitargeek]

Hi! Yes I think we need a reproducer, so this can be investigated by our macOS experts.

[andresailer]

Hi! Yes I think we need a reproducer, so this can be investigated by our macOS experts.

Jonas and I are discussing this via DM

[andresailer]

Actually here is a reproducer that should work to fail on macos26 if cmake and compilers are available.

mkdir reproCrash
cd reproCrash
git clone https://gitlab.cern.ch/sft/lcgcmake.git
mkdir preq build
cd build
cmake -DCMAKE_INSTALL_PREFIX=../preq -DLCG_VERSION=dev3 -DLCG_INSTALL_PREFIX=/cvmfs/sft.cern.ch/lcg/latest:/cvmfs/sft.cern.ch/lcg/releases -DUSE_BINARIES=ON -D LCG_ADDITIONAL_REPOS=https://lcgpackages.web.cern.ch/tarFiles/latest ../lcgcmake
make -j 10 ROOT-dependencies lcgenv
cd ..
echo "export PATH=${PWD}/preq/lcgenv/1.3.22-6078d/arm64-mac26-clang170-opt/:\$PATH\n" > setup.sh
ls -1rt ${PWD}/preq/*/*/arm64-mac26-clang170-opt/*-env.sh | sed "s@^@source @" >> setup.sh
source setup.sh
git clone https://github.com/root-project/root.git
mkdir build-ROOT
cd build-ROOT

cmake \
   -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_INSTALL_PREFIX=${PWD}/../install \
   -DCMAKE_CXX_STANDARD=20 \
   -Dcheck_connection=OFF \
   -DCINTLONGLINE=4096 \
   -DCINTMAXSTRUCT=36000 \
   -DCINTMAXTYPEDEF=36000 \
   -DPYTHON_EXECUTABLE=`which python` \
   -Dbuiltin_ftgl=ON \
   -Dbuiltin_glew=ON \
   -Dbuiltin_lz4=ON \
   -Dbuiltin_pcre=ON \
   -Dbuiltin_unuran=ON \
   -Dbuiltin_xxhash=ON \
   -Dbuiltin_zstd=ON \
   -Dcintex=ON \
   -Dexplicitlink=ON \
   -Dfail-on-missing=ON \
   -Dfftw3=ON \
   -Dfitsio=ON \
   -Dfortran=ON \
   -Dgdml=ON \
   -Dgenvector=ON \
   -Dhttp=ON \
   -Dmathmore=ON \
   -Dopengl=ON \
   -Dpyroot=ON \
   -Dpythia8=OFF \
   -Dr=OFF \
   -Dreflex=ON \
   -Droofit=ON \
   -Droofit_multiprocess=OFF \
   -Droot7=ON \
   -Dssl=ON \
   -Dtesting=OFF \
   -Dunfold=ON \
   -Dunuran=ON \
   -Dxft=ON \
   -Dxml=ON \
   -Dxrootd=ON \
   -Dzlib=ON \
   -Dvc=OFF \
   -Dtmva-sofie=ON \
   -Dmysql=OFF \
   -Ddavix=ON \
   -Dbuiltin_gif=ON \
   -Dbuiltin_jpeg=OFF \
   -Dbuiltin_png=OFF \
   -Droottest=OFF \
   -Dbuiltin_nlohmannjson=OFF \
   -G Ninja ../root
ninja

[andresailer]

Seems to be a problem in tbb (uxlfoundation/oneTBB#1849)
Not sure what caused this to appear after the PR mentioned above though.

[andresailer]

Not sure what caused this to appear after the PR mentioned above though.

The use of TBB_FOUND instead of tbb causes the linking against tbb, and the fact that there is a bug in the latest versions of tbb.

I tested also tbb 2022.3.0, which fails as well.
I applied the patch from the linked PR and that fixes the issue for us.

[guitargeek]

Thank you very much for addressing this issue yourself! Very much appreciated 👍