Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Relval wf 539.0 segfaults in el8_ppc64le_gcc12 #44059

Open
aandvalenzuela opened this issue Feb 22, 2024 · 5 comments
Open

Relval wf 539.0 segfaults in el8_ppc64le_gcc12 #44059

aandvalenzuela opened this issue Feb 22, 2024 · 5 comments
Labels
generators-pending pending-signatures

Comments

@aandvalenzuela
Copy link
Contributor

Hello,

Herwig RelVal 539.0` segfaults in el8_ppc64le_gcc12 with the following stacktrace:

Thread 5 (Thread 0x100840f491f0 (LWP 27893) "cmsRun"):
#0  0x0000100002de5b04 in nanosleep () from /lib64/libc.so.6
#1  0x0000100002de5940 in sleep () from /lib64/libc.so.6
#2  0x0000100009ed595c in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x0000100002e22ac4 in syscall () from /lib64/libc.so.6
#5  0x00001000025be308 in tbb::detail::r1::futex_wait (comparand=2, futex=0x100008c10024) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:100
#6  tbb::detail::r1::binary_semaphore::P (this=0x100008c10024) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:253
#7  tbb::detail::r1::rml::internal::thread_monitor::wait (this=0x100008c10020) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:235
#8  tbb::detail::r1::rml::private_worker::run (this=0x100008c10000) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:273
#9  tbb::detail::r1::rml::private_worker::thread_routine (arg=0x100008c10000) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:221
#10 0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#11 0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x1008405391f0 (LWP 27892) "cmsRun"):
#0  0x0000100002e19e70 in poll () from /lib64/libc.so.6
#1  0x0000100009ed63a4 in full_read.constprop () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#2  0x0000100009ed7b38 in edm::service::InitRootHandlers::stacktraceFromThread() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  0x0000100009edb63c in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x0000100848d3e6a0 in Herwig::MatchboxHybridAmplitude::canHandle(std::vector<ThePEG::Pointer::RCPtr<ThePEG::ParticleData>, std::allocator<ThePEG::Pointer::RCPtr<ThePEG::ParticleData> > > const&, ThePEG::Pointer::TransientRCPtr<Herwig::MatchboxFactory>, bool) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#6  0x0000100848cf8998 in Herwig::MatchboxFactory::makeMEs(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, unsigned int, bool) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#7  0x0000100848cfdf7c in Herwig::MatchboxFactory::setup() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#8  0x0000100848cffcd4 in Herwig::MatchboxFactory::doinit() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#9  0x0000100846e212d4 in ThePEG::InterfacedBase::init() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#10 0x0000100846e34948 in std::_Mem_fn<void (ThePEG::InterfacedBase::*)()> std::for_each<std::_Rb_tree_const_iterator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::_Mem_fn<void (ThePEG::InterfacedBase::*)()> >(std::_Rb_tree_const_iterator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::_Rb_tree_const_iterator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::_Mem_fn<void (ThePEG::InterfacedBase::*)()>) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#11 0x0000100846e156a4 in ThePEG::EventGenerator::doinit() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#12 0x0000100846e19a2c in ThePEG::EventGenerator::setup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::set<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase>, std::less<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::allocator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> > >&, std::map<long, ThePEG::Pointer::RCPtr<ThePEG::ParticleData>, std::less<long>, std::allocator<std::pair<long const, ThePEG::Pointer::RCPtr<ThePEG::ParticleData> > > >&, std::set<ThePEG::Pointer::RCPtr<ThePEG::MatcherBase>, std::less<ThePEG::Pointer::RCPtr<ThePEG::MatcherBase> >, std::allocator<ThePEG::Pointer::RCPtr<ThePEG::MatcherBase> > >&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#13 0x0000100846e77518 in ThePEG::Repository::makeRun(ThePEG::Pointer::TransientRCPtr<ThePEG::EventGenerator>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#14 0x0000100846e7a4f0 in ThePEG::Repository::exec(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::ostream&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#15 0x0000100846e7b360 in ThePEG::Repository::execAndCheckReply(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::ostream&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#16 0x0000100846e7b6ec in ThePEG::Repository::read(std::istream&, std::ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#17 0x0000100846e7bca4 in ThePEG::Repository::read(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::ostream&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#18 0x000010084578a354 in (anonymous namespace)::HerwigGenericRead(Herwig::HerwigUI const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libHerwigAPI.so.2
#19 0x0000100845013f58 in Herwig7Interface::callHerwigGenerator() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/libGeneratorInterfaceHerwig7Interface.so
#20 0x0000100845016f94 in Herwig7Interface::initRepository(edm::ParameterSet const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/libGeneratorInterfaceHerwig7Interface.so
#21 0x0000100840f6af3c in Herwig7Hadronizer::Herwig7Hadronizer(edm::ParameterSet const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginGeneratorInterfaceHerwig7HadronizerPlugins.so
#22 0x0000100840f75dd8 in edm::GeneratorFilter<Herwig7Hadronizer, gen::ExternalDecayDriver>::GeneratorFilter(edm::ParameterSet const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginGeneratorInterfaceHerwig7HadronizerPlugins.so
#23 0x0000100840f761d8 in edm::WorkerMaker<edm::GeneratorFilter<Herwig7Hadronizer, gen::ExternalDecayDriver> >::makeModule(edm::ParameterSet const&) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginGeneratorInterfaceHerwig7HadronizerPlugins.so
#24 0x00001000004015dc in edm::Maker::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#25 0x00001000002da9b0 in edm::Factory::makeModule(edm::MakeModuleParams const&, edm::ModuleTypeResolverMaker const*, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#26 0x00001000002f9048 in edm::ModuleRegistry::getModule(edm::MakeModuleParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#27 0x0000100000405a14 in edm::WorkerRegistry::getWorker(edm::WorkerParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#28 0x00001000004025c8 in edm::WorkerManager::getWorker(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#29 0x00001000003bf45c in edm::(anonymous namespace)::getWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::ParameterSet&, edm::WorkerManager&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#30 0x00001000003c4c18 in edm::StreamSchedule::fillWorkers(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<edm::WorkerInPath, std::allocator<edm::WorkerInPath> >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#31 0x00001000003c594c in edm::StreamSchedule::fillTrigPath(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<edm::HLTGlobalStatus>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#32 0x00001000003c8790 in edm::StreamSchedule::StreamSchedule(std::shared_ptr<edm::TriggerResultInserter>, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::StreamID, edm::ProcessContext const*) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#33 0x00001000003a7358 in void std::_Construct<edm::StreamSchedule, std::shared_ptr<edm::TriggerResultInserter>&, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>&, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>&, std::shared_ptr<edm::ProcessConfiguration const>&, edm::StreamID, edm::ProcessContext const*&>(edm::StreamSchedule*, std::shared_ptr<edm::TriggerResultInserter>&, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>&, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>&, std::shared_ptr<edm::ProcessConfiguration const>&, edm::StreamID&&, edm::ProcessContext const*&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#34 0x0000100000392160 in edm::Schedule::Schedule(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#35 0x00001000003a9ad4 in edm::ScheduleItems::initModules(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#36 0x00001000002337c0 in tbb::detail::d1::function_task<edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#37 0x00001000025bac44 in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter> (t=0x10000382f500, waiter=..., this=0x100003823e00) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
#38 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter> (t=0x0, waiter=..., this=0x100003823e00) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
#39 tbb::detail::r1::arena::process (tls=..., this=0x100003823780) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/arena.cpp:137
#40 tbb::detail::r1::market::process (this=0x10000384ab00, j=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/market.cpp:599
#41 0x00001000025bde30 in tbb::detail::r1::rml::private_worker::run (this=0x100008c10100) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:271
#42 tbb::detail::r1::rml::private_worker::thread_routine (arg=0x100008c10100) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:221
#43 0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#44 0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x10083fb291f0 (LWP 27891) "cmsRun"):
#0  0x0000100002de5b04 in nanosleep () from /lib64/libc.so.6
#1  0x0000100002de5940 in sleep () from /lib64/libc.so.6
#2  0x0000100009ed595c in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x0000100002e22ac4 in syscall () from /lib64/libc.so.6
#5  0x00001000025be308 in tbb::detail::r1::futex_wait (comparand=2, futex=0x100008c100a4) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:100
#6  tbb::detail::r1::binary_semaphore::P (this=0x100008c100a4) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:253
#7  tbb::detail::r1::rml::internal::thread_monitor::wait (this=0x100008c100a0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:235
#8  tbb::detail::r1::rml::private_worker::run (this=0x100008c10080) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:273
#9  tbb::detail::r1::rml::private_worker::thread_routine (arg=0x100008c10080) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:221
#10 0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#11 0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x10082c7691f0 (LWP 27811) "cmsRun"):
#0  0x0000100002cb9510 in waitpid () from /lib64/libpthread.so.0
#1  0x0000100009ed5c3c in edm::service::cmssw_stacktrace_fork() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#2  0x0000100009ed7a5c in edm::service::InitRootHandlers::stacktraceHelperThread() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  0x0000100009edc924 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#4  0x0000100002964fa0 in std::execute_native_thread_routine (__p=<optimized out>) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82
#5  0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#6  0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x10000279d530 (LWP 26578) "cmsRun"):
#0  0x0000100002de5b04 in nanosleep () from /lib64/libc.so.6
#1  0x0000100002de5940 in sleep () from /lib64/libc.so.6
#2  0x0000100009ed595c in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x0000100002e22ac4 in syscall () from /lib64/libc.so.6
#5  0x00001000025c6148 in tbb::detail::r1::futex_wait (comparand=2, futex=0x3fffeea9b2f0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:100
#6  tbb::detail::r1::binary_semaphore::P (this=0x3fffeea9b2f0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:253
#7  tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>::wait (this=0x3fffeea9b2c0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:170
#8  tbb::detail::r1::concurrent_monitor_base<tbb::detail::r1::market_context>::commit_wait (this=<optimized out>, node=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:232
#9  tbb::detail::r1::concurrent_monitor_base<tbb::detail::r1::market_context>::commit_wait (node=..., this=<optimized out>) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:228
#10 tbb::detail::r1::concurrent_monitor_base<tbb::detail::r1::market_context>::wait<tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>, tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}&>(tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}&, tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>&&) (node=..., pred=<synthetic pointer>..., this=0x10000384ab18) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:262
#11 tbb::detail::r1::sleep_waiter::sleep<tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}>(unsigned long, tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}) (this=0x3fffeea9b448, wakeup_condition=..., uniq_tag=<optimized out>) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/waiters.h:118
#12 tbb::detail::r1::external_waiter::pause (this=0x3fffeea9b448) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/waiters.h:144
#13 tbb::detail::r1::external_waiter::pause (this=0x3fffeea9b448) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/waiters.h:137
#14 tbb::detail::r1::task_dispatcher::receive_or_steal_task<true, tbb::detail::r1::external_waiter> (this=0x100003823d80, tls=..., ed=..., waiter=..., isolation=<optimized out>, fifo_allowed=<optimized out>, critical_allowed=<optimized out>) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:231
#15 0x00001000025c7d80 in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x0, this=0x100003823d80) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:350
#16 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x100003823d80) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
#17 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.cpp:168
#18 0x000010000023cb30 in edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#19 0x000010000023fe80 in edm::EventProcessor::EventProcessor(std::shared_ptr<edm::ProcessDesc>, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#20 0x0000000010007838 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#21 0x00001000025a9d60 in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/arena.cpp:688
#22 0x0000000010008dec in main::{lambda()#1}::operator()() const ()
#23 0x0000000010005b74 in main ()

Current Modules:

Module: none (crashed)

A fatal system signal has occurred: segmentation violation

The RelVal itself was introduced in #43692 and it is consistently failing for both 14_1 and 14_0 release cycles.

Thanks!
Andrea
@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 22, 2024 •
edited

cms-bot internal usage

@cmsbuild
Copy link
Contributor

A new Issue was created by @aandvalenzuela.

@Dr15Jones, @rappoccio, @sextonkennedy, @antoniovilela, @smuzaffar, @makortel can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@makortel
Copy link
Contributor

assign generators

@cmsbuild
Copy link
Contributor

New categories assigned: generators

@alberto-sanchez,@bbilin,@GurpreetSinghChahal,@mkirsano,@menglu21,@SiewYan you have been requested to review this Pull request/Issue and eventually sign? Thanks

@aandvalenzuela
Copy link
Contributor Author

It is consistently failing, e.g. also for CMSSW_14_1_X_2024-02-23-2300

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
generators-pending pending-signatures
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@makortel @cmsbuild @aandvalenzuela