Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unit test failure in PhysicsTools/UtilAlgos #33486

Closed
silviodonato opened this issue Apr 21, 2021 · 2 comments
Closed

Unit test failure in PhysicsTools/UtilAlgos #33486

silviodonato opened this issue Apr 21, 2021 · 2 comments
Labels
pending-assignment

Comments

@silviodonato
Copy link
Contributor

Since CMSSW_11_3_X_2021-04-15-2300 a unit test is failing in PhysicsTools/UtilAlgos.
This was trivially fixed in 12_0_X with #33464

https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/raw/slc7_amd64_gcc900/CMSSW_11_3_X_2021-04-20-2300/unitTestLogs/PhysicsTools/UtilAlgos

 ===== Test "runtestUtilAlgos" ====
argument 0: runtestUtilAlgos
argument 1: /bin/bash
argument 2: PhysicsTools/UtilAlgos/test
argument 3: runtests.sh
shell is: /bin/bash
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_11_3_X_2021-04-20-2300
topdir is: /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_11_3_X_2021-04-20-2300
testdir is: /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_11_3_X_2021-04-20-2300/src/PhysicsTools/UtilAlgos/test
tmpdir is: /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_11_3_X_2021-04-20-2300/tmp/slc7_amd64_gcc900
testbin is: /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_11_3_X_2021-04-20-2300/test/slc7_amd64_gcc900
Running script: /data/cmsbld/jenkins/workspace/ib-run-qa/CMSSW_11_3_X_2021-04-20-2300/src/PhysicsTools/UtilAlgos/test/runtests.sh
21-Apr-2021 00:21:36 CEST  Initiating request to open file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root
%MSG-w XrdAdaptor:  file_open 21-Apr-2021 00:21:39 CEST pre-events
Data is served from cern.ch instead of original site eoscms
%MSG
21-Apr-2021 00:21:49 CEST  Successfully opened file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root
Begin processing the 1st record. Run 1, Event 8251, LumiSection 166 on stream 0 at 21-Apr-2021 00:21:58.706 CEST

[...]

Begin processing the 100th record. Run 1, Event 350, LumiSection 7 on stream 0 at 21-Apr-2021 00:22:03.081 CEST
21-Apr-2021 00:22:03 CEST  Closed file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root

[...]

dropped waiting message count 0
21-Apr-2021 00:22:10 CEST  Initiating request to open file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root
%MSG-w XrdAdaptor:  file_open 21-Apr-2021 00:22:13 CEST pre-events
Data is served from cern.ch instead of original site eoscms
%MSG
21-Apr-2021 00:22:22 CEST  Successfully opened file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root
Begin processing the 1st record. Run 1, Event 8251, LumiSection 166 on stream 0 at 21-Apr-2021 00:22:30.663 CEST

[...]

Begin processing the 100th record. Run 1, Event 350, LumiSection 7 on stream 0 at 21-Apr-2021 00:22:33.065 CEST
21-Apr-2021 00:22:33 CEST  Closed file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root

[...]

dropped waiting message count 0
21-Apr-2021 00:22:38 CEST  Initiating request to open file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root
%MSG-w XrdAdaptor:  file_open 21-Apr-2021 00:22:40 CEST pre-events
Data is served from cern.ch instead of original site eoscms
%MSG
21-Apr-2021 00:22:48 CEST  Successfully opened file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root
Begin processing the 1st record. Run 1, Event 8251, LumiSection 166 on stream 0 at 21-Apr-2021 00:22:56.729 CEST

[...]

Begin processing the 100th record. Run 1, Event 350, LumiSection 7 on stream 0 at 21-Apr-2021 00:22:59.223 CEST
21-Apr-2021 00:22:59 CEST  Closed file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_9_3_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_92X_upgrade2017_realistic_v7-v1/00000/32EA1438-3D61-E711-8FE7-0025905B85B2.root

=============================================

MessageLogger Summary

 type     category        sev    module        subroutine        count    total
 ---- -------------------- -- ---------------- ----------------  -----    -----
    1 Root_Information     -i AfterModConstruc TClass::Init          2*       2
    2 Root_Information     -i PreProcPath p    TClass::Init          1*       1
    3 XrdAdaptorInternal   -i (NoModuleName)                       178*     178
    4 XrdAdaptorInternal   -i After BeginProce                       4*       4
    5 XrdAdaptorInternal   -i AfterFile                            328*     328
    6 XrdAdaptorInternal   -i PostStreamBeginR                       4*       4
    7 XrdAdaptorInternal   -i PreProcPath p                         17*      17
    8 XrdAdaptorInternal   -i file_open                              8*       8
    9 XrdAdaptorInternal   -i source                                 5*       5
   10 XrdFileInfo          -i file_close                             1*       1
   11 XrdFileInfo          -i file_open                              2*       2
   12 FwkReport            -f AfterSource                          100      100
   13 XrdAdaptor           -w file_open                              1        1
   14 fileAction           -s file_close                             1        1
   15 fileAction           -s file_open                              2        2

* Some occurrences of this message were suppressed in all logs, due to limits.

 type    category    Examples: run/evt        run/evt          run/evt
 ---- -------------------- ---------------- ---------------- ----------------
    1 Root_Information     pre-events       pre-events       
    2 Root_Information     1/8251                            
    3 XrdAdaptorInternal   pre-events       pre-events       pre-events
    4 XrdAdaptorInternal   pre-events       pre-events       pre-events
    5 XrdAdaptorInternal   pre-events       pre-events       pre-events
    6 XrdAdaptorInternal   PostStreamBeginR PostStreamBeginR PostStreamBeginR
    7 XrdAdaptorInternal   1/8256           1/8256           1/8256
    8 XrdAdaptorInternal   pre-events       pre-events       pre-events
    9 XrdAdaptorInternal   PreSource        PreSource        PreSource
   10 XrdFileInfo          PostGlobalEndRun                  
   11 XrdFileInfo          pre-events       pre-events       
   12 FwkReport            AfterSource      AfterSource      AfterSource
   13 XrdAdaptor           pre-events                        
   14 fileAction           PostGlobalEndRun                  
   15 fileAction           pre-events       pre-events       

Severity    # Occurrences   Total Occurrences
--------    -------------   -----------------
Info                  550                 550
FwkInfo               100                 100
Warning                 1                   1
System                  3                   3

dropped waiting message count 0
Error in <TNetXNGFile::Open>: [ERROR] Server responded with an error: [3011] Unable to open file /eos/cms/store/relval/CMSSW_7_2_0_pre5/RelValProdTTbar/GEN-SIM-RECO/START72_V1-v1/00000/022350A9-AC30-E411-B225-0025905A6076.root; No such file or directory

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 6 (Thread 0x2b33d0cad700 (LWP 2397)):
#0  0x00002b33bb045b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
#1  0x00002b33bb045bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2  0x00002b33bb045c6b in sem_wait

GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3  0x00002b33cfeb2816 in XrdCl::JobManager::RunJobs() () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#4  0x00002b33cfeb28c9 in RunRunnerThread () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#5  0x00002b33bb03fea5 in start_thread () from /lib64/libpthread.so.0
#6  0x00002b33bb3529fd in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x2b33d0aac700 (LWP 2396)):
#0  0x00002b33bb045b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
#1  0x00002b33bb045bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2  0x00002b33bb045c6b in sem_wait

GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3  0x00002b33cfeb2816 in XrdCl::JobManager::RunJobs() () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#4  0x00002b33cfeb28c9 in RunRunnerThread () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#5  0x00002b33bb03fea5 in start_thread () from /lib64/libpthread.so.0
#6  0x00002b33bb3529fd in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x2b33d08ab700 (LWP 2395)):
#0  0x00002b33bb045b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
#1  0x00002b33bb045bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2  0x00002b33bb045c6b in sem_wait

GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3  0x00002b33cfeb2816 in XrdCl::JobManager::RunJobs() () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#4  0x00002b33cfeb28c9 in RunRunnerThread () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#5  0x00002b33bb03fea5 in start_thread () from /lib64/libpthread.so.0
#6  0x00002b33bb3529fd in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x2b33d06aa700 (LWP 2394)):
#0  0x00002b33bb046e9d in nanosleep () from /lib64/libpthread.so.0
#1  0x00002b33cfcda6ff in XrdSysTimer::Wait(int) () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdUtils.so.2
#2  0x00002b33cfe54b0a in XrdCl::TaskManager::RunTasks() () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#3  0x00002b33cfe54c69 in RunRunnerThread () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdCl.so.2
#4  0x00002b33bb03fea5 in start_thread () from /lib64/libpthread.so.0
#5  0x00002b33bb3529fd in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x2b33d04a9700 (LWP 2393)):
#0  0x00002b33bb352fd3 in epoll_wait () from /lib64/libc.so.6
#1  0x00002b33cfcd4ed2 in XrdSys::IOEvents::PollE::Begin(XrdSysSemaphore*, int&, char const**) () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdUtils.so.2
#2  0x00002b33cfcd178d in XrdSys::IOEvents::BootStrap::Start(void*) () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdUtils.so.2
#3  0x00002b33cfcd9f68 in XrdSysThread_Xeq () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libXrdUtils.so.2
#4  0x00002b33bb03fea5 in start_thread () from /lib64/libpthread.so.0
#5  0x00002b33bb3529fd in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x2b33bcffc100 (LWP 2023)):
#0  0x00002b33bb319549 in waitpid () from /lib64/libc.so.6
#1  0x00002b33bb296fb2 in do_system () from /lib64/libc.so.6
#2  0x00002b33bb297361 in system () from /lib64/libc.so.6
#3  0x00002b33b8a74535 in TUnixSystem::StackTrace() () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libCore.so
#4  0x00002b33b8a71705 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_X_2021-04-20-2300/external/slc7_amd64_gcc900/lib/libCore.so
#5  <signal handler called>
#6  0x00002b33b6aad32c in fwlite::ChainEvent::ChainEvent(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_X_2021-04-18-0000/lib/slc7_amd64_gcc900/libDataFormatsFWLite.so
#7  0x0000000000405d00 in fwlite::AnalyzerWrapper<BasicMuonAnalyzer>::analyze() ()
#8  0x0000000000404695 in main ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6  0x00002b33b6aad32c in fwlite::ChainEvent::ChainEvent(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) () from /cvmfs/cms-ib.cern.ch/nweek-02677/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_X_2021-04-18-0000/lib/slc7_amd64_gcc900/libDataFormatsFWLite.so
#7  0x0000000000405d00 in fwlite::AnalyzerWrapper<BasicMuonAnalyzer>::analyze() ()
#8  0x0000000000404695 in main ()
===========================================================


Failure using fwliteWithPythonConfig: status 139
status = 35584

---> test runtestUtilAlgos had ERRORS
 
^^^^ End Test runtestUtilAlgos ^^^^
>> Tests for package PhysicsTools/UtilAlgos ran.
@cmsbuild
Copy link
Contributor

A new Issue was created by @silviodonato Silvio Donato.

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@silviodonato
Copy link
Contributor Author

This failure will be fixed soon with #33487.
Since this failure will appear also in all other branches, I've opened this issue to keep track of it

@qliphy qliphy mentioned this issue Apr 21, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pending-assignment
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cmsbuild @silviodonato