Add esConsumes to RecoMuon/MuonIsolation #34591

Dr15Jones · 2021-07-22T15:50:18Z

PR description:

Added esConsumes to all classes inheriting from reco::isodeposit::IsoDepositExtractor in this package

PR validation:

Code compiles.

cmsbuild · 2021-07-22T16:06:12Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34591/24147

This PR adds an extra 20KB to repository

cmsbuild · 2021-07-22T16:06:33Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

RecoMuon/MuonIsolation (reconstruction)

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@HuguesBrun, @bellan, @abbiendi, @Fedespring, @calderona, @sscruz, @jhgoh, @trocino, @cericeci, @rociovilar this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2021-07-22T16:12:24Z

please test

cmsbuild · 2021-07-22T17:54:29Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bba74f/17099/summary.html
COMMIT: bae3113
CMSSW: CMSSW_12_0_X_2021-07-22-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17099/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

11634.911

----- Begin Fatal Exception 22-Jul-2021 18:46:07 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Running EventSetup component DDDetectorESProducer/'
Exception Message:
A std::exception was thrown.
An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file Geometry/VeryForwardData/data/RP_Vertical_Device/2021/v1/RP_Vertical_Device.xml anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17099/CMSSW_12_0_X_2021-07-22-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17099/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17099/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

dd4hep: Error interpreting XML nodes of type <Include/>
dd4hep: Error interpreting XML nodes of type <IncludeSection/>
dd4hep: while parsing /cvmfs/cms-ib.cern.ch/nweek-02690/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src/Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml
dd4hep: with plugin:DD4hep_CompactLoader
----- End Fatal Exception -------------------------------------------------

slava77 · 2021-07-22T22:06:42Z

@cmsbuild please test

cmsbuild · 2021-07-23T03:55:44Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bba74f/17121/summary.html
COMMIT: bae3113
CMSSW: CMSSW_12_0_X_2021-07-22-1700/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17121/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

11601.011601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
11602.011602.0_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
11603.011603.0_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log

Expand to see more relval errors ...

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998535
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

slava77 · 2021-07-23T12:26:08Z

@smuzaffar
in the logs I see

A fatal system signal has occurred: external termination request

did somebody kill this manually?

smuzaffar · 2021-07-23T12:46:04Z

@slava77 , I do not see any thing killing these jobs, system logs on the VM ( where the job ran) also look good. I can restart the input relval test to see how it behaves

slava77 · 2021-07-23T12:58:05Z

@slava77 , I do not see any thing killing these jobs, system logs on the VM ( where the job ran) also look good. I can restart the input relval test to see how it behaves

Thanks for restarting.
That's perhaps better than rerunning everything.

Dr15Jones · 2021-07-23T12:58:55Z

@smuzaffar about a half dozen of my PR tests run last night failed for the exact same reasons.

smuzaffar · 2021-07-23T13:52:24Z

@Dr15Jones , last night many relvals test jobs took over 3 hours ( https://cmssdt.cern.ch/jenkins/job/ib-run-pr-relvals/buildTimeTrend ) which is not normal. Normally these jobs take 60-80 mins, may be there was issue with EOS or network.

smuzaffar · 2021-07-23T15:11:22Z

@Dr15Jones , relvalinput test for this PR is again hanging . The jobs were timeout after 900s. You can do the following to reproduce it

mkdir test
cd test/
cd /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17121/CMSSW_12_0_X_2021-07-22-1700
cmsenv
cd -
LOCALRT=`pwd`
runTheMatrix.py -j 16 -i all --maxSteps=2 -l 11601.0 --ibeos --command '-n 1'

Dr15Jones · 2021-07-23T17:26:26Z

@smuzaffar I've run the test as you suggested at FNAL and it passes just fine. It does take about 15 minutes to get to the first event. I attribute that to having to read the files trans-atlantically via Russia

Data is served from jinr-t1.ru instead of original site US-Regional-FNAL

Dr15Jones · 2021-07-23T17:40:42Z

@makortel
So I was wrong about where the time goes. I copied the file locally and it was still taking 20+ minutes to get to the first event. I found that it is spending all of its time looking for circular dependencies between modules! It eventually decides there aren't any.

Dr15Jones · 2021-07-23T17:51:13Z

So what I do not understand is this code change did NOT change any dependencies between modules as the check only looks at ED module dependencies, never ES modules.

Dr15Jones · 2021-07-23T18:14:33Z

@smuzaffar it looks like a problem with CMSSW_12_0_X_2021-07-22-1700. If I look at the log for wf 11601.0 for step2
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_12_0_X_2021-07-22-1700/pyRelValMatrixLogs/run/11601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log#/

I see it takes 19 minutes to get to the first event:

22-Jul-2021 20:54:39 CEST  Successfully opened file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/relval/CMSSW_11_3_0_pre4/RelValSingleElectronPt10/GEN-SIM/113X_mcRun3_2021_realistic_v7-v1/00000/68daad43-7280-46b8-a189-2d507531e87c.root
[2021-07-22 21:14:38.530999 +0200][Error  ][PostMaster        ] [eoscms.cern.ch:1094 #0] Forcing error on disconnect: [ERROR] Operation interrupted.
%MSG-e TkDetLayers:  TSGFromL2Muon:hltL3TrajSeedOIHit@streamBeginRun  22-Jul-2021 21:23:38 CEST Run: 1 Stream: 2

Dr15Jones · 2021-07-23T18:19:09Z

please test

cmsbuild · 2021-07-23T21:18:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bba74f/17155/summary.html
COMMIT: bae3113
CMSSW: CMSSW_12_0_X_2021-07-23-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34591/17155/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 5 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2998529
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 38 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

slava77 · 2021-07-24T00:47:53Z

+reconstruction

for #34591 bae3113

code changes are in line with the PR description
jenkins tests pass and comparisons with the baseline show no (relevant) differences

cmsbuild · 2021-07-24T00:48:15Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-07-24T02:10:38Z

+1

Dr15Jones added 2 commits July 22, 2021 10:25

Use esConsumes in muon IsoDepositExtractors

2f844fc

JetExtractor now uses std::unique_ptr

bae3113

cmsbuild added this to the CMSSW_12_0_X milestone Jul 22, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 22, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 22, 2021

cmsbuild added tests-started and removed tests-rejected labels Jul 22, 2021

cmsbuild added tests-rejected and removed tests-started labels Jul 23, 2021

cmsbuild added tests-started and removed tests-rejected labels Jul 23, 2021

Dr15Jones mentioned this pull request Jul 23, 2021

Migrate modules used in HLT to esConsumes cms-sw/framework-team#199

Closed

cmsbuild added tests-rejected and removed tests-started labels Jul 23, 2021

cmsbuild added tests-started and removed tests-rejected labels Jul 23, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 23, 2021

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Jul 24, 2021

cmsbuild added orp-approved and removed orp-pending labels Jul 24, 2021

cmsbuild merged commit 7dd0324 into cms-sw:master Jul 24, 2021

cmsbuild mentioned this pull request Jul 24, 2021

Updated root to tip of branch rootnext cms-sw/cmsdist#7160

Merged

Dr15Jones deleted the muonIsolation_esConsumes branch July 26, 2021 13:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add esConsumes to RecoMuon/MuonIsolation #34591

Add esConsumes to RecoMuon/MuonIsolation #34591

Dr15Jones commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Dr15Jones commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

slava77 commented Jul 22, 2021

cmsbuild commented Jul 23, 2021

slava77 commented Jul 23, 2021

smuzaffar commented Jul 23, 2021

slava77 commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

smuzaffar commented Jul 23, 2021

smuzaffar commented Jul 23, 2021 •

edited

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

cmsbuild commented Jul 23, 2021

slava77 commented Jul 24, 2021

cmsbuild commented Jul 24, 2021

qliphy commented Jul 24, 2021

Add esConsumes to RecoMuon/MuonIsolation #34591

Add esConsumes to RecoMuon/MuonIsolation #34591

Conversation

Dr15Jones commented Jul 22, 2021

PR description:

PR validation:

cmsbuild commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

Dr15Jones commented Jul 22, 2021

cmsbuild commented Jul 22, 2021

RelVals

slava77 commented Jul 22, 2021

cmsbuild commented Jul 23, 2021

RelVals-INPUT

Comparison Summary

slava77 commented Jul 23, 2021

smuzaffar commented Jul 23, 2021

slava77 commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

smuzaffar commented Jul 23, 2021

smuzaffar commented Jul 23, 2021 • edited

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

Dr15Jones commented Jul 23, 2021

cmsbuild commented Jul 23, 2021

Comparison Summary

slava77 commented Jul 24, 2021

cmsbuild commented Jul 24, 2021

qliphy commented Jul 24, 2021

smuzaffar commented Jul 23, 2021 •

edited