Avoid rerunning particlelevel producer in NanoAOD

[kdlong]

Previously Rivet particle level producer was run separately to build collections for different kinematic criteria needed in matching. Now just run once with loose cuts, then apply selections before matching and in the RivetTable. Also exploit the changes in #34575 to speed up by not running jet clustering. @mseidel42

[mariadalfonso]

enable profiling

[mariadalfonso]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34590/24146

• This PR adds an extra 36KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File GeneratorInterface/RivetInterface/interface/RivetAnalysis.h modified in PR(s): ParticleLevelProducer switch for jet clustering #34575
  • File GeneratorInterface/RivetInterface/python/particleLevel_cfi.py modified in PR(s): ParticleLevelProducer switch for jet clustering #34575
  • File PhysicsTools/NanoAOD/python/electrons_cff.py modified in PR(s): DRAFT: nano: adaptation to run in Prompt #34329
  • File PhysicsTools/NanoAOD/python/lowPtElectrons_cff.py modified in PR(s): DRAFT: nano: adaptation to run in Prompt #34329
  • File PhysicsTools/NanoAOD/python/particlelevel_cff.py modified in PR(s): DRAFT: nano: adaptation to run in Prompt #34329

[cmsbuild]

A new Pull Request was created by @kdlong (Kenneth Long) for master.

It involves the following packages:

• GeneratorInterface/RivetInterface (generators)
• PhysicsTools/NanoAOD (xpog)

@SiewYan, @gouskos, @mkirsano, @Saptaparna, @GurpreetSinghChahal, @fgolf, @alberto-sanchez, @agrohsje, @mariadalfonso can you please review it and eventually sign? Thanks.
@gpetruc, @alberto-sanchez, @mkirsano, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c937f8/17097/summary.html
COMMIT: 0b5e570
CMSSW: CMSSW_12_0_X_2021-07-22-1100/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34590/17097/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

• 11634.911

----- Begin Fatal Exception 22-Jul-2021 17:44:17 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Running EventSetup component DDDetectorESProducer/'
Exception Message:
A std::exception was thrown.
An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file Geometry/VeryForwardData/data/RP_Vertical_Device/2021/v1/RP_Vertical_Device.xml anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34590/17097/CMSSW_12_0_X_2021-07-22-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34590/17097/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34590/17097/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src:/cvmfs/cms-ib.cern.ch/week0/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/external/slc7_amd64_gcc900/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA

dd4hep: Error interpreting XML nodes of type <Include/>
dd4hep: Error interpreting XML nodes of type <IncludeSection/>
dd4hep: while parsing /cvmfs/cms-ib.cern.ch/nweek-02690/slc7_amd64_gcc900/cms/cmssw/CMSSW_12_0_X_2021-07-22-1100/src/Geometry/CMSCommonData/data/dd4hep/cmsExtendedGeometry2021.xml
dd4hep: with plugin:DD4hep_CompactLoader
----- End Fatal Exception -------------------------------------------------

[mariadalfonso]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c937f8/17150/summary.html
COMMIT: 0b5e570
CMSSW: CMSSW_12_0_X_2021-07-22-2300/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34590/17150/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2998564
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2998541
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: found differences in 7 / 38 workflows

[mariadalfonso]

please test workflow 546,547,548

(test nanoGEN)

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c937f8/17159/summary.html
COMMIT: 0b5e570
CMSSW: CMSSW_12_0_X_2021-07-23-1100/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34590/17159/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-c937f8/546.0_DYToLL_M-50_13TeV_pythia8+DYToLL_M-50_13TeV_pythia8+NANOGENFromGen
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-c937f8/547.0_DYToll01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV+DYToll01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV+Hadronizer_TuneCP5_13TeV_MLM_5f_max4j_LHE_pythia8+NANOGENFromGen
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-c937f8/548.0_TTbar_Pow_LHE_13TeV+TTbar_Pow_LHE_13TeV+Hadronizer_TuneCP5_13TeV_powhegEmissionVeto2p_pythia8+NANOGENFromGen

Summary:

• No significant changes to the logs found
• Reco comparison results: 5 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2998564
• DQMHistoTests: Total failures: 12
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 2998529
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 38 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: found differences in 7 / 38 workflows

[mariadalfonso]

• release --> GeneratorInterface | 145.4 ms | 17.3 %
• with PR --> GeneratorInterface | 33.9 ms | 4.7 %

https://cmssdt.cern.ch/circles/web/piechart.php?local=false&dataset=CMSSW_12_0_X_2021-07-22-2300%2Fslc7_amd64_gcc900%2F11834.21%2Fstep5_NANO.resources&resource=time_thread&colours=default&groups=packages&threshold=0

https://cmssdt.cern.ch/circles/web/piechart.php?local=false&dataset=CMSSW_12_0_X_2021-07-22-2300%2Fslc7_amd64_gcc900%2F11834.21%2FPR-c937f8%2F17150%2Fstep5_NANO.resources&resource=time_thread&colours=default&groups=packages&threshold=0

[mariadalfonso]

+xpog

significant change in speed #34590 (comment) w/o change in nano content

[agrohsje]

+generators

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy, @perrotta (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1