[backport 106X] Adding UL16 PUID and fixing QGL values in nanoAOD #33964

alefisico · 2021-06-03T12:09:13Z

PR description:

This is the backport of Adding UL16 PUID and fixing QGL values in nanoAOD #33925 targetting nanoAODv9
This PR has to be tested with [backport 106X] PUID UL16 and UL16APV training and working points #33940

PR validation:

The PR passes the usual tests.

FYI @camclean @singh-ramanpreet

cmsbuild · 2021-06-03T12:09:39Z

A new Pull Request was created by @alefisico (Alejandro Gomez Espinosa) for CMSSW_10_6_X.

It involves the following packages:

PhysicsTools/NanoAOD

@cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please review it and eventually sign? Thanks.
@gpetruc, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2021-06-03T12:17:22Z

Pull request #33964 was updated. @cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please check and sign again.

qliphy · 2021-06-04T01:09:56Z

please test with #33940

mariadalfonso · 2021-06-05T08:00:06Z

please abort

mariadalfonso · 2021-06-05T08:01:41Z

please test

cmsbuild · 2021-06-05T08:02:48Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02f764/15635/summary.html
COMMIT: 316189e
CMSSW: CMSSW_10_6_X_2021-06-03-1100/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

1000.0

----- Begin Fatal Exception 04-Jun-2021 03:31:45 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_RAW2DIGI_L1Reco_RECO_EI_ALCAPRODUCER_DQM_ENDJOB.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/external/slc7_amd64_gcc700/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/external/slc7_amd64_gcc700/data
Current directory is: /build/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(639): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Mixins.py(356): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(713): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(1001): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Mixins.py(356): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(713): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(706): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Config.py(888): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Config.py(1089): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

1325.81

----- Begin Fatal Exception 04-Jun-2021 03:31:48 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_NANO_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/external/slc7_amd64_gcc700/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/external/slc7_amd64_gcc700/data
Current directory is: /build/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/1325.81_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(639): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Mixins.py(356): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(713): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(1001): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Mixins.py(356): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(713): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(706): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Config.py(888): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Config.py(1089): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

136.8523

----- Begin Fatal Exception 04-Jun-2021 03:31:49 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_NANO_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoJets/JetProducers/data/pileupJetId_UL16APV_Eta0p0To2p5_chs_BDT.weights.xml.gz anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15635/CMSSW_10_6_X_2021-06-03-1100/external/slc7_amd64_gcc700/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/src:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/external/slc7_amd64_gcc700/data
Current directory is: /build/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/136.8523_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2


At:
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(639): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Mixins.py(356): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(713): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(1001): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Mixins.py(356): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(713): insertContentsInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Types.py(706): insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Config.py(888): _insertInto
  /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2021-06-03-1100/python/FWCore/ParameterSet/Config.py(1089): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
136.72411136.72411_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
136.7611136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

Expand to see more relval errors ...

mariadalfonso · 2021-06-05T08:04:26Z

please abort

mariadalfonso · 2021-06-05T08:16:44Z

please test with cms-sw/cmsdist#6977

cmsbuild · 2021-06-05T12:21:09Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02f764/15669/summary.html
COMMIT: 316189e
CMSSW: CMSSW_10_6_X_2021-06-04-2300/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33964/15669/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 3215543
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3215204
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 143 log files, 29 edm output root files, 35 DQM output files
TriggerResults: found differences in 3 / 34 workflows

mariadalfonso · 2021-06-07T07:28:44Z

+xpog

puID for the Ul 2016 updated (verified that the 106Xul16 i.e. V8nano do not change)
general fix of the nan
external is merged

cmsbuild · 2021-06-07T07:29:06Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-06-07T09:42:37Z

+1

adding UL16 PUID and fix to QGL

ed9d593

cmsbuild added this to the CMSSW_10_6_X milestone Jun 3, 2021

cmsbuild added orp-pending pending-signatures tests-pending xpog-pending labels Jun 3, 2021

fix modifiers

316189e

mariadalfonso mentioned this pull request Jun 3, 2021

features for V9 nanoAOD cms-nanoAOD/cmssw#555

Closed

55 tasks

cmsbuild added requires-external tests-started and removed tests-pending labels Jun 4, 2021

cmsbuild added tests-pending and removed tests-started labels Jun 5, 2021

cmsbuild added tests-rejected tests-started and removed requires-external tests-pending tests-rejected labels Jun 5, 2021

cmsbuild added tests-pending and removed tests-started labels Jun 5, 2021

cmsbuild added requires-external and removed tests-pending labels Jun 5, 2021

cmsbuild added the tests-started label Jun 5, 2021

cmsbuild added tests-approved and removed tests-started labels Jun 5, 2021

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Jun 7, 2021

cmsbuild added orp-approved and removed orp-pending labels Jun 7, 2021

cmsbuild merged commit d6bf357 into cms-sw:CMSSW_10_6_X Jun 7, 2021

cmsbuild mentioned this pull request Jun 7, 2021

LowPtElectrons: NanoAOD integration (back port of #33817) #33992

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[backport 106X] Adding UL16 PUID and fixing QGL values in nanoAOD #33964

[backport 106X] Adding UL16 PUID and fixing QGL values in nanoAOD #33964

alefisico commented Jun 3, 2021

cmsbuild commented Jun 3, 2021

cmsbuild commented Jun 3, 2021

qliphy commented Jun 4, 2021

mariadalfonso commented Jun 5, 2021

mariadalfonso commented Jun 5, 2021

cmsbuild commented Jun 5, 2021

mariadalfonso commented Jun 5, 2021

mariadalfonso commented Jun 5, 2021

cmsbuild commented Jun 5, 2021

mariadalfonso commented Jun 7, 2021

cmsbuild commented Jun 7, 2021

qliphy commented Jun 7, 2021

[backport 106X] Adding UL16 PUID and fixing QGL values in nanoAOD #33964

[backport 106X] Adding UL16 PUID and fixing QGL values in nanoAOD #33964

Conversation

alefisico commented Jun 3, 2021

PR description:

PR validation:

cmsbuild commented Jun 3, 2021

cmsbuild commented Jun 3, 2021

qliphy commented Jun 4, 2021

mariadalfonso commented Jun 5, 2021

mariadalfonso commented Jun 5, 2021

cmsbuild commented Jun 5, 2021

Unit Tests

RelVals

RelVals-INPUT

mariadalfonso commented Jun 5, 2021

mariadalfonso commented Jun 5, 2021

cmsbuild commented Jun 5, 2021

Comparison Summary

mariadalfonso commented Jun 7, 2021

cmsbuild commented Jun 7, 2021

qliphy commented Jun 7, 2021