[122X] Update Run-3 data and MC GTs with several updates #36948

tvami · 2022-02-12T20:45:42Z

PR description:

Please see #36940

The new GTs and their diffs are

~~122X_dataRun3_HLT_v3~~
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_HLT_v1/122X_dataRun3_HLT_v3
122X_dataRun3_HLT_v4 (replaces v3 by adding back the L1TCaloGeom)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_HLT_v1/122X_dataRun3_HLT_v4
122X_dataRun3_Express_v3
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_Express_v1/122X_dataRun3_Express_v3
122X_dataRun3_Prompt_v3
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_dataRun3_Prompt_v1/122X_dataRun3_Prompt_v3
122X_mcRun3_2021_design_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2021_design_v8/122X_mcRun3_2021_design_v9
122X_mcRun3_2021_realistic_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2021_realistic_v8/122X_mcRun3_2021_realistic_v9
122X_mcRun3_2021cosmics_realistic_deco_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2021cosmics_realistic_deco_v8/122X_mcRun3_2021cosmics_realistic_deco_v9
~~122X_mcRun3_2021_realistic_HI_v9~~
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2021_realistic_HI_v8/122X_mcRun3_2021_realistic_HI_v9
122X_mcRun3_2021_realistic_HI_v10 (this was needed to be in line with 122X_mcRun3_2021_realistic_v9)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2021_realistic_HI_v8/122X_mcRun3_2021_realistic_HI_v10
122X_mcRun3_2023_realistic_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2023_realistic_v8/122X_mcRun3_2023_realistic_v9
122X_mcRun3_2024_realistic_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun3_2024_realistic_v8/122X_mcRun3_2024_realistic_v9
122X_mcRun4_realistic_v9
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/122X_mcRun4_realistic_v5/122X_mcRun4_realistic_v9

if this PR is a backport please specify the original PR and why you need to backport that PR:

Backport of #36940

tvami · 2022-02-12T20:45:50Z

Backport of #36940

cmsbuild · 2022-02-12T20:46:06Z

A new Pull Request was created by @tvami (Tamas Vami) for CMSSW_12_2_X.

It involves the following packages:

Configuration/AlCa (alca)

@cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from Update Run-3 data and MC GTs with several updates #36940

tvami · 2022-02-12T20:46:59Z

test parameters:

workflows = 12034.0,11634.0,7.23,159.0,12434.0,12834.0,138.4,138.5
pull_request = [122X] Move ecal_dqm_sourceclient and hlt_dqm_sourceclient to Run-3 era #36947

tvami · 2022-02-12T20:47:29Z

@cmsbuild , please test

Configuration/AlCa/python/autoCond.py

cmsbuild · 2022-02-12T22:14:18Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab5f37/22379/summary.html
COMMIT: 9c5e549
CMSSW: CMSSW_12_2_X_2022-02-12-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36948/22379/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test GeometryTrackerGeometryBuilderTestDriver had ERRORS

RelVals

23234.0

----- Begin Fatal Exception 12-Feb-2022 22:36:25 CET-----------------------
An exception of category 'ConditionDatabase' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'
Exception Message:
Global Tag "122X_mcRun4_realistic_v6" has not been found in the database. from GTProxy::load 
----- End Fatal Exception -------------------------------------------------

28234.0

----- Begin Fatal Exception 12-Feb-2022 22:36:25 CET-----------------------
An exception of category 'ConditionDatabase' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'
Exception Message:
Global Tag "122X_mcRun4_realistic_v6" has not been found in the database. from GTProxy::load 
----- End Fatal Exception -------------------------------------------------

35034.0

----- Begin Fatal Exception 12-Feb-2022 22:36:40 CET-----------------------
An exception of category 'ConditionDatabase' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'
Exception Message:
Global Tag "122X_mcRun4_realistic_v6" has not been found in the database. from GTProxy::load 
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

23234.023234.0_TTbar_14TeV+2026D49+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D49+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log
23234.10323234.103_TTbar_14TeV+2026D49Aging3000+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D49Aging3000+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log
23234.923234.9_TTbar_14TeV+2026D49_vectorHits+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D49_vectorHits+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log

Expand to see more relval errors ...

cmsbuild · 2022-02-12T22:23:00Z

Pull request #36948 was updated. @cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please check and sign again.

tvami · 2022-02-12T22:23:25Z

@cmsbuild , please test

cmsbuild · 2022-02-13T01:51:59Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab5f37/22382/summary.html
COMMIT: ec505bf
CMSSW: CMSSW_12_2_X_2022-02-12-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36948/22382/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/138.4_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/138.5_ExpressCollisions+RunMinimumBias2021+TIER0EXPRUN3+ALCARECOEXPR3+HARVESTDEXPR3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 13314 differences found in the comparisons
DQMHistoTests: Total files compared: 42
DQMHistoTests: Total histograms compared: 3251294
DQMHistoTests: Total failures: 11583
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3239689
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
Checked 177 log files, 37 edm output root files, 42 DQM output files
TriggerResults: no differences found

malbouis · 2022-02-13T14:45:28Z

+alca

This PR should be merged together with PR [122X] Move ecal_dqm_sourceclient and hlt_dqm_sourceclient to Run-3 era #36947
@cms-sw/egamma-pog-l2 please take a look at the results for the MC GTs and let us know if they are as expected.

francescobrivio · 2022-02-15T12:37:09Z

I added back the L1T Calo tag

@tvami could you edit the PR description to add this info and to put the correct link for the latest HLT GT (v4)?

tvami · 2022-02-15T13:00:41Z

I added back the L1T Calo tag

@tvami could you edit the PR description to add this info and to put the correct link for the latest HLT GT (v4)?

Done!

cmsbuild · 2022-02-16T17:58:38Z

Pull request #36948 was updated. @cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please check and sign again.

tvami · 2022-02-16T18:00:22Z

We received a request for a HI MC production, so I updated the Run-3 HI MC so it's now in line with the Run-3 realistic MC in commit 3caf6bd (I also updated the PR description)

tvami · 2022-02-16T18:01:42Z

@cmsbuild , please test

cmsbuild · 2022-02-16T21:45:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab5f37/22468/summary.html
COMMIT: 3caf6bd
CMSSW: CMSSW_12_2_X_2022-02-16-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36948/22468/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/138.4_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/138.5_ExpressCollisions+RunMinimumBias2021+TIER0EXPRUN3+ALCARECOEXPR3+HARVESTDEXPR3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/139.003_RunHLTPhy2021+RunHLTPhy2021+HLTDR3_2021+RECODR3_HLTPhysics_Offline+HARVESTD2021HLTPhy
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/139.004_RunNoBPTX2021+RunNoBPTX2021+HLTDR3_2021+RECODR3_AlCaTkCosmics_Offline+HARVESTDR3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-ab5f37/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

No significant changes to the logs found
Reco comparison results: 13869 differences found in the comparisons
DQMHistoTests: Total files compared: 42
DQMHistoTests: Total histograms compared: 3251294
DQMHistoTests: Total failures: 12035
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3239236
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 41 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 177 log files, 37 edm output root files, 42 DQM output files
TriggerResults: no differences found

tvami · 2022-02-16T21:47:05Z

+alca

cmsbuild · 2022-02-16T21:47:22Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-02-18T01:44:25Z

+1

cmsbuild added this to the CMSSW_12_2_X milestone Feb 12, 2022

cmsbuild added alca-pending backport orp-pending pending-signatures tests-pending labels Feb 12, 2022

cmsbuild added the requires-external label Feb 12, 2022

cmsbuild added tests-started and removed tests-pending labels Feb 12, 2022

mmusich reviewed Feb 12, 2022

View reviewed changes

Configuration/AlCa/python/autoCond.py Outdated Show resolved Hide resolved

cmsbuild added tests-rejected and removed tests-started labels Feb 12, 2022

Update Run-3 data and MC GTs with several updates

ec505bf

tvami force-pushed the NewRun3MCandDataTags_Backport branch from 9c5e549 to ec505bf Compare February 12, 2022 22:22

cmsbuild added tests-pending and removed tests-rejected labels Feb 12, 2022

cmsbuild added tests-started and removed tests-pending labels Feb 12, 2022

cmsbuild added tests-approved and removed tests-started labels Feb 13, 2022

cmsbuild removed the alca-pending label Feb 13, 2022

cmsbuild added tests-approved and removed tests-started labels Feb 15, 2022

francescobrivio mentioned this pull request Feb 15, 2022

Add back L1CaloGeometry tag to HLT GT #36971

Merged

Synchronize Run-3 HI GT with the other Run-3 GTs

3caf6bd

cmsbuild added alca-pending pending-signatures tests-pending and removed alca-approved fully-signed tests-approved labels Feb 16, 2022

cmsbuild added tests-started and removed tests-pending labels Feb 16, 2022

cmsbuild added tests-approved and removed tests-started labels Feb 16, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Feb 16, 2022

cmsbuild added orp-approved and removed orp-pending labels Feb 18, 2022

cmsbuild merged commit 0ffce6e into cms-sw:CMSSW_12_2_X Feb 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[122X] Update Run-3 data and MC GTs with several updates #36948

[122X] Update Run-3 data and MC GTs with several updates #36948

tvami commented Feb 12, 2022 •

edited

tvami commented Feb 12, 2022

cmsbuild commented Feb 12, 2022 •

edited

tvami commented Feb 12, 2022

tvami commented Feb 12, 2022

cmsbuild commented Feb 12, 2022

cmsbuild commented Feb 12, 2022

tvami commented Feb 12, 2022

cmsbuild commented Feb 13, 2022

malbouis commented Feb 13, 2022

francescobrivio commented Feb 15, 2022

tvami commented Feb 15, 2022

cmsbuild commented Feb 16, 2022

tvami commented Feb 16, 2022

tvami commented Feb 16, 2022

cmsbuild commented Feb 16, 2022

tvami commented Feb 16, 2022

cmsbuild commented Feb 16, 2022

qliphy commented Feb 18, 2022

[122X] Update Run-3 data and MC GTs with several updates #36948

[122X] Update Run-3 data and MC GTs with several updates #36948

Conversation

tvami commented Feb 12, 2022 • edited

PR description:

if this PR is a backport please specify the original PR and why you need to backport that PR:

tvami commented Feb 12, 2022

cmsbuild commented Feb 12, 2022 • edited

tvami commented Feb 12, 2022

tvami commented Feb 12, 2022

cmsbuild commented Feb 12, 2022

Unit Tests

RelVals

RelVals-INPUT

cmsbuild commented Feb 12, 2022

tvami commented Feb 12, 2022

cmsbuild commented Feb 13, 2022

Comparison Summary

malbouis commented Feb 13, 2022

francescobrivio commented Feb 15, 2022

tvami commented Feb 15, 2022

cmsbuild commented Feb 16, 2022

tvami commented Feb 16, 2022

tvami commented Feb 16, 2022

cmsbuild commented Feb 16, 2022

Comparison Summary

tvami commented Feb 16, 2022

cmsbuild commented Feb 16, 2022

qliphy commented Feb 18, 2022

tvami commented Feb 12, 2022 •

edited

cmsbuild commented Feb 12, 2022 •

edited