Adding JECs, PF and L1T menu to Run-3 MC GTs and data relval GTs

[tvami]

PR description:

Include JEC and PF calibrations as request in https://cms-talk.web.cern.ch/t/gt-new-run3-hlt-jec-tags-for-123x-125x-for-mc/13290 and https://cms-talk.web.cern.ch/t/gt-updated-pf-calibration-hlt-tag-for-run-3/13293 then add new L1T menu as requested in https://cms-talk.web.cern.ch/t/run-3-gt-update-of-the-l1-menu-tag-v1-3-0-in-run-3-mc-gts-and-run-3-data-relvals-gts/13513

The new GTs and their diff wrt to the last GTs in autoCond are here

phase1_2022_design

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2022_design_v7/125X_mcRun3_2022_design_v2

phase1_2022_realistic

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2022_realistic_v10/125X_mcRun3_2022_realistic_v2

phase1_2022_cosmics

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2022cosmics_realistic_deco_v11/125X_mcRun3_2022cosmics_realistic_deco_v2

phase1_2022_cosmics_design

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2022cosmics_design_deco_v7/125X_mcRun3_2022cosmics_design_deco_v2

phase1_2022_realistic_hi

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2022_realistic_HI_v10/125X_mcRun3_2022_realistic_HI_v2

phase1_2023_realistic

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2023_realistic_v11/125X_mcRun3_2023_realistic_v2

phase1_2024_realistic

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun3_2024_realistic_v11/125X_mcRun3_2024_realistic_v2

phase2_realistic

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_mcRun4_realistic_v8/125X_mcRun4_realistic_v2

run3_hlt_relval

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_dataRun3_HLT_relval_v8/125X_dataRun3_HLT_relval_v1

run3_data_relval

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts//124X_dataRun3_relval_v8/125X_dataRun3_relval_v1

PR validation:

test parameters:
  - workflows = 12034.0,11634.0,7.23,159.0,12434.0,12834.0

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, backport might be needed, under discussion

[tvami]

test parameters:

• workflows = 12034.0,11634.0,7.23,159.0,12434.0,12834.0

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38916/31362

• This PR adds an extra 12KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/AlCa/python/autoCond.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760

[cmsbuild]

A new Pull Request was created by @tvami (Tamas Vami) for master.

It involves the following packages:

• Configuration/AlCa (alca)

@malbouis, @yuanchao, @cmsbuild, @saumyaphor4252, @francescobrivio, @ChrisMisan, @tvami can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38916/31365

• This PR adds an extra 12KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/AlCa/python/autoCond.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760

[cmsbuild]

Pull request #38916 was updated. @malbouis, @yuanchao, @cmsbuild, @saumyaphor4252, @francescobrivio, @ChrisMisan, @tvami can you please check and sign again.

[tvami]

@cmsbuild , please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a50019/26565/summary.html
COMMIT: 5d47cce
CMSSW: CMSSW_12_5_X_2022-08-01-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38916/26565/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-a50019/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-a50019/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 55
• DQMHistoTests: Total histograms compared: 4121886
• DQMHistoTests: Total failures: 8732
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4113132
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 54 files compared)
• Checked 229 log files, 47 edm output root files, 55 DQM output files
• TriggerResults: found differences in 8 / 54 workflows

[tvami]

assign hlt

• all these changes are affecting HLT step

[cmsbuild]

New categories assigned: hlt

@missirol,@Martin-Grunewald you have been requested to review this Pull request/Issue and eventually sign? Thanks

[missirol]

@tvami

I can check the comparisons later today. If there aren't other updates to autoCond expected in the next 2/3 days, we could integrate this PR by the end of the week together with the corresponding update of the HLT menus (PR to come). Otherwise, I'll check and possibly sign this PR sooner.

[tvami]

So my conclusion from the discussion above is that it's fine to keep the GTs as they are for now, in the future we should pay more attention to have the exact same snapshot for that two GTs.

[tvami]

@cmsbuild , please abort

• tests are stuck....

[tvami]

@cmsbuild , please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a50019/26672/summary.html
COMMIT: 5d47cce
CMSSW: CMSSW_12_5_X_2022-08-06-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38916/26672/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-a50019/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-a50019/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 55
• DQMHistoTests: Total histograms compared: 4147366
• DQMHistoTests: Total failures: 8732
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4138612
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 54 files compared)
• Checked 231 log files, 49 edm output root files, 55 DQM output files
• TriggerResults: found differences in 8 / 54 workflows

[tvami]

hi @missirol we have now another update for the autoCond, so this should be merged, should we do it semi-blind? given that the DQM tool is not working...

[tvami]

@cms-sw/alca-l2
looking at the diffs in
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_12_5_X_2022-08-06-1100+a50019/52000/11634.7_TTbar_14TeV+2021_trackingMkFit+TTbar_14TeV_TuneCP5_GenSim+Digi+RecoNano+HARVESTNano/HLT.html

I see jets and trigger related changes, which are all expected by the changes in this PR. Can you please sign so I dont sign my own PR?

[missirol]

we have now another update for the autoCond, so this should be merged, should we do it semi-blind? given that the DQM tool is not working...

I would try to wait a bit longer if possible, e.g. until tomorrow, hoping the DQM issue gets fixed. Would that be okay? In the meantime, I will check the available comparisons, like those from the link you shared.

[francescobrivio]

+alca

• given the conclusion in Adding JECs, PF and L1T menu to Run-3 MC GTs and data relval GTs #38916 (comment) and the fact that we have more updates for autoCond in the pipeline

[francescobrivio]

we have now another update for the autoCond, so this should be merged, should we do it semi-blind? given that the DQM tool is not working...

I would try to wait a bit longer if possible, e.g. until tomorrow, hoping the DQM issue gets fixed. Would that be okay? In the meantime, I will check the available comparisons, like those from the link you shared.

I guess we can hold off the changes in the pipeline untill tomorrow, but I would not delay too much if possible.
@cms-sw/dqm-l2 any update on the DQM tool?

[missirol]

+hlt

Did not find evidence of issues. Afaict, differences in PR comparisons are small, and I think that's consistent with expectations (more detailed physics validation of these changes was, or will be, done offline by the JME-POG and TSG).

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[tvami]

urgent

• @cms-sw/orp-l2 we have another update in the queue, please merge this asap thanks!

[qliphy]

+1