Update BeamSpot for 2022 EOY MC #41123

francescobrivio · 2023-03-21T17:22:06Z

PR description:

This PR updates the MC BeamSpot to be used in the "2022 EOY" MC production planned for later this year.
The updates are only related to 2022 (since in 2023 the BeamSpot will be most probably be moved) and include:

2022 MC GTs updated with new BeamSpot tags:
- BeamSpotObjects_Realistic25ns_13p6TeVCollisions_EOY2022_v2_mc
- BeamSpotOnlineObjects_Realistic25ns_13p6TeVCollisions_EOY2022_v2_mc
Updated in upgradeWorkflowComponents.py the vtx smearing for 2021 (used in 2022 relvals) to Realistic25ns13p6TeVEOY2022Collision in order to be consistent with the GT
Updated the GEN-SIM MinBias sample used for 2021 (realistic case) in relval_steps.py

GT diffs:

phase1_2022_realistic: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/131X_mcRun3_2022_realistic_v1/131X_mcRun3_2022_realistic_v2
phase1_2022_realistic_postEE: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/131X_mcRun3_2022_realistic_postEE_v1/131X_mcRun3_2022_realistic_postEE_v2
phase1_2022_cosmics: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/131X_mcRun3_2022cosmics_realistic_deco_v1/131X_mcRun3_2022cosmics_realistic_deco_v2

The [RFC] is due to the fact that i'm not 100% sure if the updates I made are enough or more changes are needed, e.g. looking at #38760 I can see that several files that I didn't touch are using the (now) deprecated Realistic25ns13p6TeVEarly2022Collision vertex smearing...should I update those as well to be consistent?
FYI @mmusich

PR validation:

Succesfully ran 11834.0 workflow

Backport:

Not a backport - no backport needed

francescobrivio · 2023-03-21T17:26:47Z

test parameters:

workflow = 11834.0

cmsbuild · 2023-03-21T17:29:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41123/34761

This PR adds an extra 68KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Include Phase2 TkAl Rcds in Phase2 MC GT with T21 tags #41119, fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120, HI 2023 relval wfs + realistic 2022 vtx smearing #41095
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Move RecoPixelVertexing subsystems to RecoTracker #41058, reverting back to run3_data_relval for making rereco GT default in relval [13_1_X] #41077, Update short matrix to use Phase-2 D95 geometry #41003, Improved method for EB digi simulation #40855, HI 2023 relval wfs + realistic 2022 vtx smearing #41095
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Adding all geometries and the FullRecoGPU for Patatrack wfs for Phase2 #41011, Move RecoPixelVertexing subsystems to RecoTracker #41058, Add run3 tracking low pu era #33532, Displaced regional tracking iteration #40643

cmsbuild · 2023-03-21T17:29:26Z

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

Configuration/AlCa (alca)
Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @cmsbuild, @AdrianoDee, @srimanob, @saumyaphor4252, @kskovpen, @sunilUIET, @tvami, @francescobrivio can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @Martin-Grunewald, @missirol, @tocheng, @mmusich, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

francescobrivio · 2023-03-22T08:47:16Z

@cmsbuild please test

let's see how the tests come out while we wait for comments...

cmsbuild · 2023-03-22T16:54:02Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16f741/31504/summary.html
COMMIT: eac0fb7
CMSSW: CMSSW_13_1_X_2023-03-21-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41123/31504/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

11601.0DAS Error
11602.0DAS Error
11603.0DAS Error

Expand to see more relval errors ...

11604.0
11605.0
11606.0
11607.0
11608.0
11609.0
11630.0
11634.0
11634.1
11634.15
11634.24
11634.5
11634.501
11634.505
11634.511
11634.521
11634.601
11634.7
11634.91
11640.0
11643.0
11646.0
11650.0
11650.501
11650.505
11723.17
11834.0
11834.13
11834.19
11834.21
11834.24
11834.99
11846.0
11846.19

Comparison Summary

Summary:

You potentially added 8 lines to the logs
Reco comparison results: 16778 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3552750
DQMHistoTests: Total failures: 36726
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3516002
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.064 KiB( 48 files compared)
DQMHistoSizes: changed ( 11834.0 ): 1.064 KiB SiStrip/MechanicalView
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: found differences in 5 / 47 workflows

francescobrivio · 2023-03-23T08:55:16Z

Well the tests failed due to DAS errors.
Before restarting them, @cms-sw/pdmv-l2 @cms-sw/upgrade-l2 do you have answers/comments to my questions in the PR description?

francescobrivio · 2023-03-24T07:18:35Z

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 a kind ping

srimanob · 2023-03-24T07:38:22Z

Hi @francescobrivio

The error is not actually the DAS error, but because datasets do not exist. If you would like to merge this PR, full sets of relvals need to be produced.

francescobrivio · 2023-03-24T07:43:15Z

Hi Phat!

But the GS dataset actually exists on DAS (see link here) and locally I was able to succesfully run:

runTheMatrix.py -l 11834.0 -j 8

so what am I missing? 😄
What is the full set of relval that is needed? @cms-sw/pdmv-l2 I guess then your intervention is needed here.

srimanob · 2023-03-24T07:50:58Z

From https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16f741/31504/dasqueries/runall-report-step123-.log, if you search on exit: 1, you should get a list of missing samples. For example, /RelValSingleElectronPt10/CMSSW_13_1_0_pre1-130X_mcRun3_2022_realistic_withNewBSFromEOY2022Data_v2_RV186-v1/GEN-SIM is missing (at least, I don't see in DAS).

francescobrivio · 2023-03-24T09:09:31Z

Thanks a lot Phat!

So I guess we should generate the relvals with:

CMSSW_13_1_0_pre1 (or pre2 - doesn't really matter here)
GT: 131X_mcRun3_2022_realistic_v2
GS step customized with --beamspot Realistic25ns13p6TeVEOY2022Collision

Should I open a JIRA @cms-sw/pdmv-l2 ?

srimanob · 2023-03-24T09:12:20Z

Hi @francescobrivio @cms-sw/pdmv-l2
It's matter on release and process string. The relvals need to be produced with CMSSW_13_1_0_pre1, and the process string that you will get
CMSSW_13_1_0_pre1-130X_mcRun3_2022_realistic_withNewBSFromEOY2022Data_v2_RV186-v in the dataset name.

Basically, the same setting that pdmv used to produce minbias.

francescobrivio · 2023-03-24T13:06:35Z

Just as update: @bbilin was super kind to submit the needed relvals very fast, see
https://cms-pdmv.cern.ch/relval/relvals?ticket=CMSSW_13_1_0_pre1__fullsim_noPU_2022_14TeV-00002&shown=33791&page=0&limit=50&sort=prepid&sort_asc=true

AFAIU the proc string should be unchanged, so once the relvals are done I will re-trigger the tests.

cmsbuild · 2023-04-03T06:28:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41123/34995

This PR adds an extra 68KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Adding all geometries and the FullRecoGPU for Patatrack wfs for Phase2 #41011, Add Wf *.81 for BPH DQM #41236, Add run3 tracking low pu era #33532, Displaced regional tracking iteration #40643

cmsbuild · 2023-04-03T06:28:51Z

Pull request #41123 was updated. @bbilin, @AdrianoDee, @srimanob, @saumyaphor4252, @kskovpen, @sunilUIET, @tvami, @francescobrivio can you please check and sign again.

cmsbuild · 2023-04-03T14:22:58Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16f741/31740/summary.html
COMMIT: eecdd67
CMSSW: CMSSW_13_1_X_2023-04-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41123/31740/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 16 lines to the logs
Reco comparison results: 16822 differences found in the comparisons
DQMHistoTests: Total files compared: 47
DQMHistoTests: Total histograms compared: 3364817
DQMHistoTests: Total failures: 37667
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3327128
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.229 KiB( 46 files compared)
DQMHistoSizes: changed ( 11834.0 ): -0.229 KiB SiStrip/MechanicalView
Checked 202 log files, 155 edm output root files, 47 DQM output files
TriggerResults: found differences in 5 / 45 workflows

tvami · 2023-04-03T15:07:54Z

+alca

widespread changes, which is expected since regenerated samples

srimanob · 2023-04-03T15:36:09Z

Finally!!

srimanob · 2023-04-03T15:36:17Z

+Upgrade

bbilin · 2023-04-03T15:39:37Z

+pdmv

cmsbuild · 2023-04-03T15:40:00Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2023-04-03T16:12:21Z

+1

cmsbuild added this to the CMSSW_13_1_X milestone Mar 21, 2023

cmsbuild added alca-pending pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending labels Mar 21, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 21, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 22, 2023

cmsbuild mentioned this pull request Mar 22, 2023

fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120

Merged

cmsbuild added tests-rejected and removed tests-started labels Mar 22, 2023

cmsbuild mentioned this pull request Mar 23, 2023

Add HCAL LLP Jet Skim config #41111

Closed

cmsbuild mentioned this pull request Mar 24, 2023

Add upper cut on jet time to HLTJetTimingFilter #41089

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 3, 2023

cmsbuild mentioned this pull request Apr 3, 2023

Guard for No Digis for Phase2 and Patatrack NuGun #41255

Merged

cmsbuild added tests-approved and removed tests-started labels Apr 3, 2023

cmsbuild added alca-approved and removed alca-pending labels Apr 3, 2023

francescobrivio changed the title ~~[RFC] Update BeamSpot for 2022 EOY MC~~ Update BeamSpot for 2022 EOY MC Apr 3, 2023

cmsbuild added upgrade-approved and removed upgrade-pending labels Apr 3, 2023

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Apr 3, 2023

cmsbuild added orp-approved and removed orp-pending labels Apr 3, 2023

cmsbuild merged commit e55bc82 into cms-sw:master Apr 3, 2023

This was referenced Apr 3, 2023

Adding HCAL FB LUT to caloParams for uploading to DB #41221

Merged

[PGO] Initial changes to support PGO builds cms-sw/cmsdist#8420

Merged

mmusich mentioned this pull request Apr 27, 2023

RECO timing increase in CMSSW_13_1_0_pre3 for 11834.21 #41441

Open

francescobrivio deleted the alca-2022_MC_BS_EOY_13_1_X branch July 12, 2023 16:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update BeamSpot for 2022 EOY MC #41123

Update BeamSpot for 2022 EOY MC #41123

francescobrivio commented Mar 21, 2023

francescobrivio commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

francescobrivio commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

francescobrivio commented Mar 23, 2023

francescobrivio commented Mar 24, 2023

srimanob commented Mar 24, 2023

francescobrivio commented Mar 24, 2023

srimanob commented Mar 24, 2023

francescobrivio commented Mar 24, 2023

srimanob commented Mar 24, 2023 •

edited

francescobrivio commented Mar 24, 2023

cmsbuild commented Apr 3, 2023

cmsbuild commented Apr 3, 2023

cmsbuild commented Apr 3, 2023

tvami commented Apr 3, 2023

srimanob commented Apr 3, 2023

srimanob commented Apr 3, 2023

bbilin commented Apr 3, 2023

cmsbuild commented Apr 3, 2023

rappoccio commented Apr 3, 2023

Update BeamSpot for 2022 EOY MC #41123

Update BeamSpot for 2022 EOY MC #41123

Conversation

francescobrivio commented Mar 21, 2023

PR description:

PR validation:

Backport:

francescobrivio commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

francescobrivio commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

RelVals-INPUT

Comparison Summary

francescobrivio commented Mar 23, 2023

francescobrivio commented Mar 24, 2023

srimanob commented Mar 24, 2023

francescobrivio commented Mar 24, 2023

srimanob commented Mar 24, 2023

francescobrivio commented Mar 24, 2023

srimanob commented Mar 24, 2023 • edited

francescobrivio commented Mar 24, 2023

cmsbuild commented Apr 3, 2023

cmsbuild commented Apr 3, 2023

cmsbuild commented Apr 3, 2023

Comparison Summary

tvami commented Apr 3, 2023

srimanob commented Apr 3, 2023

srimanob commented Apr 3, 2023

bbilin commented Apr 3, 2023

cmsbuild commented Apr 3, 2023

rappoccio commented Apr 3, 2023

srimanob commented Mar 24, 2023 •

edited