Read VertexSmearing from GT in Run-1/2/3 MC

[francescobrivio]

PR description:

Following the inclusion of the SimBeamSpotObject tags in all Run-1/2/3 MC GTs in #43197, this PR aims at modifying the corresponding MC workflows to actually consume it.
This has several advantages:

• makes obsolete the need to build a new release each time a new vertex smearing scenario is introduced (more than twice per year), easing operations for both ORP and PPD
• this should also ease operations for PdmV by removing the need of updating the --beamspot cmsdriver argument for each different campaign (MC production and relvals)

Following the conservation law, of course the burden is now completely moved onto @cms-sw/alca-l2 and BeamSpot experts! 😉

Two new VtxSmearingParameters configs are defined:

• IOMC.EventVertexGenerators.VtxSmearedDesign_cfi which uses the GaussEvtVtxGenerator
• IOMC.EventVertexGenerators.VtxSmearedRealistic_cfi which uses the BetafuncEvtVtxGenerator
• both are reading the configurable parameters from the DB

The new configs are used in Configuration/StandardSequences/python/VtxSmeared.py by defining two new keys:

• DBdesign for the MC productions in ideal conditions (GaussSmearing)
• DBrealistic for the MC production in realistic conditions (BetafuncSmearing)

PR validation:

Successfully tested wf 12434.0 which shows as (updated) step1 driver:

cmsDriver.py TTbar_14TeV_TuneCP5_cfi  -s GEN,SIM -n 10 \
  --conditions auto:phase1_2023_realistic \
  --beamspot DBrealistic \
  --datatier GEN-SIM --eventcontent FEVTDEBUG \
  --geometry DB:Extended --era Run3_2023 --relval 9000,100 

Given the SimBeamSpot tags introduced in the GTs mimic exactly the previously used parameters, we don't expect any difference in the test comparisons.

Backport:

Not a backport, but a 13_3_X backport will be provided.

[francescobrivio]

@cms-sw/pdmv-l2 please advise if I should update also some other configuration(s), I only changed upgradeWorkflowComponents, but most probably I have missed a few sposts.

Also, I noticed that here:

cmssw/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

Lines 2557 to 2563 in 95c6900

	'2017' : {
	'Geom' : 'DB:Extended',
	'GT' : 'auto:phase1_2017_realistic',
	'HLTmenu': '@relval2017',
	'Era' : 'Run2_2017',
	'ScenToRun' : ['GenSim','Digi','RecoFakeHLT','HARVESTFakeHLT','ALCA','Nano'],
	},

we are missing the 'BeamSpot' key, is that expected?

[francescobrivio]

test parameters:

• pull_request = Add SimBeamSpotObject record to Run-1/2/3 MC GTs #43197

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43242/37599

• This PR adds an extra 36KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, Porting Pixel Tracks to Alpaka [Not to Merge]  #41117, Guard for No Digis for Phase2 and Patatrack NuGun #41255, Add Phase-2 workflow with Phase-2 ECal trigger primitive, also with ECal component #43057
  • File Configuration/StandardSequences/python/VtxSmeared.py modified in PR(s): Vertex smearing for PbPb MC based on 2023 PbPb data #43217

[cmsbuild]

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)
• Configuration/StandardSequences (operations)
• IOMC/EventVertexGenerators (simulation)

@civanch, @miquork, @mdhildreth, @AdrianoDee, @antoniovilela, @cmsbuild, @rappoccio, @fabiocos, @sunilUIET, @srimanob, @davidlange6 can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @grzanka, @felicepantaleo, @sameasy, @slomeo, @Martin-Grunewald, @rovere, @missirol, @mtosi, @mmusich, @fabiocos, @makortel, @VinInn, @dgulhan, @JanFSchulte, @VourMa this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

[francescobrivio]

@cmsbuild please test

• just to make sure no differences are present in the comparison tests, while waiting for comments on the PR

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-567a75/35736/summary.html
COMMIT: f88656d
CMSSW: CMSSW_14_0_X_2023-11-09-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43242/35736/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 354 lines from the logs
• Reco comparison results: 2472 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3363010
• DQMHistoTests: Total failures: 4120
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3358868
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 214 log files, 167 edm output root files, 50 DQM output files
• TriggerResults: found differences in 1 / 48 workflows

[mmusich]

The new configs are used in Configuration/StandardSequences/python/VtxSmeared.py by defining two new keys:

• DBdesign for the MC productions in ideal conditions (GaussSmearing)
• DBrealistic for the MC production in realistic conditions (BetafuncSmearing)

I have to admit that when I originally thought about this, I had in mind something more like the mechanism from which we fetch the geometry, in which we specify in the cmsDriver command DB:<label_in_GT>:

cmssw/Configuration/Applications/python/ConfigBuilder.py

Lines 1168 to 1180 in 95c6900

	if (geoms[0].startswith('DB:')):
	self.SimGeometryCFF='Configuration/StandardSequences/GeometrySimDB_cff'
	self.geometryDBLabel=geoms[0][3:]
	print("with DB:")
	else:
	if '/' in geoms[0] or '_cff' in geoms[0]:
	self.SimGeometryCFF=geoms[0]
	else:
	simGeometry=geoms[0]
	if self._options.gflash==True:
	self.SimGeometryCFF='Configuration/Geometry/Geometry'+geoms[0]+'GFlash_cff'
	else:
	self.SimGeometryCFF='Configuration/Geometry/Geometry'+geoms[0]+'_cff'

such that a single GlobalTag can cater different vertex smearing scenarios. On the other hand having two different EvtVtxGenerator-s complicates the picture. Just leaving the comment here for the record, in case someone finds this more appropriate.

[mmusich]

VtxSmearedCommon is a simple enough PSet to be used directly instead of being imported IMHO. Unfortunately we can't loose the cms type specification as neither GaussEvtVtxGenerator nor BetafuncEvtVtxGenerator implement fillDescription methods.

[francescobrivio]

In 1a98ef1 I:

• added the filDescription methods to GaussEvtVtxGenerator and BetafuncEvtVtxGenerator
• used the autogenerated cfi(s) in VtxSmearedDesign_cfi.py and VtxSmearedRealistic_cfi.py (thus dropping the cms type specifications)

[francescobrivio]

I'll wait comments/feedback, especially from @cms-sw/pdmv-l2, before squashing and retesting.

[civanch]

+1

[AdrianoDee]

+1

[srimanob]

+Upgrade

[francescobrivio]

@cms-sw/orp-l2 this is now basically fully signed and only waiting on your signature.

[rappoccio]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

[AdrianoDee]

Hi, I understand this approach is usable also for HI and I was looking at these:

cmssw/Configuration/PyReleaseValidation/python/relval_steps.py

Lines 38 to 43 in 8e27618

	step1Up2022HiProdDefaults = merge ([{'--conditions':'auto:phase1_2022_realistic_hi','--era':'Run3_pp_on_PbPb','--beamspot':'Realistic2022PbPbCollision','--eventcontent':'RAWSIM','--geometry':'DB:Extended'},step1Up2018HiDefaults])
	step1Up2023HiProdDefaults = merge ([{'--conditions':'auto:phase1_2023_realistic_hi','--era':'Run3_pp_on_PbPb_2023','--beamspot':'Realistic2023PbPbCollision','--eventcontent':'RAWSIM','--geometry':'DB:Extended'},step1Up2018HiDefaults])
	# step1 gensim: for 2024
	step1Up2024Defaults = merge ([{'--conditions':'auto:phase1_2024_realistic','--era':'Run3','--beamspot':'Realistic25ns13p6TeVEarly2023Collision'},step1Up2018Defaults])
	# step1 gensim: for 2024 HI prod
	step1Up2024HiProdDefaults = merge ([{'--conditions':'auto:phase1_2024_realistic_hi','--era':'Run3_pp_on_PbPb_2024','--beamspot':'Realistic2023PbPbCollision'},step1Up2023HiProdDefaults])

Seems to me we could also update it there to DBRealistic, right?

(@cms-sw/alca-l2 )

[cmsbuild]

cms-bot internal usage

[mmusich]

Seems to me we could also update it there to DBRealistic, right?

I think so: #45346