Add workflow to cross run number boundaries in IB tests

[christopheralanwest]

PR description:

PR #28214 implemented workflow 250202.183, which was the only workflow that would cross run boundaries for tests of run-dependent MC. It was removed a few months later when run-dependent workflows were "cleaned up" in PR #29457. Other run-dependent workflows are intended for use with full relvals and so many lumiblocks must be crossed before changing the run number. Since AlCa would like to foster further development of run-dependent MC, it is very useful to test that IOV boundaries can be crossed in MC without crashing, and I would like to have this workflow back.

I also added an autoCond key phase1_2018_realistic_rd with a value corresponding to a MC GT for 2018 run-dependent MC. The time-dependent relvals have been updated to use this key rather than conditions overrides.

PR validation:

After running runTheMatrix.py -l 250202.183, I checked that the DIGI-PREMIX and RECO files contain events with two different run numbers:

• Run 315257: events 1-5
• Run 316083: events 6-10

This is the expectation given the configuration.

runTheMatrix.py -l 250202.182 also completes successfully.

if this PR is a backport please specify the original PR and why you need to backport that PR:

This PR is not a backport and should not be backported.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32786/20953

• This PR adds an extra 48KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel workflows (12/N) #31854, adding MWGR5 run with all detectors (including GEM) in Run3 data workflow #32780

[cmsbuild]

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

Configuration/PyReleaseValidation

@jordan-martins, @chayanit, @wajidalikhan, @kpedro88, @cmsbuild, @srimanob can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @fabiocos, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[christopheralanwest]

test parameters

• workflow = 250202.182, 250202.183

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32786/21033

• This PR adds an extra 52KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, adding MWGR5 run with all detectors (including GEM) in Run3 data workflow #32780, Patatrack integration - Pixel workflows (12/N) #31854

[christopheralanwest]

test parameters

• workflow = 250202.182, 250202.183

[cmsbuild]

Pull request #32786 was updated. @malbouis, @yuanchao, @jordan-martins, @chayanit, @wajidalikhan, @christopheralanwest, @kpedro88, @cmsbuild, @srimanob, @tlampen, @pohsun, @francescobrivio can you please check and sign again.

[christopheralanwest]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8c6ab/12743/summary.html
COMMIT: b17b60e
CMSSW: CMSSW_11_3_X_2021-02-05-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32786/12743/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e8c6ab/250202.182_TTbar_13UP18_RD+TTbar_13UP18_RD+DIGIPRMXUP18_PU25_RD+RECOPRMXUP18_PU25_RD+HARVESTUP18_PU25_RD
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e8c6ab/250202.183_TTbar_13UP18_RD+TTbar_13UP18_RD+DIGIPRMXUP18_PU25_RD_IB+RECOPRMXUP18_PU25_RD+HARVESTUP18_PU25_RD

Summary:

• No significant changes to the logs found
• Reco comparison results: 3 differences found in the comparisons
• DQMHistoTests: Total files compared: 37
• DQMHistoTests: Total histograms compared: 2751765
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 2751736
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 36 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 156 log files, 37 edm output root files, 37 DQM output files

[christopheralanwest]

+1

From the logs for step 2 of workflow 250202.183, we see that the IOV boundary is crossed at the expected location:

Begin processing the 5th record. Run 315257, Event 5, LumiSection 1 on stream 0 at 05-Feb-2021 20:38:33.889 CET
Begin processing the 6th record. Run 316083, Event 6, LumiSection 2 on stream 0 at 05-Feb-2021 20:38:51.571 CET

[srimanob]

+Upgrade

@christopheralanwest
Do you intend to run 250202.183 in IB? How to add it? Thanks.

[chayanit]

+1

the same questions as @srimanob, where would you add 250202.183 in IB?

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[qliphy]

+1