Final global tags for 2017 UL reprocessing #27342

christopheralanwest · 2019-06-26T02:50:07Z

PR description:

This PR updates the global tags relevant for the 2017 UL reprocessing, as well as a fix for PPS conditions in prompt-like global tags. Both data and MC global tags are updated.

The changes are as follows:

106X_dataRun2_v17:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_v17/106X_dataRun2_v13

106X_dataRun2_relval_v16:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_relval_v16/106X_dataRun2_relval_v12

106X_dataRun2_PromptLike_HEfail_v9:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_HEfail_v9/106X_dataRun2_PromptLike_HEfail_v8

106X_dataRun2_PromptLike_v10:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_v10/106X_dataRun2_PromptLike_v9

106X_dataRun2_PromptLike_HI_v10:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_HI_v10/106X_dataRun2_PromptLike_HI_v9

106X_mc2017_realistic_v7:
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_mc2017_realistic_v7/106X_mc2017_realistic_v5

PR validation:

An enormous amount of work went into validating the conditions for this PR. Details can be found at the sign-off talks listed at:

As a technical check, I ran runTheMatrix.py -l limited -i all --ibeos

if this PR is a backport please specify the original PR:

This PR is not a backport but a backport will be prepared for 10_6_X.

cmsbuild · 2019-06-26T02:50:38Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-06-26T02:57:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27342/10605

This PR adds an extra 12KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Adding Run 330098 (2019 MWGR3) in data relvals #27341, Update 2021 global tags to include new pixel conditions #27334

cmsbuild · 2019-06-26T02:57:47Z

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

Configuration/AlCa

@christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

christopheralanwest · 2019-06-26T03:04:25Z

please test

cmsbuild · 2019-06-26T03:04:51Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1161/console Started: 2019/06/26 05:19

cmsbuild · 2019-06-26T06:14:42Z

+1
Tested at: 95a0cac
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1161/summary.html

cmsbuild · 2019-06-26T06:14:45Z

Comparison job queued.

cmsbuild · 2019-06-26T09:03:51Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1161/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 10052 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3254598
DQMHistoTests: Total failures: 2630
DQMHistoTests: Total nulls: 28
DQMHistoTests: Total successes: 3251606
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.008 KiB( 32 files compared)
DQMHistoSizes: changed ( 4.53,... ): -0.012 KiB JetMET/SUSYDQM
DQMHistoSizes: changed ( 1000.0 ): 0.004 KiB JetMET/SUSYDQM
DQMHistoSizes: changed ( 136.85 ): 0.016 KiB JetMET/SUSYDQM
DQMHistoSizes: changed ( 136.731 ): -0.004 KiB JetMET/SUSYDQM
Checked 137 log files, 14 edm output root files, 33 DQM output files

christopheralanwest · 2019-06-26T12:54:26Z

The differences in pre-2017 workflows are not expected. They appear to only affect EcalRecHitsSorted collections and downstream quantities. The differences arise because a change has been made to the pre-2017 history of the EcalPFRecHitThresholdsRcd tag:

lxplus793>conddb diff EcalPFRecHitThresholds_UL_Run1_Run2_2017_v1 EcalPFRecHitThresholds_2018_v1_offline
[2019-06-26 14:49:33,059] INFO: Connecting to pro [frontier://PromptProd/CMS_CONDITIONS]
Range               pro::EcalPFRecHitThresholds_UL_Run1_Run2_2017_v1 Payload  pro::EcalPFRecHitThresholds_2018_v1_offline Payload  
------------------  --------------------------------------------------------  ---------------------------------------------------  
[1, 265475)         9fdf3a78064032372f6107368c8da1c1a44a94f2                  9f22e391bcdcabe1f9f13608961bbed938c8a05c             
[265475, 286694)    0d6c3b61b14fec97f10bfba826350bc7f2a8f413                  9f22e391bcdcabe1f9f13608961bbed938c8a05c             
[286694, Infinity)  f6ae126e7828a2b6f89869d65bdfd1902f9574f7                  9f22e391bcdcabe1f9f13608961bbed938c8a05c

I will follow up with ECAL experts offline to have this fixed.

cmsbuild · 2019-07-01T16:32:42Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27342/10716

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Adding Run 330098 (2019 MWGR3) in data relvals #27341

cmsbuild · 2019-07-01T16:34:37Z

Pull request #27342 was updated. @christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please check and sign again.

christopheralanwest · 2019-07-01T16:37:48Z

please test

cmsbuild · 2019-07-01T16:38:14Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1287/console Started: 2019/07/01 18:38

christopheralanwest · 2019-07-01T16:44:37Z

The last update introduced two changes:

The beamspot tag used by the offline data GT was updated by appending the BeamSpotObjects_post2017_LumiBased_v1 to a copy of the BeamSpotObjects_2017UL_LumiBased_v1 tag. This was done to ensure that the 2018 history is unchanged.
A purely technical change to the 2017 realistic MC GT: an obsolete, unused record was removed.

cmsbuild · 2019-07-01T18:21:18Z

+1
Tested at: ca5fbe7
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1287/summary.html

cmsbuild · 2019-07-01T18:21:21Z

Comparison job queued.

cmsbuild · 2019-07-01T20:38:44Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-470899/1287/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 6874 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3290686
DQMHistoTests: Total failures: 9266
DQMHistoTests: Total nulls: 6
DQMHistoTests: Total successes: 3281080
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.012 KiB( 32 files compared)
DQMHistoSizes: changed ( 136.788 ): -0.012 KiB JetMET/SUSYDQM
Checked 137 log files, 14 edm output root files, 33 DQM output files

christopheralanwest · 2019-07-01T21:49:55Z

+1

Changes seen only in 2017 workflows, except for PPS quantities in 2018 data. PPS is a special case as its reconstruction software was developed in parallel to the 2017 UL preparation and therefore there is no requirement to leave the 2018 PPS quantities unchanged during the 2017 UL preparation.

cmsbuild · 2019-07-01T21:50:22Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-07-02T08:32:14Z

+1

christopheralanwest added 2 commits June 26, 2019 04:36

Update offline data GTs for 2017 ultra-legacy reprocessing

65f0300

Update prompt-like global tags to fix vertical dispersion issue for PPS

95a0cac

cmsbuild added this to the CMSSW_11_0_X milestone Jun 26, 2019

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending pending-signatures tests-pending labels Jun 26, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 26, 2019

cmsbuild added tests-started and removed tests-pending labels Jun 26, 2019

cmsbuild added tests-approved and removed tests-started labels Jun 26, 2019

cmsbuild added comparison-available and removed comparison-pending labels Jun 26, 2019

Finalize 2017 MC GT for ultra-legacy reprocessing

858668c

cmsbuild removed code-checks-approved comparison-available tests-approved labels Jun 26, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 1, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 1, 2019

cmsbuild added tests-approved and removed tests-started labels Jul 1, 2019

cmsbuild added comparison-available and removed comparison-pending labels Jul 1, 2019

christopheralanwest mentioned this pull request Jul 1, 2019

Final global tags for 2017 UL reprocessing [10_6_X] #27420

Merged

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Jul 1, 2019

cmsbuild added orp-approved and removed orp-pending labels Jul 2, 2019

cmsbuild merged commit 6afa483 into cms-sw:master Jul 2, 2019

cmsbuild mentioned this pull request Jul 2, 2019

make some internal globals const in TkBfield.cc #27422

Merged

mmusich mentioned this pull request Aug 5, 2019

remove stuck-TBM specific workflows #27692

Merged

christopheralanwest deleted the alca-2017ul-gt branch August 21, 2019 14:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Final global tags for 2017 UL reprocessing #27342

Final global tags for 2017 UL reprocessing #27342

christopheralanwest commented Jun 26, 2019 •

edited

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

christopheralanwest commented Jun 26, 2019

cmsbuild commented Jun 26, 2019 •

edited

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

christopheralanwest commented Jun 26, 2019

cmsbuild commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

christopheralanwest commented Jul 1, 2019

cmsbuild commented Jul 1, 2019 •

edited

christopheralanwest commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

christopheralanwest commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

fabiocos commented Jul 2, 2019

Final global tags for 2017 UL reprocessing #27342

Final global tags for 2017 UL reprocessing #27342

Conversation

christopheralanwest commented Jun 26, 2019 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

christopheralanwest commented Jun 26, 2019

cmsbuild commented Jun 26, 2019 • edited

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

cmsbuild commented Jun 26, 2019

christopheralanwest commented Jun 26, 2019

cmsbuild commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

christopheralanwest commented Jul 1, 2019

cmsbuild commented Jul 1, 2019 • edited

christopheralanwest commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

christopheralanwest commented Jul 1, 2019

cmsbuild commented Jul 1, 2019

fabiocos commented Jul 2, 2019

christopheralanwest commented Jun 26, 2019 •

edited

cmsbuild commented Jun 26, 2019 •

edited

cmsbuild commented Jul 1, 2019 •

edited