Fix snapshot in data offline GTs and Egamma regression, and update Strip 2018 MC conditions #28053

tocheng · 2019-09-23T21:03:38Z

PR description:

This PR contains a fix of snapshot for data offline GTs addressed at [1].
EGamma PF cluster correction tags[2] for 2018 ultra-legacy reprocessing are updated due a bug described at [3]. The corrections affect 2018 MC and the 2018 IOVs of the offline GTs. Since the same corrections used for 2018 are intended to be used for Run 3 as well, the Run 3 MC GTs are updated as well.
Strip MC conditions for 2018 are updated to have a better data V.S. MC comparison[4]

[1] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4223.html
[2] https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4222/1/1.html
[3] https://indico.cern.ch/event/842863/contributions/3562026/attachments/1906927/3149650/updates_2018ULPFCorrections.pdf
[4]https://indico.cern.ch/event/854616/#7-strip-mc-conditions-for-ul-2

GT diffs:
Data offline GTs
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_v24/106X_dataRun2_v23
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_relval_v22/106X_dataRun2_relval_v21

Data prompt-like GTs
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_v13/106X_dataRun2_PromptLike_v12
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_HEfail_v12/106X_dataRun2_PromptLike_HEfail_v11
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_PromptLike_HI_v13/106X_dataRun2_PromptLike_HI_v12

2018 MC GTs
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018_design_v6/106X_upgrade2018_design_v5
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018_realistic_v11/106X_upgrade2018_realistic_v8
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018_realistic_HEfail_v11/106X_upgrade2018_realistic_HEfail_v8
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018_realistic_HI_v6/106X_upgrade2018_realistic_HI_v5
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018cosmics_realistic_deco_v7/106X_upgrade2018cosmics_realistic_deco_v4
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018cosmics_realistic_peak_v8/106X_upgrade2018cosmics_realistic_peak_v4

Run 3 MC GTs
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2021_design_v5/106X_upgrade2021_design_v4
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2021_realistic_v11/106X_upgrade2021_realistic_v10
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2021cosmics_realistic_deco_v4/106X_upgrade2021cosmics_realistic_deco_v3

PR validation:

See the description above for the physics validation of the tags.

In addition, a technical test was performed: runTheMatrix.py -l limited -i all --ibeos

cmsbuild · 2019-09-23T21:05:43Z

A new Pull Request was created by @tocheng (Tongguang) for CMSSW_10_6_X.

It involves the following packages:

Configuration/AlCa

@christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

tocheng · 2019-09-23T21:06:21Z

@cmsbuild please test

cmsbuild · 2019-09-23T21:06:55Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2629/console Started: 2019/09/23 23:07

cmsbuild · 2019-09-23T23:18:26Z

+1
Tested at: d9b8da3
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b8d484/2629/summary.html

cmsbuild · 2019-09-23T23:18:30Z

Comparison job queued.

cmsbuild · 2019-09-24T01:16:52Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b8d484/2629/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4365 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3210961
DQMHistoTests: Total failures: 3546
DQMHistoTests: Total nulls: 3
DQMHistoTests: Total successes: 3207078
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 32 files compared)
DQMHistoSizes: changed ( 136.85 ): -0.004 KiB JetMET/SUSYDQM
Checked 137 log files, 14 edm output root files, 33 DQM output files

srimanob · 2019-09-24T12:49:31Z

@tocheng @tlampen @christopheralanwest @ggovi
Could you please check and sign?

@fabiocos I see that you don't build the release yet, should we integrate this PR to patch1 also? It's an update on autocond for the last GT for UL-2018.

cmsbuild · 2019-09-24T12:58:40Z

Pull request #28053 was updated. @christopheralanwest, @tocheng, @cmsbuild, @franzoni, @tlampen, @pohsun can you please check and sign again.

tocheng · 2019-09-24T12:59:02Z

@cmsbuild please test

cmsbuild · 2019-09-24T12:59:08Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2646/console Started: 2019/09/24 14:59

tocheng · 2019-09-24T12:59:56Z

@tocheng @tlampen @christopheralanwest @ggovi
Could you please check and sign?

@srimano Design GTs were not updated. I added new commit. If test passes, I will sign off this PR.

srimanob · 2019-09-24T13:37:38Z

We are really waiting for the new release for final validation. If it's not ready, I think I would like to propose to make the release now, and this goes to the next one.

cmsbuild · 2019-10-22T18:57:11Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b8d484/3095/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 6395 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3212305
DQMHistoTests: Total failures: 7078
DQMHistoTests: Total nulls: 3
DQMHistoTests: Total successes: 3204890
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -4.144 KiB( 32 files compared)
DQMHistoSizes: changed ( 136.85 ): -0.004 KiB JetMET/SUSYDQM
DQMHistoSizes: changed ( 250202.181 ): -0.632 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 7.3 ): -3.508 KiB SiStrip/MechanicalView
Checked 137 log files, 14 edm output root files, 33 DQM output files

tocheng · 2019-10-22T19:11:41Z

@cmsbuild please test workflow 7.2,7.4

cmsbuild · 2019-10-22T19:12:11Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/3103/console Started: 2019/10/22 21:12

cmsbuild · 2019-10-22T21:14:31Z

+1
Tested at: f25b40a
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b8d484/3103/summary.html

cmsbuild · 2019-10-22T21:14:34Z

Comparison job queued.

cmsbuild · 2019-10-22T23:57:22Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b8d484/3103/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 6395 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3212305
DQMHistoTests: Total failures: 7078
DQMHistoTests: Total nulls: 3
DQMHistoTests: Total successes: 3204890
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -4.144 KiB( 32 files compared)
DQMHistoSizes: changed ( 136.85 ): -0.004 KiB JetMET/SUSYDQM
DQMHistoSizes: changed ( 250202.181 ): -0.632 KiB SiStrip/MechanicalView
DQMHistoSizes: changed ( 7.3 ): -3.508 KiB SiStrip/MechanicalView
Checked 137 log files, 14 edm output root files, 33 DQM output files

tocheng · 2019-10-23T11:16:56Z

@tocheng

We hope this PR #28134 can be merged first.

I see that #28134 is merged. What's the plan to:
* update the data offline GT snapshot also for master?

* introduce the new SiStrip tags for 2018 simulation also for master?
Thanks.

Hello @mmusich , the last commit adds the peak mode noise to the GT.

tocheng · 2019-11-05T09:31:29Z

+1
2018UL validation reported at PPD general meetings
https://indico.cern.ch/event/853103/
https://indico.cern.ch/event/857175/
The 2018 UL conditions are merged into master.

cmsbuild · 2019-11-05T09:32:05Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-11-05T17:58:27Z

+1

cmsbuild added this to the CMSSW_10_6_X milestone Sep 23, 2019

cmsbuild added alca-pending comparison-pending orp-pending pending-signatures tests-pending labels Sep 23, 2019

cmsbuild added tests-started and removed tests-pending labels Sep 23, 2019

cmsbuild added tests-approved and removed tests-started labels Sep 23, 2019

cmsbuild added comparison-available and removed comparison-pending labels Sep 24, 2019

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Sep 24, 2019

cmsbuild added tests-started and removed tests-pending labels Sep 24, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 22, 2019

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Oct 22, 2019

cmsbuild added tests-started and removed tests-pending labels Oct 22, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 22, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 22, 2019

christopheralanwest mentioned this pull request Oct 24, 2019

Use corrected PF cluster correction tags for 2018 design MC scenario #28263

Merged

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Nov 5, 2019

cmsbuild added orp-approved and removed orp-pending labels Nov 5, 2019

cmsbuild merged commit e240e6d into cms-sw:CMSSW_10_6_X Nov 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix snapshot in data offline GTs and Egamma regression, and update Strip 2018 MC conditions #28053

Fix snapshot in data offline GTs and Egamma regression, and update Strip 2018 MC conditions #28053

tocheng commented Sep 23, 2019 •

edited

Loading

cmsbuild commented Sep 23, 2019

tocheng commented Sep 23, 2019

cmsbuild commented Sep 23, 2019 •

edited

Loading

cmsbuild commented Sep 23, 2019

cmsbuild commented Sep 23, 2019

cmsbuild commented Sep 24, 2019

srimanob commented Sep 24, 2019

cmsbuild commented Sep 24, 2019

tocheng commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 •

edited

Loading

tocheng commented Sep 24, 2019 •

edited

Loading

srimanob commented Sep 24, 2019 •

edited

Loading

cmsbuild commented Oct 22, 2019

tocheng commented Oct 22, 2019

cmsbuild commented Oct 22, 2019 •

edited

Loading

cmsbuild commented Oct 22, 2019

cmsbuild commented Oct 22, 2019

cmsbuild commented Oct 22, 2019

tocheng commented Oct 23, 2019

tocheng commented Nov 5, 2019

cmsbuild commented Nov 5, 2019

fabiocos commented Nov 5, 2019

Fix snapshot in data offline GTs and Egamma regression, and update Strip 2018 MC conditions #28053

Fix snapshot in data offline GTs and Egamma regression, and update Strip 2018 MC conditions #28053

Conversation

tocheng commented Sep 23, 2019 • edited Loading

PR description:

PR validation:

cmsbuild commented Sep 23, 2019

tocheng commented Sep 23, 2019

cmsbuild commented Sep 23, 2019 • edited Loading

cmsbuild commented Sep 23, 2019

cmsbuild commented Sep 23, 2019

cmsbuild commented Sep 24, 2019

srimanob commented Sep 24, 2019

cmsbuild commented Sep 24, 2019

tocheng commented Sep 24, 2019

cmsbuild commented Sep 24, 2019 • edited Loading

tocheng commented Sep 24, 2019 • edited Loading

srimanob commented Sep 24, 2019 • edited Loading

cmsbuild commented Oct 22, 2019

tocheng commented Oct 22, 2019

cmsbuild commented Oct 22, 2019 • edited Loading

cmsbuild commented Oct 22, 2019

cmsbuild commented Oct 22, 2019

cmsbuild commented Oct 22, 2019

tocheng commented Oct 23, 2019

tocheng commented Nov 5, 2019

cmsbuild commented Nov 5, 2019

fabiocos commented Nov 5, 2019

tocheng commented Sep 23, 2019 •

edited

Loading

cmsbuild commented Sep 23, 2019 •

edited

Loading

cmsbuild commented Sep 24, 2019 •

edited

Loading

tocheng commented Sep 24, 2019 •

edited

Loading

srimanob commented Sep 24, 2019 •

edited

Loading

cmsbuild commented Oct 22, 2019 •

edited

Loading