Update QGL tag for 2016 and 2018 MC and Run 2 data + Introduce scenario for 2016 cosmics with realistic conditions [10_6_X] #32247

christopheralanwest · 2020-11-23T19:44:20Z

PR description:

This PR is a combined backport of PRs #32224 and #31846. It updates the quark-gluon likelihood tag for 2016 and 2018 MC and Run 2 data; 2017 MC already has the correct tag. The update has already been included in the 102X series for the nanoAOD campaign but the change was never propagated to 10_6_X and master.

See the HN thread for details.

Also, the 2016 Run 2 cosmics scenario is changed from representing conditions at Run 2 startup (which is no longer of interest) to realistic 2016 cosmics conditions (which remains of interest).

The GT diffs are as follows:

2016 realistic pre-VFP era
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_mcRun2_asymptotic_preVFP_v8/106X_mcRun2_asymptotic_preVFP_v9

2016 realistic post-VFP era
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_mcRun2_asymptotic_v13/106X_mcRun2_asymptotic_v15

2016 cosmics (realistic conditions)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_mcRun2cosmics_startup_deco_v7/106X_mcRun2cosmics_asymptotic_deco_v2

Offline data
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_v31/106X_dataRun2_v32

Offline data relval
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_dataRun2_relval_v29/106X_dataRun2_relval_v30

2018 realistic
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018_realistic_v13/106X_upgrade2018_realistic_v15

2018 realistic (HEM15/16 failure)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018_realistic_HEfail_v14/106X_upgrade2018_realistic_HEfail_v15

2018 cosmics (tracker deco mode)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018cosmics_realistic_deco_v11/106X_upgrade2018cosmics_realistic_deco_v12

2018 cosmics (tracker peak mode)
https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/106X_upgrade2018cosmics_realistic_peak_v11/106X_upgrade2018cosmics_realistic_peak_v12

The 106X_mcRun2_asymptotic_v15 GT in addition removes several obsolete records.

PR validation:

These tags have already been validated and are in use in the 10_2_X series.

In addition, a technical test was performed: runTheMatrix.py -l limited,1325.516,7.22,136.8642,11024.2,7.4 --ibeos

if this PR is a backport please specify the original PR and why you need to backport that PR:

This PR is backport of PRs #32224 and #31846.

christopheralanwest · 2020-11-23T19:44:31Z

backport #32224

cmsbuild · 2020-11-23T19:44:45Z

A new Pull Request was created by @christopheralanwest for CMSSW_10_6_X.

It involves the following packages:

Configuration/AlCa

@yuanchao, @christopheralanwest, @tocheng, @cmsbuild, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from Update QGL tag for 2016 and 2018 MC and Run 2 data #32224

christopheralanwest · 2020-11-23T19:44:47Z

test parameters

workflow = 1325.516,7.22,136.8642,11024.2,7.4

christopheralanwest · 2020-11-23T19:50:59Z

please test

cmsbuild · 2020-11-23T19:51:17Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 1325.516,7.22,136.8642,11024.2,7.4
CMSSW_10_6_X_2020-11-23-1100/slc7_amd64_gcc700: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10962/console Started: 2020/11/23 20:53

cmsbuild · 2020-11-23T21:41:13Z

+1
Tested at: 34745b4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-275585/10962/summary.html
CMSSW: CMSSW_10_6_X_2020-11-23-1100
SCRAM_ARCH: slc7_amd64_gcc700

cmsbuild · 2020-11-23T21:41:15Z

Comparison job queued.

cmsbuild · 2020-11-23T23:28:27Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-275585/10962/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-275585/11024.2_TTbar_13UP18HEfailINPUT+TTbar_13UP18HEfailINPUT+DigiFullHEfail+RecoFullHEfail+HARVESTFullHEfail+NanoFullHEfail
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-275585/136.8642_RunJetHT2018BHEfail+RunJetHT2018BHEfail+HLTDR2_2018+RECODR2_2018reHLT_skimJetHT_Prompt_HEfail+HARVEST2018_HEfail
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-275585/7.22_Cosmics_UP16+Cosmics_UP16+DIGICOS_UP16+RECOCOS_UP16+ALCACOS_UP16+HARVESTCOS_UP16

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
Reco comparison had 2 failed jobs
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 3214712
DQMHistoTests: Total failures: 43
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3214335
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 140 log files, 26 edm output root files, 34 DQM output files

cmsbuild · 2020-11-24T00:02:07Z

Pull request #32247 was updated. @yuanchao, @christopheralanwest, @tocheng, @cmsbuild, @tlampen, @pohsun can you please check and sign again.

christopheralanwest · 2020-11-24T00:04:22Z

please test

cmsbuild · 2020-11-24T00:04:36Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 1325.516,7.22,136.8642,11024.2,7.4
CMSSW_10_6_X_2020-11-23-2300/slc7_amd64_gcc700: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10968/console Started: 2020/11/24 04:10

cmsbuild · 2020-11-24T04:57:50Z

+1
Tested at: 25b9e08
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-275585/10968/summary.html
CMSSW: CMSSW_10_6_X_2020-11-23-2300
SCRAM_ARCH: slc7_amd64_gcc700

cmsbuild · 2020-11-24T04:57:53Z

Comparison job queued.

cmsbuild · 2020-11-24T08:50:34Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-275585/10968/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-275585/11024.2_TTbar_13UP18HEfailINPUT+TTbar_13UP18HEfailINPUT+DigiFullHEfail+RecoFullHEfail+HARVESTFullHEfail+NanoFullHEfail
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-275585/136.8642_RunJetHT2018BHEfail+RunJetHT2018BHEfail+HLTDR2_2018+RECODR2_2018reHLT_skimJetHT_Prompt_HEfail+HARVEST2018_HEfail
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-275585/7.22_Cosmics_UP16+Cosmics_UP16+DIGICOS_UP16+RECOCOS_UP16+ALCACOS_UP16+HARVESTCOS_UP16

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
Reco comparison had 2 failed jobs
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 3214712
DQMHistoTests: Total failures: 43
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3214335
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 140 log files, 26 edm output root files, 34 DQM output files

christopheralanwest · 2020-11-24T14:43:40Z

+1

changes seen in PR comparison tests only seen in QG likelihood distributions and in expected workflows (offline data, 2016 MC and 2018 MC)
consistent backport of PRs Update QGL tag for 2016 and 2018 MC and Run 2 data #32224 and Introduce scenario for 2016 cosmics with realistic conditions #31846

cmsbuild · 2020-11-24T14:44:04Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-11-24T18:30:50Z

+1

Update QGL tag for 2016 and 2018 MC and Run 2 data

34745b4

cmsbuild added this to the CMSSW_10_6_X milestone Nov 23, 2020

cmsbuild added alca-pending backport-ok comparison-pending orp-pending pending-signatures labels Nov 23, 2020

cmsbuild added the tests-pending label Nov 23, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 23, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 23, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 23, 2020

Introduce scenario for 2016 cosmics with realistic conditions

25b9e08

christopheralanwest changed the title ~~Update QGL tag for 2016 and 2018 MC and Run 2 data~~ Update QGL tag for 2016 and 2018 MC and Run 2 data + Introduce scenario for 2016 cosmics with realistic conditions [10_6_X] Nov 24, 2020

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Nov 24, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 24, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 24, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 24, 2020

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Nov 24, 2020

cmsbuild added orp-approved and removed orp-pending labels Nov 24, 2020

cmsbuild merged commit 86f5795 into cms-sw:CMSSW_10_6_X Nov 24, 2020

mariadalfonso mentioned this pull request Nov 29, 2020

Nan with quark-gluon in NanoAOD cms-nanoAOD/cmssw#556

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update QGL tag for 2016 and 2018 MC and Run 2 data + Introduce scenario for 2016 cosmics with realistic conditions [10_6_X] #32247

Update QGL tag for 2016 and 2018 MC and Run 2 data + Introduce scenario for 2016 cosmics with realistic conditions [10_6_X] #32247

christopheralanwest commented Nov 23, 2020 •

edited

christopheralanwest commented Nov 23, 2020

cmsbuild commented Nov 23, 2020 •

edited

christopheralanwest commented Nov 23, 2020

christopheralanwest commented Nov 23, 2020

cmsbuild commented Nov 23, 2020 •

edited

cmsbuild commented Nov 23, 2020

cmsbuild commented Nov 23, 2020

cmsbuild commented Nov 23, 2020

cmsbuild commented Nov 24, 2020

christopheralanwest commented Nov 24, 2020

cmsbuild commented Nov 24, 2020 •

edited

cmsbuild commented Nov 24, 2020

cmsbuild commented Nov 24, 2020

cmsbuild commented Nov 24, 2020

christopheralanwest commented Nov 24, 2020

cmsbuild commented Nov 24, 2020

silviodonato commented Nov 24, 2020

Update QGL tag for 2016 and 2018 MC and Run 2 data + Introduce scenario for 2016 cosmics with realistic conditions [10_6_X] #32247

Update QGL tag for 2016 and 2018 MC and Run 2 data + Introduce scenario for 2016 cosmics with realistic conditions [10_6_X] #32247

Conversation

christopheralanwest commented Nov 23, 2020 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

christopheralanwest commented Nov 23, 2020

cmsbuild commented Nov 23, 2020 • edited

christopheralanwest commented Nov 23, 2020

christopheralanwest commented Nov 23, 2020

cmsbuild commented Nov 23, 2020 • edited

cmsbuild commented Nov 23, 2020

cmsbuild commented Nov 23, 2020

cmsbuild commented Nov 23, 2020

cmsbuild commented Nov 24, 2020

christopheralanwest commented Nov 24, 2020

cmsbuild commented Nov 24, 2020 • edited

cmsbuild commented Nov 24, 2020

cmsbuild commented Nov 24, 2020

cmsbuild commented Nov 24, 2020

christopheralanwest commented Nov 24, 2020

cmsbuild commented Nov 24, 2020

silviodonato commented Nov 24, 2020

christopheralanwest commented Nov 23, 2020 •

edited

cmsbuild commented Nov 23, 2020 •

edited

cmsbuild commented Nov 23, 2020 •

edited

cmsbuild commented Nov 24, 2020 •

edited