Convert workflows that use pp prompt-like GTs to use offline GTs #28825

christopheralanwest · 2020-01-29T23:14:33Z

PR description:

PR #28762 introduced changes that broke workflow 136.891, which PR #28804 attempted to fix. While PR #28804 did fix workflow 136.891, it broke workflows 136.8642,136.8643,136.886,136.8861,136.8862,136.888,136.889,136.89,136.893,136.895,136.896 and 137.8.

These workflows currently use "prompt-like" GTs, which are similar to the prompt GTs that were used for prompt reconstruction of the Run2018 datasets. Now that an offline GT is available for Run 2018D, a prompt-like GT is no longer needed. The errors in the failing workflows are resolved after switching to the offline GTs. The key auto:run2_data_promptlike_hi is kept because there is no offline GT for the Run II HI data.

Attn: @mmusich @silviodonato

PR validation:

The PR was tested with addOnTests.py -j 8 and runTheMatrix.py -l 136.8642,136.8643,136.886,136.8861,136.8862,136.888,136.889,136.89,136.893,136.895,136.896,137.8,136.891 --ibeos.

if this PR is a backport please specify the original PR and why you need to backport that PR:

This PR is not a backport.

cmsbuild · 2020-01-29T23:15:03Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-01-29T23:20:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28825/13544

This PR adds an extra 56KB to repository

cmsbuild · 2020-01-29T23:20:44Z

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

Alignment/OfflineValidation
CalibTracker/SiPixelQuality
Configuration/AlCa
Configuration/HLT
Configuration/PyReleaseValidation

@pgunnell, @chayanit, @zhenhu, @fwyzard, @christopheralanwest, @tocheng, @cmsbuild, @franzoni, @kpedro88, @Martin-Grunewald, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@adewit, @makortel, @tocheng, @Martin-Grunewald, @dkotlins, @tlampen, @mschrode, @mmusich this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

christopheralanwest · 2020-01-29T23:22:00Z

please test workflow 136.8642,136.8643,136.886,136.8861,136.8862,136.888,136.889,136.89,136.893,136.895,136.896,137.8,136.891

cmsbuild · 2020-01-29T23:22:24Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 136.8642,136.89,136.8862,136.8643,136.8861,136.891,136.893,136.886,136.896,136.889,136.888,137.8,136.895
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4421/console Started: 2020/01/30 05:05

cmsbuild · 2020-01-30T06:12:49Z

+1
Tested at: 8e01667
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22d6dc/4421/summary.html
CMSSW: CMSSW_11_1_X_2020-01-29-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-01-30T06:12:52Z

Comparison job queued.

cmsbuild · 2020-01-30T07:55:05Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22d6dc/4421/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.8642_RunJetHT2018BHEfail+RunJetHT2018BHEfail+HLTDR2_2018+RECODR2_2018reHLT_skimJetHT_Prompt_HEfail+HARVEST2018_HEfail
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.8643_RunJetHT2018BBadHcalMitig+RunJetHT2018BBadHcalMitig+HLTDR2_2018_BadHcalMitig+RECODR2_2018reHLT_skimJetHT_Prompt_BadHcalMitig+HARVEST2018_BadHcalMitig
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.8861_RunEGamma2018Dml1+RunEGamma2018Dml1+HLTDR2_2018ml+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQMml+HARVEST2018_L1TEgDQM_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.8862_RunEGamma2018Dml2+RunEGamma2018Dml2+HLTDR2_2018ml+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQMml+HARVEST2018_L1TEgDQM_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.889_RunMET2018D+RunMET2018D+HLTDR2_2018+RECODR2_2018reHLT_skimMET_Prompt+HARVEST2018_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.891_RunSingleMu2018D+RunSingleMu2018D+HLTDR2_2018+RECODR2_2018reHLT_skimSingleMu_Prompt_Lumi+HARVEST2018_L1TMuDQM_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.893_RunMuOnia2018D+RunMuOnia2018D+HLTDR2_2018+RECODR2_2018reHLT_skimMuOnia_Prompt+HARVEST2018_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.895_RunDisplacedJet2018D+RunDisplacedJet2018D+HLTDR2_2018+RECODR2_2018reHLT_skimDisplacedJet_Prompt+HARVEST2018_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.896_RunCharmonium2018D+RunCharmonium2018D+HLTDR2_2018+RECODR2_2018reHLT_skimCharmonium_Prompt+HARVEST2018_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/136.89_RunMuonEG2018D+RunMuonEG2018D+HLTDR2_2018+RECODR2_2018reHLT_skimMuonEG_Prompt+HARVEST2018_Prompt
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-22d6dc/137.8_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+RunEGamma2018D+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Prompt_L1TEgDQM+HARVEST2018_L1TEgDQM_MULTIRUN

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2697068
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2696721
DQMHistoTests: Total skipped: 346
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

silviodonato · 2020-01-30T11:25:32Z

I know this is a very recent PR, but it solves several crashing workflow. I would like to merge it as soon as possible.
alca: @christopheralanwest @franzoni @tlampen @pohsun @tocheng
hlt: @Martin-Grunewald @fwyzard
pdmv: @chayanit @pgunnell @zhenhu
upgrade: @kpedro88

chayanit · 2020-01-30T11:32:48Z

+1

christopheralanwest · 2020-01-30T13:21:17Z

+1

kpedro88 · 2020-01-30T13:50:00Z

+upgrade

cmsbuild · 2020-01-30T13:50:26Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-01-30T14:37:24Z

please test

cmsbuild · 2020-01-30T14:37:52Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4433/console Started: 2020/01/30 15:40

cmsbuild · 2020-01-30T16:03:44Z

+1
Tested at: 8e01667
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22d6dc/4433/summary.html
CMSSW: CMSSW_11_1_X_2020-01-30-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-01-30T16:03:47Z

Comparison job queued.

cmsbuild · 2020-01-30T17:33:11Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-22d6dc/4433/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2697068
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2696721
DQMHistoTests: Total skipped: 346
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

silviodonato · 2020-01-30T17:33:55Z

+1
thanks for the fix and the quick review

Convert workflows that use pp prompt-like GTs to use offline GTs

8e01667

cmsbuild added this to the CMSSW_11_1_X milestone Jan 29, 2020

cmsbuild added alca-pending code-checks-pending comparison-pending hlt-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jan 29, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 29, 2020

cmsbuild added tests-started and removed tests-pending labels Jan 29, 2020

cmsbuild added tests-approved and removed tests-started labels Jan 30, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jan 30, 2020

cmsbuild removed the pdmv-pending label Jan 30, 2020

cmsbuild added the hlt-approved label Jan 30, 2020

cmsbuild added alca-approved and removed alca-pending labels Jan 30, 2020

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Jan 30, 2020

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Jan 30, 2020

cmsbuild added tests-started and removed tests-pending labels Jan 30, 2020

cmsbuild added tests-approved and removed tests-started labels Jan 30, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jan 30, 2020

cmsbuild added orp-approved and removed orp-pending labels Jan 30, 2020

cmsbuild merged commit 7f1d6e0 into cms-sw:master Jan 30, 2020

christopheralanwest mentioned this pull request Feb 18, 2020

Correct ECAL timing calibration for Run2018D #28981

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert workflows that use pp prompt-like GTs to use offline GTs #28825

Convert workflows that use pp prompt-like GTs to use offline GTs #28825

christopheralanwest commented Jan 29, 2020

cmsbuild commented Jan 29, 2020

cmsbuild commented Jan 29, 2020

cmsbuild commented Jan 29, 2020

christopheralanwest commented Jan 29, 2020

cmsbuild commented Jan 29, 2020 •

edited

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

silviodonato commented Jan 30, 2020

chayanit commented Jan 30, 2020

christopheralanwest commented Jan 30, 2020

kpedro88 commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

silviodonato commented Jan 30, 2020

cmsbuild commented Jan 30, 2020 •

edited

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

silviodonato commented Jan 30, 2020

Convert workflows that use pp prompt-like GTs to use offline GTs #28825

Convert workflows that use pp prompt-like GTs to use offline GTs #28825

Conversation

christopheralanwest commented Jan 29, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jan 29, 2020

cmsbuild commented Jan 29, 2020

cmsbuild commented Jan 29, 2020

christopheralanwest commented Jan 29, 2020

cmsbuild commented Jan 29, 2020 • edited

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

silviodonato commented Jan 30, 2020

chayanit commented Jan 30, 2020

christopheralanwest commented Jan 30, 2020

kpedro88 commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

silviodonato commented Jan 30, 2020

cmsbuild commented Jan 30, 2020 • edited

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

cmsbuild commented Jan 30, 2020

silviodonato commented Jan 30, 2020

cmsbuild commented Jan 29, 2020 •

edited

cmsbuild commented Jan 30, 2020 •

edited