Remove `DQMReferenceHistogramRootFileRcd` from all GTs #35533

tvami · 2021-10-04T22:13:54Z

PR description:

DQM confirmed that the tags corresponding to the DQMReferenceHistogramRootFileRcd record are not used anymore.
Removal of the consumption of the tag has happened in #35491

Now I've removed this tag from the queues, and created new GTs without them as discussed in the HN [1]. Here are the diffs wrt to the previous tags [2]

[1]
https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4483.html

if this PR is a backport please specify the original PR and why you need to backport that PR:

Not a backport and no backport is needed

resolves cms-AlCaDB/AlCaTools#40

tvami · 2021-10-04T22:21:33Z

test parameters:

workflows = 136.897,140.0,200.0,202.0,203.0,204.0,205.0,4001.0,4006.0,4007.0,4008.0,11634.0,12434.0,12834.0,1325.516,50200.0,7.22,145.0,281.0,10424.0,7.21,11224.0,250200.182,11024.2,7.4,12034.0,7.23,159.0,12834.0,7.24

cmsbuild · 2021-10-04T22:22:27Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35533/25746

This PR adds an extra 12KB to repository

cmsbuild · 2021-10-04T22:22:49Z

A new Pull Request was created by @tvami (Tamas Vami) for master.

It involves the following packages:

Configuration/AlCa (alca)

@yuanchao, @malbouis, @cmsbuild, @tvami, @francescobrivio can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2021-10-04T22:23:05Z

@cmsbuild , please test

cmsbuild · 2021-10-05T02:08:08Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2512d4/19393/summary.html
COMMIT: 5614855
CMSSW: CMSSW_12_1_X_2021-10-04-1300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35533/19393/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/10424.0_TTbar_13+2017Design+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/11024.2_TTbar_13UP18HEfailINPUT+TTbar_13UP18HEfailINPUT+DigiFullHEfail+RecoFullHEfail+HARVESTFullHEfail+NanoFullHEfail
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/11224.0_TTbar_13+2018Design+TTbar_13TeV_TuneCUETP8M1_GenSim+Digi+RecoFakeHLT+HARVESTFakeHLT
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/12034.0_TTbar_14TeV+2021Design+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/136.897_RunCosmics2021CRUZET+RunCosmics2021CRUZET+RECOCOSDRUN3+ALCACOSDRUN3+HARVESTDCRUN3
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/140.0_HydjetQ_B12_5020GeV_2011+HydjetQ_B12_5020GeV_2011+DIGIHI2011+RECOHI2011+HARVESTHI2011
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/145.0_HydjetQ_B12_5020GeV_2015+HydjetQ_B12_5020GeV_2015+DIGIHI2015+RECOHI2015+HARVESTHI2015
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/203.0_H130GGgluonfusion+H130GGgluonfusion+DIGIPU1+RECOPU1+HARVEST
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/204.0_QQH1352T+QQH1352T+DIGIPU1+RECOPU1+HARVEST
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/205.0_ZTT+ZTT+DIGIPU1+RECOPU1+HARVEST
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/250200.182_ZEE_13UP18_RD+ZEE_13UP18_RD+DIGIPRMXUP18_PU25_RD+RECOPRMXUP18_PU25_L1TEgDQM_RD+HARVESTUP18_PU25_L1TEgDQM_RD
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/4001.0_TTbar+TTbar+DIGIPU1+RECOPUDBG
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/4006.0_SingleElectronFlatPt1To100+SingleElectronFlatPt1To100+DIGIPU1+RECOPU1
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/4007.0_QCD_Pt_30_80_BCtoE_8TeV+QCD_Pt_30_80_BCtoE_8TeV+DIGIPU1+RECOPUDBG
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/4008.0_QCD_Pt_80_170_BCtoE_8TeV+QCD_Pt_80_170_BCtoE_8TeV+DIGIPU1+RECOPUDBG
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/50200.0_ZEE_13+ZEE_13+DIGIUP15_PU50+RECOUP15_PU50_L1TEgDQM+HARVESTUP15_PU50_L1TEgDQM
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/7.21_Cosmics_UP17+Cosmics_UP17+DIGICOS_UP17+RECOCOS_UP17+ALCACOS_UP17+HARVESTCOS_UP17
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/7.22_Cosmics_UP16+Cosmics_UP16+DIGICOS_UP16+RECOCOS_UP16+ALCACOS_UP16+HARVESTCOS_UP16
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/7.23_Cosmics_UP21+Cosmics_UP21+DIGICOS_UP21+RECOCOS_UP21+ALCACOS_UP21+HARVESTCOS_UP21
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-2512d4/7.24_Cosmics_UP21_0T+Cosmics_UP21_0T+DIGICOS_UP21_0T+RECOCOS_UP21_0T+ALCACOS_UP21_0T+HARVESTCOS_UP21_0T

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 40
DQMHistoTests: Total histograms compared: 3219394
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3219360
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 39 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 169 log files, 37 edm output root files, 40 DQM output files
TriggerResults: no differences found

tvami · 2021-10-05T02:14:44Z

Differences are only in the MsgLogger, anyway all clean results, as expected!

francescobrivio · 2021-10-05T07:45:52Z

@tvami

120X_mc2017_realistic_forppRef5TeV_v2 was not changed, please remove it from the list of the diffs
There are two cases (121X_dataRun3_HLT_v* and 121X_dataRun2_HLT_relval_v*) where you created new GTs but they are identical to the old ones because the DQMReferenceHistogramRootFileRcd record was not present in the orginal GT. Is this due to an automatic script? Or is it done on purpose?

tvami · 2021-10-05T12:06:52Z

120X_mc2017_realistic_forppRef5TeV_v2 was not changed, please remove it from the list of the diffs

Done. Also I propose to deal with this for good with an autoCond modifier. I can deal with this in a follow-up PR

There are two cases (121X_dataRun3_HLT_v* and 121X_dataRun2_HLT_relval_v*) where you created new GTs but they are identical to the old ones because the DQMReferenceHistogramRootFileRcd record was not present in the orginal GT. Is this due to an automatic script? Or is it done on purpose?

Right, this is due to my semi-automatized process, and indeed I noticed it as well, but only after the GT was created, so I thought it makes sense to include the new GT. It doesnt make a difference but it's good for bookkeeping.

francescobrivio · 2021-10-05T12:31:47Z

+alca

see comment Remove DQMReferenceHistogramRootFileRcd from all GTs #35533 (comment)

cmsbuild · 2021-10-05T12:32:08Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-10-05T13:19:52Z

+1

GT update/cleaning

Remove DQMReferenceHistogramRootFileRcd from GTs

5614855

cmsbuild added this to the CMSSW_12_1_X milestone Oct 4, 2021

cmsbuild added alca-pending code-checks-pending orp-pending pending-signatures tests-pending labels Oct 4, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 4, 2021

cmsbuild added tests-started and removed tests-pending labels Oct 4, 2021

cmsbuild added tests-approved and removed tests-started labels Oct 5, 2021

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Oct 5, 2021

cmsbuild added orp-approved and removed orp-pending labels Oct 5, 2021

cmsbuild merged commit 1c158e7 into cms-sw:master Oct 5, 2021

tvami mentioned this pull request Oct 5, 2021

Remove DQMReferenceHistogramRootFileRcd from GTs cms-AlCaDB/AlCaTools#40

Closed

This was referenced Oct 5, 2021

Remove use of legacy module classes from framework tests #35526

Merged

Replace tags that have [cms_orcon_prod/...] in their name #35546

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `DQMReferenceHistogramRootFileRcd` from all GTs #35533

Remove `DQMReferenceHistogramRootFileRcd` from all GTs #35533

tvami commented Oct 4, 2021 •

edited

tvami commented Oct 4, 2021

cmsbuild commented Oct 4, 2021

cmsbuild commented Oct 4, 2021

tvami commented Oct 4, 2021

cmsbuild commented Oct 5, 2021

tvami commented Oct 5, 2021

francescobrivio commented Oct 5, 2021

tvami commented Oct 5, 2021

francescobrivio commented Oct 5, 2021

cmsbuild commented Oct 5, 2021

perrotta commented Oct 5, 2021

Remove DQMReferenceHistogramRootFileRcd from all GTs #35533

Remove DQMReferenceHistogramRootFileRcd from all GTs #35533

Conversation

tvami commented Oct 4, 2021 • edited

PR description:

if this PR is a backport please specify the original PR and why you need to backport that PR:

tvami commented Oct 4, 2021

cmsbuild commented Oct 4, 2021

cmsbuild commented Oct 4, 2021

tvami commented Oct 4, 2021

cmsbuild commented Oct 5, 2021

Comparison Summary

tvami commented Oct 5, 2021

francescobrivio commented Oct 5, 2021

tvami commented Oct 5, 2021

francescobrivio commented Oct 5, 2021

cmsbuild commented Oct 5, 2021

perrotta commented Oct 5, 2021

Remove `DQMReferenceHistogramRootFileRcd` from all GTs #35533

Remove `DQMReferenceHistogramRootFileRcd` from all GTs #35533

tvami commented Oct 4, 2021 •

edited