fix stray Global Tags post GeometricDetExtra Migration #31871

mmusich · 2020-10-20T10:16:14Z

PR description:

After PR #31729 has been integrated, there have been falilures in 20 unit tests in DQM/Integration and exceptions in wf 1001.2, 1030.0, 1040.0, 1040.1 (as reported in issue: #31869)

This PR fixes that state of affairs, by doing two things:

fixes some stray GTs left around in relval tests and migrate them with the autoCond key: auto:run3_data_express
updates the DQM/Integration code to use the new Express Reco GT: 111X_dataRun3_Express_v4

PR validation:

I've run successfully :

unit tests: scramv1 b runtests
relval tests: runTheMatrix.py -l 1001.2,1030.0,1040.0,1040.1 -t 4 -j 8

if this PR is a backport please specify the original PR and why you need to backport that PR:

This PR is not a backport, no backport is needed.

cmsbuild · 2020-10-20T10:16:42Z

The code-checks are being triggered in jenkins.

DQM/Integration/python/config/FrontierCondition_GT_cfi.py

cmsbuild · 2020-10-20T10:24:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31871/19241

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Nano gen dqm #31591, Bad PF candidate filter for 2018 PbPb data #31668, Patatrack integration - Pixel workflows (12/N) #31854, Adding ability to output PF Candidate information in JME NTuples #31795

cmsbuild · 2020-10-20T10:25:05Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

Configuration/PyReleaseValidation
DQM/Integration

@andrius-k, @jordan-martins, @chayanit, @wajidalikhan, @ErnestaP, @kpedro88, @cmsbuild, @kmaeshima, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@batinkov, @battibass, @makortel, @Martin-Grunewald, @threus, @fabiocos, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2020-10-20T10:40:06Z

The code-checks are being triggered in jenkins.

silviodonato · 2020-10-20T10:42:22Z

urgent

cmsbuild · 2020-10-20T10:45:44Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31871/19244

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Nano gen dqm #31591, Bad PF candidate filter for 2018 PbPb data #31668, Patatrack integration - Pixel workflows (12/N) #31854, Adding ability to output PF Candidate information in JME NTuples #31795

cmsbuild · 2020-10-20T10:46:06Z

Pull request #31871 was updated. @andrius-k, @jordan-martins, @chayanit, @wajidalikhan, @ErnestaP, @kpedro88, @cmsbuild, @kmaeshima, @jfernan2, @fioriNTU can you please check and sign again.

chayanit · 2020-10-20T10:52:08Z

please test

cmsbuild · 2020-10-20T10:52:32Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-10-19-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10125/console Started: 2020/10/20 14:04

mmusich · 2020-10-20T14:30:37Z

DQM/Integration/python/clients/beam_dqm_sourceclient-live_cfg.py

@@ -73,7 +73,7 @@
    from Configuration.AlCa.GlobalTag import GlobalTag as gtCustomise
    process.GlobalTag = gtCustomise(process.GlobalTag, 'auto:run2_data', '')
    # you may need to set manually the GT in the line below
-    process.GlobalTag.globaltag = '100X_upgrade2018_realistic_v10'
+    #process.GlobalTag.globaltag = '100X_upgrade2018_realistic_v10'


actually wondering what was being tested here exactly, by running a "data"-like workflow with a MC Global Tag?!?

cmsbuild · 2020-10-20T14:34:27Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79e604/10125/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2544100
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2544071
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

cmsbuild · 2020-10-20T15:15:42Z

+1
Tested at: 917004d
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79e604/10133/summary.html
CMSSW: CMSSW_11_2_X_2020-10-19-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-20T15:15:45Z

Comparison job queued.

cmsbuild · 2020-10-20T16:30:03Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79e604/10133/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-79e604/1001.2_RunZeroBias2017F+RunZeroBias2017F+TIER0EXPRUN2+ALCAEXPRUN2+ALCAHARVDSIPIXELCAL
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-79e604/1030.0_RunHLTPhy2017B+RunHLTPhy2017B+TIER0EXPHPBS+ALCASPLITHPBS+ALCAHARVDHPBS+ALCAHARVDHPBSLOWPU
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-79e604/1040.0_RunZeroBias2017F+RunZeroBias2017F+TIER0RAWSIPIXELCAL+ALCASPLITSIPIXELCAL+ALCAHARVDSIPIXELCAL
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-79e604/1040.1_RunExpressPhy2017F+RunExpressPhy2017F+TIER0EXPSIPIXELCAL+ALCASPLITSIPIXELCAL+ALCAHARVDSIPIXELCAL

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2544100
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2544077
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

jfernan2 · 2020-10-20T16:37:49Z

+1

silviodonato · 2020-10-20T17:25:19Z

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 if there are no objections, I will merge this PR before the night IB

kpedro88 · 2020-10-20T17:35:55Z

+upgrade

silviodonato · 2020-10-20T19:40:55Z

merge

mmusich mentioned this pull request Oct 20, 2020

Exception while attempting to load plugin PGeometricDetExtraRcd@NewProxy #31869

Closed

cmsbuild added this to the CMSSW_11_2_X milestone Oct 20, 2020

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Oct 20, 2020

mmusich commented Oct 20, 2020

View reviewed changes

DQM/Integration/python/config/FrontierCondition_GT_cfi.py Outdated Show resolved Hide resolved

mmusich mentioned this pull request Oct 20, 2020

Patatrack integration - Pixel local reconstruction (9/N) #31721

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 20, 2020

fix stray Global Tags post GeometricDetExtra Migration

7f46207

mmusich force-pushed the fixStrayGTsPostGeometricDetExtraMigration branch from 9bb0a5f to 7f46207 Compare October 20, 2020 10:39

cmsbuild added code-checks-pending and removed code-checks-approved labels Oct 20, 2020

cmsbuild added the urgent label Oct 20, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 20, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 20, 2020

mmusich commented Oct 20, 2020

View reviewed changes

cmsbuild added comparison-available and removed comparison-pending labels Oct 20, 2020

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Oct 20, 2020

cmsbuild mentioned this pull request Oct 20, 2020

Bad PF candidate filter for 2018 PbPb data #31668

Merged

cmsbuild added comparison-available and removed comparison-pending labels Oct 20, 2020

cmsbuild added dqm-approved and removed dqm-pending labels Oct 20, 2020

cmsbuild added upgrade-approved and removed upgrade-pending labels Oct 20, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 20, 2020

cmsbuild merged commit 3b0ba7a into cms-sw:master Oct 20, 2020

mmusich deleted the fixStrayGTsPostGeometricDetExtraMigration branch October 20, 2020 19:43

cmsbuild mentioned this pull request Oct 20, 2020

Remove duplicate dependency from L1Trigger/CSCTriggerPrimitives BuildFile.xml #31877

Merged

silviodonato mentioned this pull request Nov 26, 2020

DQMOnlineClient-beam_dqm_sourceclient error #31896

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix stray Global Tags post GeometricDetExtra Migration #31871

fix stray Global Tags post GeometricDetExtra Migration #31871

mmusich commented Oct 20, 2020 •

edited

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

silviodonato commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

chayanit commented Oct 20, 2020

cmsbuild commented Oct 20, 2020 •

edited

mmusich Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

jfernan2 commented Oct 20, 2020

silviodonato commented Oct 20, 2020

kpedro88 commented Oct 20, 2020

silviodonato commented Oct 20, 2020

fix stray Global Tags post GeometricDetExtra Migration #31871

fix stray Global Tags post GeometricDetExtra Migration #31871

Conversation

mmusich commented Oct 20, 2020 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

silviodonato commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

chayanit commented Oct 20, 2020

cmsbuild commented Oct 20, 2020 • edited

mmusich Oct 20, 2020

Choose a reason for hiding this comment

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

jfernan2 commented Oct 20, 2020

silviodonato commented Oct 20, 2020

kpedro88 commented Oct 20, 2020

silviodonato commented Oct 20, 2020

mmusich commented Oct 20, 2020 •

edited

cmsbuild commented Oct 20, 2020 •

edited