Migrate BMixingModule and EcalShapeBase-derived classes to esConsumes #34902

makortel · 2021-08-17T01:02:11Z

PR description:

Part of #31061. This PR should cover the last uses of the deprecated EventSetupRecord::get() in MixingModule and PreMixingModule. In addition I removed all bare new calls from EcalMixingModuleValidation.

In EcalDigiProducer and EcalDigiProducer_Ph2 (used by MixingModule and PreMixingModule) the changes were in APDShape, EBShape, and EBShape classes, that are also used several places outside of the MixingModules.

Within the BMixingModule infrastructure there are still some calls in DataMixingModule that would either need to be migrated or removed (in a timescale of 9 months or so). I'm not planning to touch those.

Resolves cms-sw/framework-team#221

PR validation:

Code compiles.

cmsbuild · 2021-08-17T01:07:19Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34902/24709

This PR adds an extra 108KB to repository

cmsbuild · 2021-08-17T01:07:45Z

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

CalibCalorimetry/EcalSRTools (alca)
CalibCalorimetry/EcalTPGTools (l1, alca)
Mixing/Base (simulation)
RecoLocalCalo/EcalRecProducers (reconstruction)
RecoTBCalo/EcalTBRecProducers (reconstruction)
SimCalorimetry/EcalSimAlgos (simulation)
SimCalorimetry/EcalSimProducers (simulation)
SimGeneral/MixingModule (simulation)
Validation/EcalDigis (dqm)

@perrotta, @malbouis, @civanch, @yuanchao, @kmaeshima, @tlampen, @andrius-k, @mdhildreth, @rvenditti, @cmsbuild, @rekovic, @jfernan2, @ahmad3213, @slava77, @jpata, @ErnestaP, @pohsun, @francescobrivio, @cecilecaillol, @tvami can you please review it and eventually sign? Thanks.
@rchatter, @rovere, @argiro, @apsallid, @tocheng, @thomreis, @simonepigazzini, @mmusich, @fabiocos, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

perrotta · 2021-08-17T05:57:04Z

please test

cmsbuild · 2021-08-17T07:44:23Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-768ae1/17815/summary.html
COMMIT: 2c94140
CMSSW: CMSSW_12_1_X_2021-08-16-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34902/17815/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 7 EventSetupRecord::get warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-768ae1/17815/llvm-analysis/esrget-sa.txt for details.
CMS StaticAnalyzer warnings: There are 1 inherits from legacy modules warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-768ae1/17815/llvm-analysis/legacy-mod-sa.txt for details.

RelVals

8.0

----- Begin Fatal Exception 17-Aug-2021 08:59:21 CEST-----------------------
An exception of category 'ESGetTokenWrongTransition' occurred while
   [0] Processing  stream begin Run run: 1 stream: 0
   [1] Calling method for module MixingModule/'mix'
Exception Message:
The transition ID stored in the ESGetToken does not match the
transition where the token is being used. The associated record
type is: EcalSimPulseShapeRcd
For producers, filters and analyzers this transition ID is
set as a template parameter to the call to the esConsumes
function that creates the token. Event is the default transition.
Other possibilities are BeginRun, EndRun, BeginLuminosityBlock,
or EndLuminosityBlock. You may need multiple tokens if you want to
get the same data in multiple transitions. The transition ID has a
different meaning in ESProducers. For ESProducers, the transition
ID identifies the function that produces the EventSetup data (often
there is one function named produce but there can be multiple
functions with different names). For ESProducers, the ESGetToken
must be used in the function associated with the ESConsumesCollector
returned by the setWhatProduced function.
----- End Fatal Exception -------------------------------------------------

5.1

----- Begin Fatal Exception 17-Aug-2021 08:59:49 CEST-----------------------
An exception of category 'ESGetTokenWrongTransition' occurred while
   [0] Processing  stream begin Run run: 1 stream: 0
   [1] Calling method for module MixingModule/'mix'
Exception Message:
The transition ID stored in the ESGetToken does not match the
transition where the token is being used. The associated record
type is: EcalSimPulseShapeRcd
For producers, filters and analyzers this transition ID is
set as a template parameter to the call to the esConsumes
function that creates the token. Event is the default transition.
Other possibilities are BeginRun, EndRun, BeginLuminosityBlock,
or EndLuminosityBlock. You may need multiple tokens if you want to
get the same data in multiple transitions. The transition ID has a
different meaning in ESProducers. For ESProducers, the transition
ID identifies the function that produces the EventSetup data (often
there is one function named produce but there can be multiple
functions with different names). For ESProducers, the ESGetToken
must be used in the function associated with the ESConsumesCollector
returned by the setWhatProduced function.
----- End Fatal Exception -------------------------------------------------

135.4

----- Begin Fatal Exception 17-Aug-2021 08:59:52 CEST-----------------------
An exception of category 'ESGetTokenWrongTransition' occurred while
   [0] Processing  stream begin Run run: 1 stream: 0
   [1] Calling method for module MixingModule/'mix'
Exception Message:
The transition ID stored in the ESGetToken does not match the
transition where the token is being used. The associated record
type is: EcalSimPulseShapeRcd
For producers, filters and analyzers this transition ID is
set as a template parameter to the call to the esConsumes
function that creates the token. Event is the default transition.
Other possibilities are BeginRun, EndRun, BeginLuminosityBlock,
or EndLuminosityBlock. You may need multiple tokens if you want to
get the same data in multiple transitions. The transition ID has a
different meaning in ESProducers. For ESProducers, the transition
ID identifies the function that produces the EventSetup data (often
there is one function named produce but there can be multiple
functions with different names). For ESProducers, the ESGetToken
must be used in the function associated with the ESConsumesCollector
returned by the setWhatProduced function.
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

6.06.0_SingleMuPt1+SingleMuPt1INPUT+DIGI+RECO+HARVEST/step2_SingleMuPt1+SingleMuPt1INPUT+DIGI+RECO+HARVEST.log
8.08.0_BeamHalo+BeamHaloINPUT+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step2_BeamHalo+BeamHaloINPUT+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log
9.09.0_Higgs200ChargedTaus+Higgs200ChargedTausINPUT+DIGI+RECO+HARVEST/step2_Higgs200ChargedTaus+Higgs200ChargedTausINPUT+DIGI+RECO+HARVEST.log

Expand to see more relval errors ...

AddOn Tests

fastsim

----- Begin Fatal Exception 17-Aug-2021 09:03:27 CEST-----------------------
An exception of category 'ESGetTokenWrongTransition' occurred while
   [0] Processing  stream begin Run run: 1 stream: 2
   [1] Calling method for module MixingModule/'mix'
Exception Message:
The transition ID stored in the ESGetToken does not match the
transition where the token is being used. The associated record
type is: EcalSimPulseShapeRcd
For producers, filters and analyzers this transition ID is
set as a template parameter to the call to the esConsumes
function that creates the token. Event is the default transition.
Other possibilities are BeginRun, EndRun, BeginLuminosityBlock,
or EndLuminosityBlock. You may need multiple tokens if you want to
get the same data in multiple transitions. The transition ID has a
different meaning in ESProducers. For ESProducers, the transition
ID identifies the function that produces the EventSetup data (often
there is one function named produce but there can be multiple
functions with different names). For ESProducers, the ESGetToken
must be used in the function associated with the ESConsumesCollector
returned by the setWhatProduced function.
----- End Fatal Exception -------------------------------------------------

fastsim1

----- Begin Fatal Exception 17-Aug-2021 09:03:26 CEST-----------------------
An exception of category 'ESGetTokenWrongTransition' occurred while
   [0] Processing  stream begin Run run: 1 stream: 3
   [1] Calling method for module MixingModule/'mix'
Exception Message:
The transition ID stored in the ESGetToken does not match the
transition where the token is being used. The associated record
type is: EcalSimPulseShapeRcd
For producers, filters and analyzers this transition ID is
set as a template parameter to the call to the esConsumes
function that creates the token. Event is the default transition.
Other possibilities are BeginRun, EndRun, BeginLuminosityBlock,
or EndLuminosityBlock. You may need multiple tokens if you want to
get the same data in multiple transitions. The transition ID has a
different meaning in ESProducers. For ESProducers, the transition
ID identifies the function that produces the EventSetup data (often
there is one function named produce but there can be multiple
functions with different names). For ESProducers, the ESGetToken
must be used in the function associated with the ESConsumesCollector
returned by the setWhatProduced function.
----- End Fatal Exception -------------------------------------------------

fastsim2

----- Begin Fatal Exception 17-Aug-2021 09:03:27 CEST-----------------------
An exception of category 'ESGetTokenWrongTransition' occurred while
   [0] Processing  stream begin Run run: 1 stream: 3
   [1] Calling method for module MixingModule/'mix'
Exception Message:
The transition ID stored in the ESGetToken does not match the
transition where the token is being used. The associated record
type is: EcalSimPulseShapeRcd
For producers, filters and analyzers this transition ID is
set as a template parameter to the call to the esConsumes
function that creates the token. Event is the default transition.
Other possibilities are BeginRun, EndRun, BeginLuminosityBlock,
or EndLuminosityBlock. You may need multiple tokens if you want to
get the same data in multiple transitions. The transition ID has a
different meaning in ESProducers. For ESProducers, the transition
ID identifies the function that produces the EventSetup data (often
there is one function named produce but there can be multiple
functions with different names). For ESProducers, the ESGetToken
must be used in the function associated with the ESConsumesCollector
returned by the setWhatProduced function.
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

cmsbuild · 2021-08-17T13:15:37Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34902/24715

This PR adds an extra 112KB to repository

cmsbuild · 2021-08-17T13:16:05Z

Pull request #34902 was updated. @perrotta, @malbouis, @civanch, @yuanchao, @kmaeshima, @tlampen, @andrius-k, @mdhildreth, @rvenditti, @cmsbuild, @rekovic, @jfernan2, @ahmad3213, @slava77, @jpata, @ErnestaP, @pohsun, @francescobrivio, @cecilecaillol, @tvami can you please check and sign again.

makortel · 2021-08-17T13:19:19Z

@cmsbuild, please test

makortel · 2021-08-26T20:45:13Z

The PR title and description look a bit as if the commits are not on the subject; some improvement to both may be useful

@makortel since this PR is still not merged, would you please do this?

How about now?

makortel · 2021-08-27T13:20:22Z

ping @cms-sw/l1-l2

makortel · 2021-08-30T13:16:00Z

ping @cms-sw/l1-l2

makortel · 2021-08-31T13:18:09Z

ping @cms-sw/l1-l2

tvami · 2021-08-31T13:25:31Z

Hi @perrotta @qliphy since this PR waits on the L1 signature for more than 10 days, would you consider merging the PR without it?

rekovic · 2021-08-31T15:01:17Z

+1

cmsbuild · 2021-08-31T15:01:40Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2021-08-31T15:50:05Z

please test
(to refresh them)

smuzaffar · 2021-09-01T08:56:51Z

please test

cmsbuild · 2021-09-01T12:11:13Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-768ae1/18198/summary.html
COMMIT: 9ebd1a9
CMSSW: CMSSW_12_1_X_2021-08-31-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34902/18198/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 7 EventSetupRecord::get warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-768ae1/18198/llvm-analysis/esrget-sa.txt for details.
CMS StaticAnalyzer warnings: There are 1 inherits from legacy modules warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-768ae1/18198/llvm-analysis/legacy-mod-sa.txt for details.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000404
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3000370
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 38 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

perrotta · 2021-09-01T12:19:51Z

+1

makortel mentioned this pull request Aug 17, 2021

Migrate last pieces of MixingModule and PreMixingModule to esConsumes cms-sw/framework-team#221

Closed

cmsbuild added this to the CMSSW_12_1_X milestone Aug 17, 2021

cmsbuild added alca-pending code-checks-pending dqm-pending l1-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Aug 17, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 17, 2021

cmsbuild added tests-started and removed tests-pending labels Aug 17, 2021

cmsbuild added tests-rejected code-checks-pending tests-pending and removed tests-started tests-rejected code-checks-approved labels Aug 17, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 17, 2021

cmsbuild mentioned this pull request Aug 30, 2021

ECAL esConsumes migration of CalibCalorimetry packages #35077

Merged

thomreis mentioned this pull request Aug 31, 2021

proposal to move CaloVShape and derived objects to CondFormats or CalibFormats #29833

Open

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Aug 31, 2021

cmsbuild added tests-started and removed tests-approved labels Aug 31, 2021

cmsbuild added tests-approved and removed tests-started labels Sep 1, 2021

cmsbuild added orp-approved and removed orp-pending labels Sep 1, 2021

cmsbuild merged commit 40f81cc into cms-sw:master Sep 1, 2021

makortel deleted the esconsumesMixing branch September 1, 2021 15:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate BMixingModule and EcalShapeBase-derived classes to esConsumes #34902

Migrate BMixingModule and EcalShapeBase-derived classes to esConsumes #34902

makortel commented Aug 17, 2021 •

edited

cmsbuild commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

perrotta commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

makortel commented Aug 17, 2021

makortel commented Aug 26, 2021

makortel commented Aug 27, 2021

makortel commented Aug 30, 2021

makortel commented Aug 31, 2021

tvami commented Aug 31, 2021

rekovic commented Aug 31, 2021

cmsbuild commented Aug 31, 2021

perrotta commented Aug 31, 2021

smuzaffar commented Sep 1, 2021

cmsbuild commented Sep 1, 2021

perrotta commented Sep 1, 2021

Migrate BMixingModule and EcalShapeBase-derived classes to esConsumes #34902

Migrate BMixingModule and EcalShapeBase-derived classes to esConsumes #34902

Conversation

makortel commented Aug 17, 2021 • edited

PR description:

PR validation:

cmsbuild commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

perrotta commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

RelVals

RelVals-INPUT

AddOn Tests

cmsbuild commented Aug 17, 2021

cmsbuild commented Aug 17, 2021

makortel commented Aug 17, 2021

makortel commented Aug 26, 2021

makortel commented Aug 27, 2021

makortel commented Aug 30, 2021

makortel commented Aug 31, 2021

tvami commented Aug 31, 2021

rekovic commented Aug 31, 2021

cmsbuild commented Aug 31, 2021

perrotta commented Aug 31, 2021

smuzaffar commented Sep 1, 2021

cmsbuild commented Sep 1, 2021

Comparison Summary

perrotta commented Sep 1, 2021

makortel commented Aug 17, 2021 •

edited