EventSetupRecord::doGet uses prefetching #31775

Dr15Jones · 2020-10-13T18:50:16Z

PR description:

Changed EventSetupRecord::doGet to work with prefetching. This required the introduction of ESGetTokenGeneric and a new esConsumes method.
Added the EDConsumerBase::registerLateConsumes to be able to safely call esConsumes based on the contents of the EventSetup.

PR validation:

Code compiles and relevant framework unit tests pass.

cmsbuild · 2020-10-13T18:50:43Z

The code-checks are being triggered in jenkins.

Dr15Jones · 2020-10-13T18:54:41Z

FWCore/Modules/src/EventSetupRecordDataGetter.cc


 //
 // class decleration
 //
 namespace edm {
-  class EventSetupRecordDataGetter : public edm::stream::EDAnalyzer<> {
+  class EventSetupRecordDataGetter
+      : public edm::global::EDAnalyzer<edm::RunCache<std::nullptr_t>, edm::LuminosityBlockCache<std::nullptr_t>> {


As a bonus, I could make it a global module :).

cmsbuild · 2020-10-13T18:56:33Z

-code-checks

ERROR: Build errors found during clang-tidy run.

FWCore/Utilities/interface/ESGetTokenGeneric.h:43:5: error: defaulted definition of move assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric&&) noexcept = default;
    ^
FWCore/Utilities/interface/ESGetTokenGeneric.h:44:5: error: defaulted definition of copy assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric const&) noexcept = default;
    ^
Suppressed 315 warnings (314 in non-user code, 1 with check filters).
--
FWCore/Utilities/interface/ESGetTokenGeneric.h:43:5: error: defaulted definition of move assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric&&) noexcept = default;
    ^
FWCore/Utilities/interface/ESGetTokenGeneric.h:44:5: error: defaulted definition of copy assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric const&) noexcept = default;
    ^
Suppressed 423 warnings (422 in non-user code, 1 with check filters).
--
FWCore/Utilities/interface/ESGetTokenGeneric.h:43:5: error: defaulted definition of move assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric&&) noexcept = default;
    ^
FWCore/Utilities/interface/ESGetTokenGeneric.h:44:5: error: defaulted definition of copy assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric const&) noexcept = default;
    ^
Suppressed 739 warnings (738 in non-user code, 1 with check filters).
--
FWCore/Utilities/interface/ESGetTokenGeneric.h:43:5: error: defaulted definition of move assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric&&) noexcept = default;
    ^
FWCore/Utilities/interface/ESGetTokenGeneric.h:44:5: error: defaulted definition of copy assignment operator is not constexpr [clang-diagnostic-error]
    constexpr ESGetTokenGeneric& operator=(ESGetTokenGeneric const&) noexcept = default;
    ^
FWCore/Utilities/interface/ProcessBlockIndex.h:39:5: warning: deleted member function should be public [modernize-use-equals-delete]
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

cmsbuild · 2020-10-13T19:45:30Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-13T19:50:35Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31775/19028

This PR adds an extra 64KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/interface/EDConsumerBase.h modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746, Delete unscheduled modules that are not consumed by any scheduled module #29553
- File FWCore/Framework/src/EDConsumerBase.cc modified in PR(s): Delete unscheduled modules that are not consumed by any scheduled module #29553

cmsbuild · 2020-10-13T19:50:57Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/Framework
FWCore/Modules
FWCore/Utilities

@makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @wddgit this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2020-10-13T19:51:18Z

please test

cmsbuild · 2020-10-13T19:51:39Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-10-13-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9922/console Started: 2020/10/13 21:54

cmsbuild · 2020-10-13T22:07:31Z

-1

Tested at: d3dd43b

CMSSW: CMSSW_11_2_X_2020-10-13-1100
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0f968/9922/summary.html

I found follow errors while testing this PR

Failed tests: Build

Build:

I found compilation error when building:

>> Building LCG reflex dict from header file src/DataFormats/L1TMuon/src/classes.h
In file included from input_line_9:67:
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-13-1100/src/DataFormats/L1TMuon/interface/L1MuBMTrack.h:36:
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-13-1100/src/FWCore/Framework/interface/EventSetup.h:33:
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-13-1100/src/FWCore/Framework/interface/EventSetupRecord.h:48:
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-13-1100/src/FWCore/Utilities/interface/ESGetTokenGeneric.h:37:15: error: exception specification of explicitly defaulted default constructor does not match the calculated one
    constexpr ESGetTokenGeneric() noexcept = default;
              ^
Error: rootcling: compilation failure (tmp/slc7_amd64_gcc820/src/DataFormats/L1TMuon/src/DataFormatsL1TMuon/a/DataFormatsL1TMuon_xr6d41ad8bbc_dictUmbrella.h)
gmake: *** [tmp/slc7_amd64_gcc820/src/DataFormats/L1TMuon/src/DataFormatsL1TMuon/a/DataFormatsL1TMuon_xr.cc] Error 1
>> Compiling LCG dictionary: tmp/slc7_amd64_gcc820/src/DataFormats/L1TMuon/src/DataFormatsL1TMuon/a/DataFormatsL1TMuon_xr.cc

cmsbuild · 2020-10-13T22:07:32Z

Comparison not run due to Build errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2020-10-14T17:57:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31775/19071

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/interface/EDConsumerBase.h modified in PR(s): Make EventSetup get to throw if called without a token when EDModule consumed any ES product #31746, Delete unscheduled modules that are not consumed by any scheduled module #29553
- File FWCore/Framework/src/EDConsumerBase.cc modified in PR(s): Delete unscheduled modules that are not consumed by any scheduled module #29553

cmsbuild · 2020-10-14T17:57:26Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-10-14-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9965/console Started: 2020/10/14 19:57

cmsbuild · 2020-10-14T17:57:27Z

Pull request #31775 was updated. @makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please check and sign again.

cmsbuild · 2020-10-14T18:03:07Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0f968/9947/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2543752
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2543723
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

cmsbuild · 2020-10-14T21:33:38Z

+1
Tested at: 1301a46
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0f968/9965/summary.html
CMSSW: CMSSW_11_2_X_2020-10-14-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-14T21:33:41Z

Comparison job queued.

cmsbuild · 2020-10-14T22:48:13Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0f968/9965/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2543752
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2543729
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

makortel · 2020-10-14T23:05:06Z

+1

cmsbuild · 2020-10-14T23:05:27Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2020-10-15T00:14:48Z

+1

cmsbuild added this to the CMSSW_11_2_X milestone Oct 13, 2020

cmsbuild added code-checks-pending comparison-pending core-pending orp-pending pending-signatures tests-pending labels Oct 13, 2020

Dr15Jones mentioned this pull request Oct 13, 2020

Update EventSetupRecordDataGetter to use esConsumes #31716

Closed

Dr15Jones commented Oct 13, 2020

View reviewed changes

cmsbuild added code-checks-rejected and removed code-checks-pending labels Oct 13, 2020

Dr15Jones force-pushed the prefetchRecord_doGet branch from efe3c4a to d3dd43b Compare October 13, 2020 19:45

cmsbuild added code-checks-pending and removed code-checks-rejected labels Oct 13, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 13, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 13, 2020

cmsbuild added comparison-notrun and removed comparison-pending tests-started labels Oct 13, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 14, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 14, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 14, 2020

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Oct 14, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 14, 2020

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Oct 14, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 15, 2020

cmsbuild merged commit 8c8fe09 into cms-sw:master Oct 15, 2020

Dr15Jones deleted the prefetchRecord_doGet branch October 15, 2020 16:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EventSetupRecord::doGet uses prefetching #31775

EventSetupRecord::doGet uses prefetching #31775

Dr15Jones commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

Dr15Jones Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

Dr15Jones commented Oct 13, 2020

cmsbuild commented Oct 13, 2020 •

edited

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020 •

edited

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

makortel commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

qliphy commented Oct 15, 2020

EventSetupRecord::doGet uses prefetching #31775

EventSetupRecord::doGet uses prefetching #31775

Conversation

Dr15Jones commented Oct 13, 2020

PR description:

PR validation:

cmsbuild commented Oct 13, 2020

Dr15Jones Oct 13, 2020

Choose a reason for hiding this comment

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

Dr15Jones commented Oct 13, 2020

cmsbuild commented Oct 13, 2020 • edited

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020 • edited

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

makortel commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

qliphy commented Oct 15, 2020

cmsbuild commented Oct 13, 2020 •

edited

cmsbuild commented Oct 14, 2020 •

edited