Apply task chains to more code in FWCore/Framework #34697

Dr15Jones · 2021-07-29T18:10:51Z

PR description:

Looked for places where edm::make_waiting_task could be replaced with use of edm::waiting_task::chain constructs.

PR validation:

Code compiles and framework unit tests all pass.

Dr15Jones · 2021-07-29T18:12:15Z

FWCore/Framework/src/EventProcessor.cc

-        sentry.completedSuccessfully();
+            status->setResumer(std::move(iResumer));
+
+            auto group = task.group();


NOTE: I think in a latter PR this routine should be subdivided into smaller functions which would be called by this routine.

cmsbuild · 2021-07-29T18:17:41Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34697/24325

This PR adds an extra 56KB to repository

cmsbuild · 2021-07-29T18:18:06Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/Framework (core)

@makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @wddgit this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2021-07-29T18:35:35Z

please test

makortel · 2021-07-30T00:02:49Z

FWCore/Framework/src/Schedule.cc

+          iTask.doneWaiting(*iPtr);
+        }
+      };
+      return wrapped;


Would

return [f = std::forward<F>(iF>)](...){...};

work?

cmsbuild · 2021-07-30T02:13:53Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-619a4a/17344/summary.html
COMMIT: 16118a8
CMSSW: CMSSW_12_0_X_2021-07-29-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34697/17344/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-619a4a/17344/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-619a4a/17344/git-merge-result

RelVals-INPUT

11602.011602.0_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt35+2021+SingleElectronPt35_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
11634.511634.5_TTbar_14TeV+2021_pixelTrackingOnly+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST/step2_TTbar_14TeV+2021_pixelTrackingOnly+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST.log
11601.011601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log

Expand to see more relval errors ...

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 705
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2997837
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-07-30T07:33:59Z

please test
Retriggering to get a clean test environment, many PRs simultaneously included on top of this one

cmsbuild · 2021-07-30T10:52:31Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-619a4a/17353/summary.html
COMMIT: 16118a8
CMSSW: CMSSW_12_0_X_2021-07-29-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34697/17353/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 2998564
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2998541
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

jfernan2 · 2021-07-30T11:04:46Z

+1

makortel · 2021-08-16T20:12:53Z

+1

makortel · 2021-08-16T20:13:14Z

@cmsbuild, please test

Let's retest to be really sure

cmsbuild · 2021-08-16T20:13:17Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2021-08-16T23:14:54Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-619a4a/17812/summary.html
COMMIT: 16118a8
CMSSW: CMSSW_12_1_X_2021-08-16-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34697/17812/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 39
DQMHistoTests: Total histograms compared: 3000352
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3000323
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 38 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 165 log files, 37 edm output root files, 39 DQM output files
TriggerResults: no differences found

makortel · 2021-08-17T01:00:20Z

Just to confirm explicitly that the tests look ok.

qliphy · 2021-08-17T01:27:28Z

+1

Apply task chains to more code in FWCore/Framework

16118a8

cmsbuild added this to the CMSSW_12_0_X milestone Jul 29, 2021

cmsbuild added code-checks-pending core-pending orp-pending pending-signatures tests-pending labels Jul 29, 2021

Dr15Jones commented Jul 29, 2021

View reviewed changes

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 29, 2021

cmsbuild added tests-started and removed tests-pending labels Jul 29, 2021

makortel reviewed Jul 30, 2021

View reviewed changes

cmsbuild added tests-rejected and removed tests-started labels Jul 30, 2021

smuzaffar modified the milestones: CMSSW_12_0_X, CMSSW_12_1_X Jul 30, 2021

cmsbuild added tests-started and removed tests-rejected labels Jul 30, 2021

cmsbuild added tests-approved and removed tests-started labels Jul 30, 2021

cmsbuild mentioned this pull request Aug 4, 2021

Make header files in FWCore/Framework/interface consistent #34771

Merged

cmsbuild mentioned this pull request Aug 16, 2021

Moved headers from src to interface in FWCore/Framework #34891

Merged

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Aug 16, 2021

cmsbuild added tests-started and removed tests-approved labels Aug 16, 2021

cmsbuild added tests-approved and removed tests-started labels Aug 16, 2021

cmsbuild added orp-approved and removed orp-pending labels Aug 17, 2021

cmsbuild merged commit 93c52a8 into cms-sw:master Aug 17, 2021

Dr15Jones deleted the FWCoreFramework_task_chains branch August 17, 2021 15:20

This was referenced Aug 17, 2021

[GCC11] Bump to 11.2.1 cms-sw/cmsdist#7215

Merged

Fix radix sort test #34929

Merged

More Eigen GPU work, with fix for CUDA 11.4 cms-externals/eigen-git-mirror#7

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apply task chains to more code in FWCore/Framework #34697

Apply task chains to more code in FWCore/Framework #34697

Dr15Jones commented Jul 29, 2021 •

edited

Dr15Jones Jul 29, 2021

cmsbuild commented Jul 29, 2021

cmsbuild commented Jul 29, 2021

Dr15Jones commented Jul 29, 2021

makortel Jul 30, 2021

cmsbuild commented Jul 30, 2021

jfernan2 commented Jul 30, 2021

cmsbuild commented Jul 30, 2021

jfernan2 commented Jul 30, 2021

makortel commented Aug 16, 2021

makortel commented Aug 16, 2021

cmsbuild commented Aug 16, 2021

cmsbuild commented Aug 16, 2021

makortel commented Aug 17, 2021

qliphy commented Aug 17, 2021

Apply task chains to more code in FWCore/Framework #34697

Apply task chains to more code in FWCore/Framework #34697

Conversation

Dr15Jones commented Jul 29, 2021 • edited

PR description:

PR validation:

Dr15Jones Jul 29, 2021

Choose a reason for hiding this comment

cmsbuild commented Jul 29, 2021

cmsbuild commented Jul 29, 2021

Dr15Jones commented Jul 29, 2021

makortel Jul 30, 2021

Choose a reason for hiding this comment

cmsbuild commented Jul 30, 2021

RelVals-INPUT

Comparison Summary

jfernan2 commented Jul 30, 2021

cmsbuild commented Jul 30, 2021

Comparison Summary

jfernan2 commented Jul 30, 2021

makortel commented Aug 16, 2021

makortel commented Aug 16, 2021

cmsbuild commented Aug 16, 2021

cmsbuild commented Aug 16, 2021

Comparison Summary

makortel commented Aug 17, 2021

qliphy commented Aug 17, 2021

Dr15Jones commented Jul 29, 2021 •

edited