Async global end LuminosityBlock transition #20327

Dr15Jones · 2017-08-30T23:17:58Z

The global end LuminosityBlock transition was supposed to be converted to asynchronous at the same time end Run was changed. This corrects that oversight. This also allowed removing many synchronous methods which were replaced with equivalent asynchronous implementations.

This was accidentally left off when the global end run was changed to asynchronous.

Removed methods which now have asynchronous versions and the synchronous versions are not used anymore.

cmsbuild · 2017-08-30T23:18:17Z

The code-checks are being triggered in jenkins.

cmsbuild · 2017-08-30T23:18:18Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/Framework
FWCore/Integration

@cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @wddgit this is something you requested to watch as well.
@davidlange6, @slava77 you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2017-08-30T23:18:23Z

please test

Dr15Jones · 2017-08-30T23:18:27Z

+1

cmsbuild · 2017-08-30T23:18:35Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22605/console Started: 2017/08/31 03:20

cmsbuild · 2017-08-30T23:18:36Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2017-08-30T23:28:02Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-20327/404

Code check has found code style and quality issues which could be resolved by applying a patch in https://cmssdt.cern.ch/SDT/code-checks/PR-20327/404/git-diff.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/PR-20327/404/git-diff.patch | patch -p1

You can run scram build code-checks to apply code checks directly (this will soon be required for PRs to be considered)

Dr15Jones · 2017-08-31T19:40:55Z

The unit test failure has nothing to do with this pull request.

cmsbuild · 2017-08-31T21:27:59Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22623/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 26
DQMHistoTests: Total histograms compared: 2656041
DQMHistoTests: Total failures: 217
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2655635
DQMHistoTests: Total skipped: 189
DQMHistoTests: Total Missing objects: 0
Checked 107 log files, 14 edm output root files, 26 DQM output files

Dr15Jones · 2017-09-01T12:21:20Z

Please test
Although the previous test only failed because of unrelated reasons

cmsbuild · 2017-09-01T12:22:27Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/22642/console Started: 2017/09/01 14:22

cmsbuild · 2017-09-01T17:10:33Z

-1

Tested at: d3907ad

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
94fab3e
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22642/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22642/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22642/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

Unit Tests:

I found errors in the following unit tests:

---> test testRecoMETMETProducers had ERRORS

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
94fab3e
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22642/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22642/git-merge-result

cmsbuild · 2017-09-01T17:10:35Z

Comparison job queued.

Dr15Jones · 2017-09-01T18:13:19Z

@smuzaffar the unit test failure is not cause by this pull request. It appears to be a race condition in the testing sytem. The error is

01-Sep-2017 16:56:32 CEST  Initiating request to open file file:ttbarForMetTests.root
----- Begin Fatal Exception 01-Sep-2017 16:56:32 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootFileSequenceBase::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling File::sysopen()
Exception Message:
Failed to open the file 'ttbarForMetTests.root'
   Additional Info:
      [a] Input file file:ttbarForMetTests.root could not be opened.
      [b] open() failed with system error 'No such file or directory' (error code 2)
----- End Fatal Exception -------------------------------------------------
+ die 'Failure using recoMET_genMet_cfg.py' 84
+ echo Failure using recoMET_genMet_cfg.py: status 84
Failure using recoMET_genMet_cfg.py: status 84

but the file ttbarForMetTests.root is generated in the same shell script that runs the test:
https://github.com/cms-sw/cmssw/blob/CMSSW_9_3_X/RecoMET/METProducers/test/runtests.sh

The file in question was even successfully used by several tests run in the same shell script

+ cmsRun /build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-08-31-2300/src/RecoMET/METProducers/test/recoMET_caloMet_cfg.py
01-Sep-2017 16:50:17 CEST  Initiating request to open file file:ttbarForMetTests.root
01-Sep-2017 16:50:19 CEST  Successfully opened file file:ttbarForMetTests.root
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 at 01-Sep-2017 16:50:49.343 CEST
01-Sep-2017 16:50:52 CEST  Closed file file:ttbarForMetTests.root

and

+ cmsRun /build/cmsbld/jenkins-workarea/workspace/ib-any-integration/CMSSW_9_3_X_2017-08-31-2300/src/RecoMET/METProducers/test/recoMET_tcMet_cfg.py
01-Sep-2017 16:50:58 CEST  Initiating request to open file file:ttbarForMetTests.root
01-Sep-2017 16:51:00 CEST  Successfully opened file file:ttbarForMetTests.root
Begin processing the 1st record. Run 1, Event 1, LumiSection 1 at 01-Sep-2017 16:51:07.395 CEST
01-Sep-2017 16:56:24 CEST  Closed file file:ttbarForMetTests.root

cmsbuild · 2017-09-01T18:54:31Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20327/22642/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 26
DQMHistoTests: Total histograms compared: 2656041
DQMHistoTests: Total failures: 210
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2655642
DQMHistoTests: Total skipped: 189
DQMHistoTests: Total Missing objects: 0
Checked 107 log files, 14 edm output root files, 26 DQM output files

Dr15Jones · 2017-09-01T19:58:10Z

@smuzaffar @davidlange6 There are several tests which do rm *.root which if run at the same time as testRecoMETMETProducers would cause the failure happening in this pull request.

cdj@cdj> git grep 'rm ' | grep '\*.root' | grep sh | grep test
Alignment/APEEstimation/test/cfgTemplate/createStep2.bash:  rm ${ROOTFILEBASE}/workingArea/*.root
DQMServices/Components/test/run_fastHadd_tests.sh:    rm -fr *.root
DQMServices/Core/python/test/test_DQMMultiThread.sh:  rm -fr *.root
FastSimulation/MaterialEffects/test/getNIFiles.sh:rm *.root
FastSimulation/MaterialEffects/test/getNIFiles_py.sh:rm *.root
HLTrigger/Configuration/test/runIntegration.csh:    rm -f  ${name}/*.root

smuzaffar · 2017-09-01T21:15:57Z

good catch @Dr15Jones , yes this explains why these root files disappear.
I can update build rule to create separate unittest-name directory to starts the test from. That should avoid these type of deletions. Some test might need fixing if they assume that they start from $CMSSW_BASE
OR fix these above mention tests to only delete those root files which they write.

Dr15Jones · 2017-09-02T14:00:29Z

@davidlange6 do you intend on including this in 9_3 or waiting until 9_4?

Dr15Jones · 2017-09-05T15:31:27Z

@davidlange6 the unit test failures are not do to this pull request.

davidlange6 · 2017-09-06T11:55:20Z

merge

Dr15Jones added 2 commits August 30, 2017 16:42

Run global endLumi transitions asynchronously

2c1642d

This was accidentally left off when the global end run was changed to asynchronous.

Removed unused synchronous methods

a75c41e

Removed methods which now have asynchronous versions and the synchronous versions are not used anymore.

cmsbuild added this to the CMSSW_9_3_X milestone Aug 30, 2017

cmsbuild added code-checks-pending comparison-pending core-pending orp-pending pending-signatures tests-pending labels Aug 30, 2017

cmsbuild removed core-pending pending-signatures labels Aug 30, 2017

cmsbuild added core-approved fully-signed labels Aug 30, 2017

cmsbuild added tests-started and removed tests-pending labels Aug 30, 2017

cmsbuild added code-checks-rejected and removed code-checks-pending labels Aug 30, 2017

Application of clang-tidy rules

5e56b90

cmsbuild added code-checks-pending and removed code-checks-rejected core-approved fully-signed tests-started labels Aug 31, 2017

cmsbuild added comparison-available and removed comparison-pending labels Aug 31, 2017

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-rejected labels Sep 1, 2017

cmsbuild added tests-started and removed tests-pending labels Sep 1, 2017

cmsbuild added tests-rejected and removed tests-started labels Sep 1, 2017

cmsbuild added comparison-available and removed comparison-pending labels Sep 1, 2017

Dr15Jones mentioned this pull request Sep 2, 2017

limited:: module types #20358

Merged

smuzaffar modified the milestones: CMSSW_9_4_X, CMSSW_9_3_X Sep 6, 2017

cmsbuild merged commit 90a4be0 into cms-sw:master Sep 6, 2017

Dr15Jones deleted the asyncGlobalEndLumi branch September 8, 2017 18:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Async global end LuminosityBlock transition #20327

Async global end LuminosityBlock transition #20327

Dr15Jones commented Aug 30, 2017

cmsbuild commented Aug 30, 2017

cmsbuild commented Aug 30, 2017

Dr15Jones commented Aug 30, 2017

Dr15Jones commented Aug 30, 2017

cmsbuild commented Aug 30, 2017 •

edited

Loading

cmsbuild commented Aug 30, 2017

cmsbuild commented Aug 30, 2017

Dr15Jones commented Aug 31, 2017

cmsbuild commented Aug 31, 2017

Dr15Jones commented Sep 1, 2017

cmsbuild commented Sep 1, 2017 •

edited

Loading

cmsbuild commented Sep 1, 2017

cmsbuild commented Sep 1, 2017

Dr15Jones commented Sep 1, 2017

cmsbuild commented Sep 1, 2017

Dr15Jones commented Sep 1, 2017

smuzaffar commented Sep 1, 2017

Dr15Jones commented Sep 2, 2017

Dr15Jones commented Sep 5, 2017

davidlange6 commented Sep 6, 2017

Async global end LuminosityBlock transition #20327

Async global end LuminosityBlock transition #20327

Conversation

Dr15Jones commented Aug 30, 2017

cmsbuild commented Aug 30, 2017

cmsbuild commented Aug 30, 2017

Dr15Jones commented Aug 30, 2017

Dr15Jones commented Aug 30, 2017

cmsbuild commented Aug 30, 2017 • edited Loading

cmsbuild commented Aug 30, 2017

cmsbuild commented Aug 30, 2017

Dr15Jones commented Aug 31, 2017

cmsbuild commented Aug 31, 2017

Dr15Jones commented Sep 1, 2017

cmsbuild commented Sep 1, 2017 • edited Loading

cmsbuild commented Sep 1, 2017

cmsbuild commented Sep 1, 2017

Dr15Jones commented Sep 1, 2017

cmsbuild commented Sep 1, 2017

Dr15Jones commented Sep 1, 2017

smuzaffar commented Sep 1, 2017

Dr15Jones commented Sep 2, 2017

Dr15Jones commented Sep 5, 2017

davidlange6 commented Sep 6, 2017

cmsbuild commented Aug 30, 2017 •

edited

Loading

cmsbuild commented Sep 1, 2017 •

edited

Loading