Update the (pre|post)Source(Lumi|Run) signals #21872

fwyzard · 2018-01-17T11:07:15Z

Update the signals to pass

the LuminosityBlockIndex to the (pre|post)SourceLumi signals;
the RunIndex to the (pre|post)SourceRun signals.

Update the FastTimerService to make use of the updated signals to account for the time spent in the Source transitions.

Update the other clients code to ignore the new parameters.

Update the signals to pass - the `LuminosityBlockIndex` to the `(pre|post)SourceLumi` signals; - the `RunIndex` to the `(pre|post)SourceRun` signals. Update the client code to ignore the new parameters.

Take advantage of the updated signatures for the `(pre|post)Source(Lumi|Run)` signals to account for the time spent in the Source transitions.

fwyzard · 2018-01-17T11:07:31Z

@cmsbuild, please test

cmsbuild · 2018-01-17T11:07:33Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-01-17T11:14:06Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-21872/2919

cmsbuild · 2018-01-17T11:14:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25491/console Started: 2018/01/17 12:14

cmsbuild · 2018-01-17T11:14:25Z

A new Pull Request was created by @fwyzard (Andrea Bocci) for master.

It involves the following packages:

DQMServices/Core
FWCore/Framework
FWCore/MessageService
FWCore/ServiceRegistry
FWCore/Services
HLTrigger/Timer

@smuzaffar, @Dr15Jones, @vazzolini, @kmaeshima, @dmitrijus, @Martin-Grunewald, @silviodonato, @cmsbuild, @jfernan2, @fwyzard, @vanbesien can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @barvic, @wddgit this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

dmitrijus · 2018-01-17T12:37:22Z

+1

Hi @fwyzard, have you tested this DQM's Standalone.h shim?
If not, it is not much of an issue, just a bit more work for whoever compiles the gui.

fwyzard · 2018-01-17T12:45:49Z

It builds, of course.
I haven't run any other tests (nor know how).

cmsbuild · 2018-01-17T16:01:19Z

-1

Tested at: 5b8e8d2

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
5f458f3
6de20cc
85ab2e1
225ce8a
53a56b5
27f0def
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25491/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25491/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25491/summary.html

I found follow errors while testing this PR

Failed tests: AddOn

AddOn:

I found errors in the following addon tests:

cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake2.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:45:49 2018-date Wed Jan 17 16:39:46 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:52:56 2018-date Wed Jan 17 16:39:49 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:51:41 2018-date Wed Jan 17 16:39:50 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:56:02 2018-date Wed Jan 17 16:39:54 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:53:55 2018-date Wed Jan 17 16:39:59 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:46:46 2018-date Wed Jan 17 16:40:01 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:52:06 2018-date Wed Jan 17 16:40:06 2018 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:52:43 2018-date Wed Jan 17 16:40:08 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake2.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:52:57 2018-date Wed Jan 17 16:40:11 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:52:45 2018-date Wed Jan 17 16:40:12 2018 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:45:35 2018-date Wed Jan 17 16:40:16 2018 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:53:07 2018-date Wed Jan 17 16:40:18 2018 s - exit: 35584
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_Fake1.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:44:47 2018-date Wed Jan 17 16:40:22 2018 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/nweek-02507/slc6_amd64_gcc630/cms/cmssw/CMSSW_10_0_X_2018-01-16-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Jan 17 16:53:32 2018-date Wed Jan 17 16:40:25 2018 s - exit: 35584

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
5f458f3
6de20cc
85ab2e1
225ce8a
53a56b5
27f0def
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25491/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25491/git-merge-result

cmsbuild · 2018-01-17T16:01:22Z

Comparison job queued.

Dr15Jones · 2018-01-17T16:06:28Z

The one add on test I looked at crashed with

#5  0x00007fc48ffff642 in edm::ErrorObj& edm::ErrorObj::opltlt<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_0_X_2018-01-16-2300/lib/slc6_amd64_gcc630/pluginL1TriggerL1TGlobalPlugins.so
#6  0x00007fc48fff73ad in L1TGlobalSummary::endRun(edm::Run const&, edm::EventSetup const&) () from /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_0_X_2018-01-16-2300/lib/slc6_amd64_gcc630/pluginL1TriggerL1TGlobalPlugins.so
#7  0x00007fc4a3ff48ab in edm::one::EDAnalyzerBase::doEndRun(edm::RunPrincipal const&, edm::EventSetup const&, edm::ModuleCallingContext const*) () from /build/cmsbld/jenkins/workspace/ib-any-integration/CMSSW_10_0_X_2018-01-16-2300/lib/slc6_amd64_gcc630/libFWCoreFramework.so

fwyzard · 2018-01-17T17:55:18Z

The error looks reasonably unrelated to this PR.
If it persists I'll have a look on top of pre4.

cmsbuild · 2018-01-26T16:34:56Z

-1

Tested at: 5b8e8d2

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25642/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

The relvals timed out after 2 hours.
When I ran the RelVals I found an error in the following worklfows:
4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

136.7611 step2

runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

cmsbuild · 2018-01-26T16:34:58Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

fabiocos · 2018-01-27T17:36:12Z

please test

cmsbuild · 2018-01-27T17:36:31Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/25682/console Started: 2018/01/27 19:47

cmsbuild · 2018-01-27T22:07:57Z

+1
Tested at: 5b8e8d2
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25682/summary.html

cmsbuild · 2018-01-27T22:08:00Z

Comparison job queued.

cmsbuild · 2018-01-27T22:42:18Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21872/25682/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-21872/1325.7_TTbar_13_94XNanoAODINPUT+TTbar_13_94XNanoAODINPUT+NANOEDMMC2017

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 26
DQMHistoTests: Total histograms compared: 2464350
DQMHistoTests: Total failures: 28
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2464153
DQMHistoTests: Total skipped: 169
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.580000000089 KiB( 22 files compared)
Checked 110 log files, 9 edm output root files, 26 DQM output files

fabiocos · 2018-01-28T11:42:52Z

+1

Update the (pre|post)Source(Lumi|Run) signals to match cms-sw#21872.

fwyzard added 2 commits January 17, 2018 11:54

Update the (pre|post)Source(Lumi|Run) signals

fa3092b

Update the signals to pass - the `LuminosityBlockIndex` to the `(pre|post)SourceLumi` signals; - the `RunIndex` to the `(pre|post)SourceRun` signals. Update the client code to ignore the new parameters.

FastTimerService: make use of the updated signals

5b8e8d2

Take advantage of the updated signatures for the `(pre|post)Source(Lumi|Run)` signals to account for the time spent in the Source transitions.

cmsbuild added this to the CMSSW_10_0_X milestone Jan 17, 2018

cmsbuild added code-checks-pending comparison-pending core-pending dqm-pending hlt-pending orp-pending pending-signatures tests-pending labels Jan 17, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 17, 2018

cmsbuild added tests-started and removed tests-pending labels Jan 17, 2018

cmsbuild added dqm-approved and removed dqm-pending labels Jan 17, 2018

cmsbuild added tests-rejected and removed tests-started labels Jan 17, 2018

cmsbuild added tests-started and removed tests-pending labels Jan 26, 2018

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jan 26, 2018

cmsbuild added comparison-pending tests-pending tests-started and removed comparison-notrun tests-rejected tests-pending labels Jan 27, 2018

cmsbuild added tests-approved and removed tests-started labels Jan 27, 2018

cmsbuild added comparison-available and removed comparison-pending labels Jan 27, 2018

cmsbuild added orp-approved and removed orp-pending labels Jan 28, 2018

cmsbuild merged commit 10b588d into cms-sw:master Jan 28, 2018

fwyzard added a commit to fwyzard/cmssw that referenced this pull request Jan 29, 2018

Update the (pre|post)Source(Lumi|Run) signals

62dbfe5

Update the (pre|post)Source(Lumi|Run) signals to match cms-sw#21872.

fwyzard mentioned this pull request Jan 29, 2018

Instrument cmssw for nvidia nvprof/nvvm (10.1.x) #22009

Merged

fwyzard deleted the Update_pre_post_Source_Lumi_Run_signals_100pre4 branch January 31, 2018 10:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the (pre|post)Source(Lumi|Run) signals #21872

Update the (pre|post)Source(Lumi|Run) signals #21872

fwyzard commented Jan 17, 2018

fwyzard commented Jan 17, 2018

cmsbuild commented Jan 17, 2018

cmsbuild commented Jan 17, 2018

cmsbuild commented Jan 17, 2018 •

edited

cmsbuild commented Jan 17, 2018

dmitrijus commented Jan 17, 2018

fwyzard commented Jan 17, 2018 •

edited

cmsbuild commented Jan 17, 2018

cmsbuild commented Jan 17, 2018

Dr15Jones commented Jan 17, 2018

fwyzard commented Jan 17, 2018

cmsbuild commented Jan 26, 2018

cmsbuild commented Jan 26, 2018

fabiocos commented Jan 27, 2018

cmsbuild commented Jan 27, 2018 •

edited

cmsbuild commented Jan 27, 2018

cmsbuild commented Jan 27, 2018

cmsbuild commented Jan 27, 2018

fabiocos commented Jan 28, 2018

Update the (pre|post)Source(Lumi|Run) signals #21872

Update the (pre|post)Source(Lumi|Run) signals #21872

Conversation

fwyzard commented Jan 17, 2018

fwyzard commented Jan 17, 2018

cmsbuild commented Jan 17, 2018

cmsbuild commented Jan 17, 2018

cmsbuild commented Jan 17, 2018 • edited

cmsbuild commented Jan 17, 2018

dmitrijus commented Jan 17, 2018

fwyzard commented Jan 17, 2018 • edited

cmsbuild commented Jan 17, 2018

cmsbuild commented Jan 17, 2018

Dr15Jones commented Jan 17, 2018

fwyzard commented Jan 17, 2018

cmsbuild commented Jan 26, 2018

cmsbuild commented Jan 26, 2018

fabiocos commented Jan 27, 2018

cmsbuild commented Jan 27, 2018 • edited

cmsbuild commented Jan 27, 2018

cmsbuild commented Jan 27, 2018

cmsbuild commented Jan 27, 2018

fabiocos commented Jan 28, 2018

cmsbuild commented Jan 17, 2018 •

edited

fwyzard commented Jan 17, 2018 •

edited

cmsbuild commented Jan 27, 2018 •

edited