Update globalreco_tracking sequence for Phase2 #27529

fabiocos · 2019-07-16T09:45:11Z

PR description:

Minimal update to the globalreco_tracking sequence for Phase2 in order to include MTD. This is needed by the new test wf 20434.1, so far systematically failing because the sequence is not running the needed module:

----- Begin Fatal Exception 16-Jul-2019 03:02:20 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 2 event: 105 stream: 0
   [1] Running path 'RECOSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'RECOSIMoutput'
   [3] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices4D'
   [4] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting4D'
   [5] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting4D'
   [6] Prefetching for module PrimaryVertexProducer/'unsortedOfflinePrimaryVertices4D'
   [7] Prefetching for module TOFPIDProducer/'tofPID'
   [8] Calling method for module PrimaryVertexProducer/'unsortedOfflinePrimaryVertices4DnoPID'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::ValueMap&lt;float&gt;
Looking for module label: trackExtenderWithMTD
Looking for productInstanceName: generalTracksigmat0

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

See issue #27507 for details.

PR validation:

Test workflow 20434.1 runs to completion without apparent issues.

cmsbuild · 2019-07-16T09:45:41Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-07-16T09:53:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27529/10899

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Reconstruction_cff.py modified in PR(s): RecoEgamma/EgammaElectronProducers cleanup #27477

cmsbuild · 2019-07-16T09:53:46Z

A new Pull Request was created by @fabiocos (Fabio Cossutti) for master.

It involves the following packages:

Configuration/StandardSequences

@cmsbuild, @franzoni, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @dgulhan this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

fabiocos · 2019-07-16T11:11:54Z

please test workflow 20434.1

fabiocos · 2019-07-16T11:12:09Z

@kpedro88 @slava77 @perrotta FYI

cmsbuild · 2019-07-16T11:12:16Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1484/console Started: 2019/07/16 13:12

cmsbuild · 2019-07-16T13:16:05Z

+1
Tested at: 412dee0
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-103d39/1484/summary.html

cmsbuild · 2019-07-16T13:16:09Z

Comparison job queued.

slava77 · 2019-07-16T13:46:17Z

Configuration/StandardSequences/python/Reconstruction_cff.py

@@ -140,6 +140,11 @@
 from Configuration.Eras.Modifier_phase2_timing_layer_bar_cff import phase2_timing_layer_bar
 (phase2_timing_layer_tile | phase2_timing_layer_bar).toReplaceWith(globalreco,_phase2_timing_layer_globalreco)

+_phase2_timing_layer_globalreco_tracking = globalreco_tracking.copy()
+_phase2_timing_layer_globalreco_tracking += fastTimingGlobalReco
+(phase2_timing_layer_tile | phase2_timing_layer_bar).toReplaceWith(globalreco_tracking,_phase2_timing_layer_globalreco_tracking)


shouldn't the piece above with .toReplaceWith(globalreco,_phase2_timing_layer_globalreco) be removed?

also, it would be better to modify globalreco_tracking right after its definition instead of after some other sequences

@slava77 I see your point, as it is now fastTimingGlobalReco appears twice inside teh globalreco sequence. As far as I can see this module should be independent on the other parts of globalreco, so it should work

@slava77 I agree with both of your comments

from https://github.com/cms-sw/cmssw/pull/27529/files#r303926710

also, it would be better to modify globalreco_tracking right after its definition instead of after some other sequences

to be more clear, I'm suggesting to

move the currently proposed L138,139,140-143 to L122

remove everything else from L137 to L144

I think you meant 141 - 143, 140 at this point should just go together with 137

kpedro88 · 2019-07-16T14:37:29Z

assign upgrade

cmsbuild · 2019-07-16T14:38:03Z

New categories assigned: upgrade

@kpedro88 you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2019-07-16T15:50:40Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-103d39/1484/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-103d39/20434.1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_trackingOnly_2023D41+HARVESTFullGlobal_trackingOnly_2023D41

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 3081858
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3081535
DQMHistoTests: Total skipped: 322
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
Checked 133 log files, 14 edm output root files, 32 DQM output files

slava77 · 2019-07-17T16:51:59Z

@slava77 I think the latest version should address your comments, reducing the amount of changes to the main configuration (although with no visible change in the final result)

Thank you.
This looks good to me know.
In retrospect, this change should have been made in #25612 (or #25063 )

cmsbuild · 2019-07-17T18:46:53Z

+1
Tested at: e714b19
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-103d39/1509/summary.html

cmsbuild · 2019-07-17T18:46:57Z

Comparison job queued.

fabiocos · 2019-07-17T20:40:31Z

@kpedro88 I would like to merge this as soon as the result of the test is out (it does not crash any more, I want to see the comparison). It is one of the last pieces I want to have for 11_0_0_pre4

kpedro88 · 2019-07-17T20:41:29Z

+upgrade

fabiocos · 2019-07-17T20:44:14Z

@slava77 probably you are correct, unfortunately at the time of the initial deployment we were in a hurry and things were not optimal...

cmsbuild · 2019-07-17T21:26:04Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-103d39/1509/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-103d39/20434.1_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_trackingOnly_2023D41+HARVESTFullGlobal_trackingOnly_2023D41

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 2622600
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2622298
DQMHistoTests: Total skipped: 302
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
Checked 133 log files, 14 edm output root files, 32 DQM output files

fabiocos · 2019-07-18T07:30:42Z

+operations

this update allows the globalreco_tracking sequence to be used also for phase2 scenarios with MTD, without changing the behaviour of the globalreco sequence (only the calling order of the modules is modified)

fabiocos · 2019-07-18T07:34:29Z

+1

fabiocos · 2019-07-18T07:38:58Z

@mrodozov is the bot stuck on this PR?

mrodozov · 2019-07-18T09:50:02Z

@fabiocos would you try again +operations and +1 please

cmsbuild · 2019-07-18T09:50:31Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

fabiocos · 2019-07-18T09:52:11Z

+operations

fabiocos · 2019-07-18T09:52:15Z

+1

fabiocos · 2019-07-18T09:52:45Z

@mrodozov although the labels were already correctly set to signed...

mrodozov · 2019-07-18T09:54:01Z

yes it did right after I commented it changed them :/

Update globalreco_tracking sequence for Phase2

412dee0

cmsbuild added this to the CMSSW_11_0_X milestone Jul 16, 2019

cmsbuild added code-checks-pending comparison-pending operations-pending orp-pending pending-signatures tests-pending labels Jul 16, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 16, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 16, 2019

cmsbuild added tests-approved and removed tests-started labels Jul 16, 2019

slava77 reviewed Jul 16, 2019

View reviewed changes

cmsbuild added the upgrade-pending label Jul 16, 2019

cmsbuild mentioned this pull request Jul 16, 2019

RecoEgamma/EgammaElectronProducers cleanup #27477

Merged

cmsbuild added comparison-available and removed comparison-pending labels Jul 16, 2019

cmsbuild added tests-approved and removed tests-started labels Jul 17, 2019

cmsbuild added upgrade-approved and removed upgrade-pending labels Jul 17, 2019

cmsbuild removed the comparison-pending label Jul 17, 2019

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending orp-pending pending-signatures labels Jul 18, 2019

cmsbuild merged commit e543447 into cms-sw:master Jul 18, 2019

perrotta mentioned this pull request Jul 22, 2019

Failure of wf 20434.1 after Phase2 relval updates #27507

Closed

fabiocos deleted the fc-fix20434_1 branch August 7, 2019 13:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update globalreco_tracking sequence for Phase2 #27529

Update globalreco_tracking sequence for Phase2 #27529

fabiocos commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

fabiocos commented Jul 16, 2019

fabiocos commented Jul 16, 2019

cmsbuild commented Jul 16, 2019 •

edited

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

slava77 Jul 16, 2019

slava77 Jul 16, 2019

fabiocos Jul 16, 2019

kpedro88 Jul 16, 2019

slava77 Jul 17, 2019

slava77 Jul 17, 2019

fabiocos Jul 17, 2019

kpedro88 commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

slava77 commented Jul 17, 2019

cmsbuild commented Jul 17, 2019

cmsbuild commented Jul 17, 2019

fabiocos commented Jul 17, 2019

kpedro88 commented Jul 17, 2019

fabiocos commented Jul 17, 2019

cmsbuild commented Jul 17, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

mrodozov commented Jul 18, 2019

cmsbuild commented Jul 18, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

mrodozov commented Jul 18, 2019

Update globalreco_tracking sequence for Phase2 #27529

Update globalreco_tracking sequence for Phase2 #27529

Conversation

fabiocos commented Jul 16, 2019

PR description:

PR validation:

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

fabiocos commented Jul 16, 2019

fabiocos commented Jul 16, 2019

cmsbuild commented Jul 16, 2019 • edited

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

slava77 Jul 16, 2019

Choose a reason for hiding this comment

slava77 Jul 16, 2019

Choose a reason for hiding this comment

fabiocos Jul 16, 2019

Choose a reason for hiding this comment

kpedro88 Jul 16, 2019

Choose a reason for hiding this comment

slava77 Jul 17, 2019

Choose a reason for hiding this comment

slava77 Jul 17, 2019

Choose a reason for hiding this comment

fabiocos Jul 17, 2019

Choose a reason for hiding this comment

kpedro88 commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

cmsbuild commented Jul 16, 2019

slava77 commented Jul 17, 2019

cmsbuild commented Jul 17, 2019

cmsbuild commented Jul 17, 2019

fabiocos commented Jul 17, 2019

kpedro88 commented Jul 17, 2019

fabiocos commented Jul 17, 2019

cmsbuild commented Jul 17, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

mrodozov commented Jul 18, 2019

cmsbuild commented Jul 18, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

fabiocos commented Jul 18, 2019

mrodozov commented Jul 18, 2019

cmsbuild commented Jul 16, 2019 •

edited