Use Modifier.toReplaceWith() in TrackValidation_cff #13216

makortel · 2016-02-08T11:04:28Z

Purely technical change to adapt the phase1 customizations introduced in #13172 with to use Modifier.toReplaceWith() introduced in #13198.

Tested in 800pre6, no changes expected.

@rovere @VinInn

makortel · 2016-02-08T11:04:38Z

@Dr15Jones You might also be interested.

cmsbuild · 2016-02-08T11:04:45Z

A new Pull Request was created by @makortel (Matti Kortelainen) for CMSSW_8_0_X.

It involves the following packages:

Validation/RecoTrack

@cmsbuild, @vanbesien, @deguio, @davidlange6 can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @rovere, @VinInn, @cerati, @wmtford, @istaslis, @dgulhan this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

davidlange6 · 2016-02-08T11:12:05Z

please test

On Feb 8, 2016, at 12:04 PM, cmsbuild notifications@github.com wrote:

A new Pull Request was created by @makortel (Matti Kortelainen) for CMSSW_8_0_X.

It involves the following packages:

Validation/RecoTrack

@cmsbuild, @vanbesien, @deguio, @davidlange6 can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @rovere, @VinInn, @cerati, @wmtford, @istaslis, @dgulhan this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

—
Reply to this email directly or view it on GitHub.

cmsbuild · 2016-02-08T11:12:23Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11062/console

cmsbuild · 2016-02-08T13:21:39Z

-1
Tested at: a471b22
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
----- Begin Fatal Exception 08-Feb-2016 12:59:55 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log
----- Begin Fatal Exception 08-Feb-2016 13:07:56 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log
----- Begin Fatal Exception 08-Feb-2016 13:08:13 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log
----- Begin Fatal Exception 08-Feb-2016 13:12:29 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 08-Feb-2016 13:16:00 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 08-Feb-2016 13:25:03 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

50202.0 step3

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step3_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log
----- Begin Fatal Exception 08-Feb-2016 13:40:53 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log
----- Begin Fatal Exception 08-Feb-2016 13:47:09 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'prevalidation_step'
   [2] Calling event method for module MultiTrackValidator/'trackValidator'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: cutsRecoTracksDetachedTripletStep
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13216/11062/summary.html

Dr15Jones · 2016-02-08T13:40:34Z

Thanks. I appreciate getting rid the of my early hack and replacing it with the cleaner alternative.

makortel · 2016-02-08T14:46:12Z

Whoops, should have checked the "run2" matrix as well. I think the problem is in missing isChosen() check in #13198, and I see @Dr15Jones has already a fix in #13220.

makortel · 2016-02-10T08:16:12Z

#13220 got merged in CMSSW_8_0_X_2016-02-09-2300, and as also discussed in #13224, some workflows (e.g. 5.1) fail in step1 with

Traceback (most recent call last):
...
  File ".../CMSSW_8_0_X_2016-02-09-2300/python/Validation/RecoTrack/TrackValidation_cff.py", line 497, in <module>
    eras.fastSim.toModify(tracksValidationSeedSelectorsTrackingOnly, _seq = _fastSimTracksValidationSeedSelectorsTrackingOnly._seq)
  File ".../CMSSW_8_0_X_2016-02-09-2300/python/FWCore/ParameterSet/Config.py", line 1183, in toModify
    temp(obj)
  File ".../CMSSW_8_0_X_2016-02-09-2300/python/FWCore/ParameterSet/Config.py", line 1113, in __call__
    _modifyParametersFromDict(params, self.__args, self._raiseUnknownKey)
  File ".../CMSSW_8_0_X_2016-02-09-2300/python/FWCore/ParameterSet/Mixins.py", line 634, in _modifyParametersFromDict
    params[key].setValue(value)
AttributeError: '_SequenceCollection' object has no attribute 'setValue'

This PR removes the "offending" code in TrackValidation_cff.py and I was able to run e.g. 5.1 with CMSSW_8_0_X_2016-02-09-2300 plus this PR.

smuzaffar · 2016-02-10T08:20:59Z

cmsbuild, please test

cmsbuild · 2016-02-10T08:21:10Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11113/console

makortel · 2016-02-10T08:23:49Z

I should maybe note explicitly that this PR does not fix the other problem discussed in #13224 (affecting e.g. 140.2).

cmsbuild · 2016-02-10T09:19:31Z

-1
Tested at: a471b22
When I ran the RelVals I found an error in the following worklfows:
1306.0 step2

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step2_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step2

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step2_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

25202.0 step2

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step2_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

50202.0 step2

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step2_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13216/11113/summary.html

makortel · 2016-02-10T09:22:59Z

The remaining problems are the ones likely caused by #13224.

smuzaffar · 2016-02-10T09:35:48Z

hold
I think this might need an updated #13224 to go in first

cmsbuild · 2016-02-10T09:36:06Z

Pull request has been put on hold by @smuzaffar
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

makortel · 2016-02-10T09:39:45Z

@smuzaffar This PR should depend only the functionality added in #13220, i.e. it should be independent of #13224.

smuzaffar · 2016-02-10T11:11:32Z

@makortel , so this means that even though we have reverted #13224 for 11h00 IBs, we will still have errors because #13216 is not in yet merged along with #13220 ....right?

makortel · 2016-02-10T11:14:06Z

@smuzaffar Yes, I think that's exactly what should happen.

smuzaffar · 2016-02-10T11:16:46Z

cmsbuild, please test
I will update the test to run it based on CMSSW_8_0_X_2016-02-09-2300 IB and additional PR #13235 (which reverted #13224)

cmsbuild · 2016-02-10T11:17:05Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11122/console
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11127/console

Dr15Jones · 2016-02-10T14:38:54Z

@smuzaffar @davidlange6 This needs to be merged in before my fix to #13224 can be added.

davidlange6 · 2016-02-10T14:41:15Z

ok- but lets not get these as small dependent pieces next time around..

Use Modifier.toReplaceWith() in TrackValidation_cff

Dr15Jones · 2016-02-10T14:44:05Z

Agreed. It was unfortunate that our choice of pull request tests didn't catch this particular problem :(.

cmsbuild · 2016-02-10T18:54:33Z

-1

Tested at: a471b22
I found errors in the following unit tests:

---> test testRecoMETMETProducers had ERRORS

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13216/11127/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
a90a637
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13216/11127/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13216/11127/git-merge-result

Dr15Jones · 2016-02-10T21:04:25Z

The unit test failure is not caused by this pull request. I ran the same test using a vanilla CMSSW_8_0_X_2016-02-09-2300 build and it failed there as well.

Use Modifier.toReplaceWith in TrackValidation_cff

a471b22

cmsbuild added this to the Next CMSSW_8_0_X milestone Feb 8, 2016

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending comparison-pending labels Feb 8, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 8, 2016

cmsbuild added tests-rejected and removed tests-started labels Feb 8, 2016

makortel changed the title ~~Use Modifier.toReplaceWith in TrackValidation_cff~~ Use Modifier.toReplaceWith() in TrackValidation_cff Feb 8, 2016

cmsbuild added tests-pending and removed tests-rejected labels Feb 10, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 10, 2016

cmsbuild added tests-rejected and removed tests-started labels Feb 10, 2016

smuzaffar mentioned this pull request Feb 10, 2016

Configuration parameter copy and modify functions extended #13224

Merged

cmsbuild added the hold label Feb 10, 2016

cmsbuild added tests-pending tests-started and removed tests-rejected tests-pending labels Feb 10, 2016

davidlange6 added a commit that referenced this pull request Feb 10, 2016

Merge pull request #13216 from makortel/mtvUseToReplaceWith

4b6956e

Use Modifier.toReplaceWith() in TrackValidation_cff

davidlange6 merged commit 4b6956e into cms-sw:CMSSW_8_0_X Feb 10, 2016

Dr15Jones mentioned this pull request Feb 10, 2016

Configuration parameter copy and modify functions extended #13244

Merged

cmsbuild added tests-rejected and removed tests-started labels Feb 10, 2016

makortel deleted the mtvUseToReplaceWith branch October 20, 2016 11:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Modifier.toReplaceWith() in TrackValidation_cff #13216

Use Modifier.toReplaceWith() in TrackValidation_cff #13216

makortel commented Feb 8, 2016

makortel commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

davidlange6 commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

Dr15Jones commented Feb 8, 2016

makortel commented Feb 8, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

makortel commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

Dr15Jones commented Feb 10, 2016

davidlange6 commented Feb 10, 2016

Dr15Jones commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

Dr15Jones commented Feb 10, 2016

Use Modifier.toReplaceWith() in TrackValidation_cff #13216

Use Modifier.toReplaceWith() in TrackValidation_cff #13216

Conversation

makortel commented Feb 8, 2016

makortel commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

davidlange6 commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

Dr15Jones commented Feb 8, 2016

makortel commented Feb 8, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

makortel commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

makortel commented Feb 10, 2016

smuzaffar commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

Dr15Jones commented Feb 10, 2016

davidlange6 commented Feb 10, 2016

Dr15Jones commented Feb 10, 2016

cmsbuild commented Feb 10, 2016

Dr15Jones commented Feb 10, 2016