Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase #4384 and add customisation function #4483

Merged
merged 37 commits into from Jul 3, 2014

Conversation

mark-grimes
Copy link

Rebased #4384 on to this afternoon's IB and added a customisation function. The 3D clustering is now OFF by default and is enabled by telling cmsDriver:

--customise SLHCUpgradeSimulations/Configuration/customise_PFlow.customise_use3DHCalClusters

Note that if using the "--command" method to add this to runTheMatrix I think it doesn't apply the original customise for the geometry, so that has to be specified as well. E.g. for 102xx the normal customisation would be SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019. So to run the 102xx matrix tests with 3D clustering:

runTheMatrix.py -w upgrade -l 10261 --command "--customise SLHCUpgradeSimulations/Configuration/combinedCustoms.cust_2019,SLHCUpgradeSimulations/Configuration/customise_PFlow.customise_use3DHCalClusters"

@lgray could you check over this and make sure I've merged all the conflicts properly, and implemented the customisation function properly?
In particular I had some merge conflicts in RecoParticleFlow/Configuration/python/RecoParticleFlow_EventContent_cff.py so that should be checked. I also had to add some code to RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h to get it to compile.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 1, 2014

A new Pull Request was created by @mark-grimes (Mark Grimes) for CMSSW_6_2_X_SLHC.

Rebase #4384 and add customisation function

It involves the following packages:

DataFormats/ParticleFlowReco
RecoMET/METAlgorithms
RecoMET/METProducers
RecoParticleFlow/Configuration
RecoParticleFlow/PFClusterProducer
SLHCUpgradeSimulations/Configuration

@civanch, @nclopezo, @mdhildreth, @cmsbuild, @StoyanStoynev, @slava77, @Degano can you please review it and eventually sign? Thanks.
@TaiSakuma, @nhanvtran, @bachtis, @lgray this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
@fratnikov, @mark-grimes you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

@lgray
Copy link
Contributor

lgray commented Jul 2, 2014

@mark-grimes This looks good to go. All conflicts you had were just due to parallel development of HGC stuff. Customization function produces the correct configuration.

Thanks for taking care of this!

pfClusteringHCAL = cms.Sequence(particleFlowRecHitHCAL*pfClusteringHCALall)

pfClusteringHO = cms.Sequence(particleFlowRecHitHO*particleFlowClusterHO)
pfClusteringHCAL = cms.Sequence()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, wait a second, we need to put the default clustering back here.

@lgray
Copy link
Contributor

lgray commented Jul 2, 2014

@mark-grimes I'll grab your branch in a bit and put the default clustering back in inside particleFlowCluster_cff.py .. Otherwise the change is hardcoded.

Sorry for the mess.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 2, 2014

-1
Tested at: c6a11fd
When I ran the RelVals I found an error in the following worklfows:
11600 step3

runTheMatrix-results/11600_FourMuPt1_200+FourMuPt_1_200_2017Aging_GenSimFull+DigiFull_2017Aging+RecoFull_2017Aging+HARVESTFull_2017Aging/step3_FourMuPt1_200+FourMuPt_1_200_2017Aging_GenSimFull+DigiFull_2017Aging+RecoFull_2017Aging+HARVESTFull_2017Aging.log
----- Begin Fatal Exception 02-Jul-2014 10:08:03 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFRecHitProducer/'particleFlowRecHitHBHEHO'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::SortedCollection >
Looking for module label: hbheUpgradeReco
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

10000 step3

runTheMatrix-results/10000_FourMuPt1_200+FourMuPt_1_200_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017/step3_FourMuPt1_200+FourMuPt_1_200_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017.log
----- Begin Fatal Exception 02-Jul-2014 10:08:04 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFRecHitProducer/'particleFlowRecHitHBHEHO'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::SortedCollection >
Looking for module label: hbheUpgradeReco
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

10200 step3

runTheMatrix-results/10200_FourMuPt1_200+FourMuPt_1_200_2019_GenSimFull+DigiFull_2019+RecoFull_2019+HARVESTFull_2019/step3_FourMuPt1_200+FourMuPt_1_200_2019_GenSimFull+DigiFull_2019+RecoFull_2019+HARVESTFull_2019.log
----- Begin Fatal Exception 02-Jul-2014 10:08:05 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

11200 step3

runTheMatrix-results/11200_FourMuPt1_200+FourMuPt_1_200_2019WithGEM_GenSimFull+DigiFull_2019WithGEM+RecoFull_2019WithGEM+HARVESTFull_2019WithGEM/step3_FourMuPt1_200+FourMuPt_1_200_2019WithGEM_GenSimFull+DigiFull_2019WithGEM+RecoFull_2019WithGEM+HARVESTFull_2019WithGEM.log
----- Begin Fatal Exception 02-Jul-2014 10:08:07 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

11800 step3

runTheMatrix-results/11800_FourMuPt1_200+FourMuPt_1_200_2019Aging_GenSimFull+DigiFull_2019Aging+RecoFull_2019Aging+HARVESTFull_2019Aging/step3_FourMuPt1_200+FourMuPt_1_200_2019Aging_GenSimFull+DigiFull_2019Aging+RecoFull_2019Aging+HARVESTFull_2019Aging.log
----- Begin Fatal Exception 02-Jul-2014 10:08:06 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

11400 step3

runTheMatrix-results/11400_FourMuPt1_200+FourMuPt_1_200_BE5DPixel10D_GenSimFull+DigiFull_BE5DPixel10D+RecoFull_BE5DPixel10D+HARVESTFull_BE5DPixel10D/step3_FourMuPt1_200+FourMuPt_1_200_BE5DPixel10D_GenSimFull+DigiFull_BE5DPixel10D+RecoFull_BE5DPixel10D+HARVESTFull_BE5DPixel10D.log
----- Begin Fatal Exception 02-Jul-2014 10:08:33 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

10400 step3

runTheMatrix-results/10400_FourMuPt1_200+FourMuPt_1_200_BE5D_GenSimFull+DigiFull_BE5D+RecoFull_BE5D+HARVESTFull_BE5D/step3_FourMuPt1_200+FourMuPt_1_200_BE5D_GenSimFull+DigiFull_BE5D+RecoFull_BE5D+HARVESTFull_BE5D.log
----- Begin Fatal Exception 02-Jul-2014 10:08:35 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

12000 step3

runTheMatrix-results/12000_FourMuPt1_200+FourMuPt_1_200_Extended2023_GenSimFull+DigiFull_Extended2023+RecoFull_Extended2023+HARVESTFull_Extended2023/step3_FourMuPt1_200+FourMuPt_1_200_Extended2023_GenSimFull+DigiFull_Extended2023+RecoFull_Extended2023+HARVESTFull_Extended2023.log
----- Begin Fatal Exception 02-Jul-2014 10:08:51 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

13000 step3

runTheMatrix-results/13000_FourMuPt1_200+FourMuPt_1_200_Extended2023Muon_GenSimFull+DigiFull_Extended2023Muon+RecoFull_Extended2023Muon+HARVESTFull_Extended2023Muon/step3_FourMuPt1_200+FourMuPt_1_200_Extended2023Muon_GenSimFull+DigiFull_Extended2023Muon+RecoFull_Extended2023Muon+HARVESTFull_Extended2023Muon.log
----- Begin Fatal Exception 02-Jul-2014 10:13:19 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module PFBlockProducer/'particleFlowBlock'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector
Looking for module label: particleFlowClusterHCAL
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4483/19/summary.html

@lgray
Copy link
Contributor

lgray commented Jul 2, 2014

@mark-grimes will fix these errors too

@mark-grimes
Copy link
Author

Yes, sorry. I ran tests overnight and every single one failed. I've been stuck in a first aid course all morning though and haven't been able to fix them. Is it okay to leave the final fixes with you @lgray?

@lgray
Copy link
Contributor

lgray commented Jul 2, 2014

Yep, working on them.

@lgray
Copy link
Contributor

lgray commented Jul 2, 2014

@mark-grimes almost done

@mark-grimes
Copy link
Author

merge

All looks good now, tested without 3D activated on all scenarios with muons and pions - only the known failures in HGCal harvesting. Tested select scenarios with 3D activated and all seem to run fine. No idea how it affects the results though.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 3, 2014

Pull request #4483 was updated. @civanch, @nclopezo, @mdhildreth, @cmsbuild, @StoyanStoynev, @slava77, @Degano can you please check and sign again.

cmsbuild added a commit that referenced this pull request Jul 3, 2014
Rebase #4384 and add customisation function
@cmsbuild cmsbuild merged commit aa86824 into cms-sw:CMSSW_6_2_X_SLHC Jul 3, 2014
@mark-grimes mark-grimes deleted the rebase4384 branch July 3, 2014 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants