Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PF relinking for the HI reco sequence #19927

Merged
merged 3 commits into from Aug 7, 2017

Conversation

mandrenguyen
Copy link
Contributor

PF relinking was never included in the heavy-ion reco sequence. The logic was that was that the charged hadron subtraction for in-time PU isn't useful, since only a single PV is reco'd. However, it appears there are additional elements of egamma ID that come in at this stage. This PR reincludes the PF relinking with (only) the charged hadron subtraction explicitly disabled. With 1.5k events of wf 150.1, changes were observed to PF candidates of ID = 2 (electrons) and 4 (photons), but no changes to charged hadrons, neutral hadrons, or muons. For now, both the "Tmp" and final PF candidates are stored, so the event size will increase. Eventually the Tmp collections will be dropped, but we would like to keep both temporarily for comparisons.

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mandrenguyen for master.

It involves the following packages:

RecoHI/Configuration
RecoHI/HiEgammaAlgos

@perrotta, @cmsbuild, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@MiheeJo, @jazzitup, @echapon, @yenjie, @kurtejung, @dgulhan, @yetkinyilmaz this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Jul 26, 2017 via email

@slava77
Copy link
Contributor

slava77 commented Jul 26, 2017

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 26, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21790/console Started: 2017/07/26 19:05

@mandrenguyen
Copy link
Contributor Author

@slava77 The PF candidates for the Tmp and final collection appear to have different values for photons and electrons. We'd like to study this a bit further to see how it affects heavy ion events. I think we have the tools, but we're still missing the samples, as the heavier events are no longer produced in relval due to memory constraints. We will look into it further with privately produced samples, which are anyway being produced for trigger studies.

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19927/21790/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 2363781
  • DQMHistoTests: Total failures: 29561
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2334054
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 93 log files, 14 edm output root files, 23 DQM output files

PFTowers.src = cms.InputTag("particleFlow")
akCs3PFJets.src = cms.InputTag("particleFlow")
akCs4PFJets.src = cms.InputTag("particleFlow")
kt4PFJetsForRho.src = cms.InputTag("particleFlow")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this all changed here instead of the places where the modules are defined

@cmsbuild
Copy link
Contributor

Pull request #19927 was updated. @perrotta, @cmsbuild, @slava77, @davidlange6 can you please check and sign again.

@davidlange6
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 31, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21900/console Started: 2017/07/31 10:43

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 3, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19927/22039/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 25
  • DQMHistoTests: Total histograms compared: 2639325
  • DQMHistoTests: Total failures: 15102
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2624042
  • DQMHistoTests: Total skipped: 181
  • DQMHistoTests: Total Missing objects: 0
  • Checked 102 log files, 14 edm output root files, 25 DQM output files

@slava77
Copy link
Contributor

slava77 commented Aug 3, 2017 via email

@mandrenguyen
Copy link
Contributor Author

@slava77 So you'd like me to remove this import from Reconstruction_HI_cff.py, as well as RecoEcal.Configuration.RecoEcal_cff.py and place it directly in RecoEcal.EgammaClusterProducers.reducedRecHitsSequence_cff ?

@slava77
Copy link
Contributor

slava77 commented Aug 3, 2017 via email

@mandrenguyen
Copy link
Contributor Author

Ok, so I leave things as is then. Thanks.

@perrotta
Copy link
Contributor

perrotta commented Aug 4, 2017

This is the CPU time taken by the additional modules, measured with wf 140.53 (re-reco of 2015 HI data).
The time per event of step2 (DQM removed) in those data is about 14s

The extra time taken by those new modules alone is of the order of 3% of the previous total one


   added      +0.02%         0.00 ms/ev ->         2.53 ms/ev interestingGamIsoDetIdEB
   added      +0.20%         0.00 ms/ev ->        26.57 ms/ev interestingGedGamIsoDetIdEB
   added      +0.04%         0.00 ms/ev ->         5.77 ms/ev interestingEcalDetIdPFEB
   added      +0.02%         0.00 ms/ev ->         2.64 ms/ev interestingGamIsoDetIdEE
   added      +0.04%         0.00 ms/ev ->         4.97 ms/ev interestingGedGamIsoDetIdEE
   added      +0.00%         0.00 ms/ev ->         0.05 ms/ev PhotonIDProdGED
   added      +0.02%         0.00 ms/ev ->         2.85 ms/ev interestingEcalDetIdPFEE
   added      +0.00%         0.00 ms/ev ->         0.12 ms/ev pfNoPileUpIso
   added      +0.00%         0.00 ms/ev ->         0.01 ms/ev pfPileUpIso
   added      +0.00%         0.00 ms/ev ->         0.03 ms/ev eidRobustHighEnergy
   added      +0.00%         0.00 ms/ev ->         0.29 ms/ev interestingEcalDetIdPFES
   added      +0.02%         0.00 ms/ev ->         2.12 ms/ev pfNoPileUpCandidates
   added      +0.00%         0.00 ms/ev ->         0.24 ms/ev hfEMClusters
   added      +0.00%         0.00 ms/ev ->         0.21 ms/ev egmElectronIsolationCITK
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev egmElectronIsolationPileUpCITK
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev electronHcalPFClusterIsolationProducer
   added      +0.04%         0.00 ms/ev ->         5.25 ms/ev particleBasedIsolationTmp
   added      +0.00%         0.00 ms/ev ->         0.62 ms/ev particleFlowPtrs
   added      +0.00%         0.00 ms/ev ->         0.07 ms/ev eidLoose
   added      +0.00%         0.00 ms/ev ->         0.04 ms/ev PhotonIDProd
   added      +0.00%         0.00 ms/ev ->         0.32 ms/ev particleFlowTmpPtrs
   added      +0.02%         0.00 ms/ev ->         2.53 ms/ev interestingEcalDetIdRefinedEB
   added      +0.01%         0.00 ms/ev ->         1.65 ms/ev interestingEcalDetIdEBU
   added      +0.00%         0.00 ms/ev ->         0.05 ms/ev hfRecoEcalCandidate
   added      +0.01%         0.00 ms/ev ->         1.05 ms/ev interestingEcalDetIdRefinedEE
   added      +0.01%         0.00 ms/ev ->         0.82 ms/ev fixedGridRhoFastjetAllTmp
   added      +0.01%         0.00 ms/ev ->         1.96 ms/ev gedPhotons
   added      +0.01%         0.00 ms/ev ->         0.95 ms/ev reducedHcalRecHits
   added      +0.00%         0.00 ms/ev ->         0.11 ms/ev interestingGedEgammaIsoHCALDetId
   added      +0.00%         0.00 ms/ev ->         0.03 ms/ev eidTight
   added      +0.01%         0.00 ms/ev ->         1.72 ms/ev interestingOotGamIsoDetIdEB
   added      +0.00%         0.00 ms/ev ->         0.35 ms/ev interestingOotGamIsoDetIdEE
   added      +0.00%         0.00 ms/ev ->         0.07 ms/ev electronEcalPFClusterIsolationProducer
   added      +0.00%         0.00 ms/ev ->         0.05 ms/ev interestingEcalDetIdRefinedES
   added      +0.01%         0.00 ms/ev ->         1.39 ms/ev photonHcalPFClusterIsolationProducer
   added      +0.04%         0.00 ms/ev ->         5.39 ms/ev particleFlow
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev interestingOotEgammaIsoESDetId
   added      +0.02%         0.00 ms/ev ->         2.75 ms/ev interestingEcalDetIdEB
   added      +0.00%         0.00 ms/ev ->         0.07 ms/ev interestingGedEleIsoDetIdEB
   added      +0.03%         0.00 ms/ev ->         3.68 ms/ev interestingEcalDetIdEE
   added      +0.00%         0.00 ms/ev ->         0.03 ms/ev interestingGedEleIsoDetIdEE
   added      +0.01%         0.00 ms/ev ->         1.89 ms/ev interestingEcalDetIdOOTPFEB
   added      +0.01%         0.00 ms/ev ->         0.79 ms/ev interestingEcalDetIdOOTPFEE
   added      +0.10%         0.00 ms/ev ->        13.19 ms/ev photonEcalPFClusterIsolationProducer
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev interestingGedEgammaIsoESDetId
   added      +1.62%         0.00 ms/ev ->       215.38 ms/ev reducedEcalRecHitsEB
   added      +0.00%         0.00 ms/ev ->         0.14 ms/ev ootPhotonHcalPFClusterIsolationProducer
   added      +0.28%         0.00 ms/ev ->        36.92 ms/ev reducedEcalRecHitsEE
   added      +0.00%         0.00 ms/ev ->         0.05 ms/ev muonEcalDetIds
   added      +0.00%         0.00 ms/ev ->         0.01 ms/ev interestingEcalDetIdOOTPFES
   added      +0.06%         0.00 ms/ev ->         7.50 ms/ev chargedHadronPFTrackIsolation
   added      +0.00%         0.00 ms/ev ->         0.01 ms/ev pfPileUpAllChargedParticles
   added      +0.01%         0.00 ms/ev ->         1.68 ms/ev selectDigi
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev eidRobustLoose
   added      +0.00%         0.00 ms/ev ->         0.22 ms/ev gedGsfElectrons
   added      +0.01%         0.00 ms/ev ->         1.22 ms/ev interestingTrackEcalDetIds
   added      +0.11%         0.00 ms/ev ->        14.42 ms/ev reducedEcalRecHitsES
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev interestingOotEgammaIsoHCALDetId
   added      +0.11%         0.00 ms/ev ->        14.09 ms/ev egmPhotonIsolationCITK
   added      +0.00%         0.00 ms/ev ->         0.02 ms/ev eidRobustTight
   added      +0.01%         0.00 ms/ev ->         1.13 ms/ev ootPhotonEcalPFClusterIsolationProducer
   added      +0.04%         0.00 ms/ev ->         5.38 ms/ev particleBasedIsolation

@perrotta
Copy link
Contributor

perrotta commented Aug 4, 2017

About the event size, it increases overall by some 8.3%: 4397870 -> 4762106


  0.0 ->        49.1         49     NEWO   0.00     floatedmValueMap_eidLoose__reRECO.
  0.0 ->        59.3         59     NEWO   0.00     booledmValueMap_PhotonIDProd_PhotonCutBasedIDLoose_reRECO.
  0.0 ->      7130.2       7130     NEWO   0.16     recoPhotons_gedPhotons__reRECO.
  0.0 ->       335.5        335     NEWO   0.01     recoSuperClusters_hfEMClusters__reRECO.
  0.0 ->        60.8         61     NEWO   0.00     booledmValueMap_PhotonIDProdGED_PhotonCutBasedIDLoose_reRECO.
  0.0 ->    109837.0     109837     NEWO   2.50     EcalRecHitsSorted_reducedEcalRecHitsEB__reRECO.
  0.0 ->        75.1         75     NEWO   0.00     recoSuperClustersToOnerecoHFEMClusterShapesAssociation_hfEMClusters__reRECO.
  0.0 ->       132.8        133     NEWO   0.00     floatedmValueMap_photonHcalPFClusterIsolationProducer__reRECO.
  0.0 ->     11933.5      11934     NEWO   0.27     recoPFCandidateedmFwdPtrs_particleFlowTmpPtrs__reRECO.
  0.0 ->       139.7        140     NEWO   0.00     booledmValueMap_chargedHadronPFTrackIsolation__reRECO.
  0.0 ->        52.1         52     NEWO   0.00     floatedmValueMap_eidRobustHighEnergy__reRECO.
  0.0 ->        61.3         61     NEWO   0.00     booledmValueMap_PhotonIDProdGED_PhotonCutBasedIDLooseEM_reRECO.
  0.0 ->        46.0         46     NEWO   0.00     recoCaloClusters_hfEMClusters__reRECO.
  0.0 ->       580.3        580     NEWO   0.01     recoHFEMClusterShapes_hfEMClusters__reRECO.
  0.0 ->        46.6         47     NEWO   0.00     double_fixedGridRhoFastjetAllTmp__reRECO.
  0.0 ->       109.7        110     NEWO   0.00     recoGsfElectrons_gedGsfElectrons__reRECO.
  0.0 ->        87.5         88     NEWO   0.00     HFRecHitsSorted_reducedHcalRecHits_hfreco_reRECO.
  0.0 ->        65.9         66     NEWO   0.00     floatedmValueMap_ootPhotonEcalPFClusterIsolationProducer__reRECO.
  0.0 ->     12394.6      12395     NEWO   0.28     recoPFCandidateedmFwdPtrs_particleFlowPtrs__reRECO.
  0.0 ->      4153.5       4154     NEWO   0.09     EEDigiCollection_selectDigi_selectedEcalEEDigiCollection_reRECO.
  0.0 ->     57337.6      57338     NEWO   1.30     EcalRecHitsSorted_reducedEcalRecHitsES__reRECO.
  0.0 ->        76.1         76     NEWO   0.00     recoPhotonsrecoPhotonrecoPhotonsrecoPhotonedmrefhelperFindUsingAdvanceedmRefedmValueMap_gedPhotons_valMapPFEgammaCandToPhoton_reRECO.
  0.0 ->        79.0         79     NEWO   0.00     recoPFCandidateedmPtredmValueMap_particleFlow_muons_reRECO.
  0.0 ->        59.3         59     NEWO   0.00     booledmValueMap_PhotonIDProd_PhotonCutBasedIDTight_reRECO.
  0.0 ->    109507.0     109507     NEWO   2.49     recoPFCandidates_particleFlow__reRECO.
  0.0 ->     46947.0      46947     NEWO   1.07     EcalRecHitsSorted_reducedEcalRecHitsEE__reRECO.
  0.0 ->        60.8         61     NEWO   0.00     booledmValueMap_PhotonIDProdGED_PhotonCutBasedIDTight_reRECO.
  0.0 ->        49.9         50     NEWO   0.00     recoRecoEcalCandidates_hfRecoEcalCandidate__reRECO.
  0.0 ->        58.0         58     NEWO   0.00     recoPFCandidateedmPtredmValueMap_particleFlow_electrons_reRECO.
  0.0 ->        57.7         58     NEWO   0.00     floatedmValueMap_electronEcalPFClusterIsolationProducer__reRECO.
  0.0 ->        50.6         51     NEWO   0.00     floatedmValueMap_eidRobustTight__reRECO.
  0.0 ->        59.8         60     NEWO   0.00     booledmValueMap_PhotonIDProd_PhotonCutBasedIDLooseEM_reRECO.
  0.0 ->        49.1         49     NEWO   0.00     floatedmValueMap_eidTight__reRECO.
  0.0 ->        65.6         66     NEWO   0.00     floatedmValueMap_ootPhotonHcalPFClusterIsolationProducer__reRECO.
  0.0 ->        50.1         50     NEWO   0.00     HORecHitsSorted_reducedHcalRecHits_horeco_reRECO.
  0.0 ->       881.0        881     NEWO   0.02     EBDigiCollection_selectDigi_selectedEcalEBDigiCollection_reRECO.
  0.0 ->        57.7         58     NEWO   0.00     floatedmValueMap_electronHcalPFClusterIsolationProducer__reRECO.
  0.0 ->       652.0        652     NEWO   0.01     recoPFCandidatesrecoPFCandidaterecoPFCandidatesrecoPFCandidateedmrefhelperFindUsingAdvanceedmRefsedmValueMap_particleBasedIsolation_gedPhotons_reRECO.
  0.0 ->       373.3        373     NEWO   0.01     HBHERecHitsSorted_reducedHcalRecHits_hbhereco_reRECO.
  0.0 ->       106.0        106     NEWO   0.00     recoPFCandidateedmPtredmValueMap_particleFlow_photons_reRECO.
  0.0 ->       133.9        134     NEWO   0.00     floatedmValueMap_photonEcalPFClusterIsolationProducer__reRECO.
  0.0 ->        97.1         97     NEWO   0.00     recoPFCandidatesrecoPFCandidaterecoPFCandidatesrecoPFCandidateedmrefhelperFindUsingAdvanceedmRefsedmValueMap_particleBasedIsolation_gedGsfElectrons_reRECO.
  0.0 ->        50.6         51     NEWO   0.00     floatedmValueMap_eidRobustLoose__reRECO.

4397870 -> 4762106 364236 8.3 ALL BRANCHES

@perrotta
Copy link
Contributor

perrotta commented Aug 7, 2017

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 7, 2017

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

@davidlange6
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 3d50ec9 into cms-sw:master Aug 7, 2017
mandrenguyen added a commit to mandrenguyen/cmssw that referenced this pull request Sep 27, 2017
@mandrenguyen mandrenguyen deleted the pfRelinkingHI9XY branch March 4, 2022 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants