Trackster iteration filter for HGCAL electrons and photons

[SohamBhattacharya]

PR description:

This PR includes the following:

• Option to filter the tracksters using their iterations [1] in RecoParticleFlow/PFClusterProducer/python/particleFlowClusterHGC_cfi.py.
• Apply a default selection for the above filter such that only the TRKEM and EM iterations are selected.

The effects of this filter can be found in [2]. To summarize:

• Electron/photon energy resolution improves.
• Reconstruction:
  • Electron reconstruction efficiency decreases by a few percent at low pT.
  • Photon reconstruction efficiency decreases overall (being studied).
  • Electron/photon mis-reconstruction rate decreases significantly -- so there's an overall gain in terms of reconstruction to mis-reconstruction efficiencies.

It'll be good to test with profiling enabled -- we should see some timing gains because of the reduction in mis-reconstructed electrons/photons.

[1] https://cmsdoxygen.web.cern.ch/cmsdoxygen/CMSSW_12_2_0_pre2/doc/html/d5/de3/classticl_1_1Trackster.html#ab8b9ce1d52b6b6f74dfbbce754079398

[2] https://indico.cern.ch/event/1095370/contributions/4622717/attachments/2349624/4007590/presentation_test_tracksterIterationFilter_EgammaReco_2021-11-19.pdf

Tagging the egamma validators @archiron and @beaudett.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36204/26802

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @SohamBhattacharya for master.

It involves the following packages:

• RecoParticleFlow/PFClusterProducer (reconstruction)

@jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@mmarionncern, @felicepantaleo, @cbernet, @rovere, @lgray, @lecriste, @hatakeyamak, @ebrondol, @seemasharmafnal this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36204/26806

• This PR adds an extra 16KB to repository

[cmsbuild]

Pull request #36204 was updated. @jpata, @cmsbuild, @slava77 can you please check and sign again.

[jpata]

test parameters:

• enable_tests = profiling

[jpata]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5fc39f/20701/summary.html
COMMIT: dc04465
CMSSW: CMSSW_12_2_X_2021-11-22-2300/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36204/20701/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 2486 differences found in the comparisons
• DQMHistoTests: Total files compared: 42
• DQMHistoTests: Total histograms compared: 3247021
• DQMHistoTests: Total failures: 3229
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3243769
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 41 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 177 log files, 37 edm output root files, 42 DQM output files
• TriggerResults: no differences found

[jpata]

Looks like we see about a factor 2x timing improvement in several electron-related modules: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5fc39f/20701/profiling/23434.21/RES_CPU_compare_23434.21.txt

[6    -> 15  ] [3.5    -> 1.5   ]  [52.2      -> 21.37    ]         [ 21.37    /1416.87   - 52.2     /1491.56   *100% ] = [-1.99 % ]         ElectronSeedProducer::produce
[11   -> 20  ] [2.6    -> 1.0   ]  [38.45     -> 14.52    ]         [ 14.52    /1416.87   - 38.45    /1491.56   *100% ] = [-1.55 % ]         GsfTrackProducer::produce
[26   -> 35  ] [0.7    -> 0.3   ]  [10.11     -> 4.85     ]         [ 4.85     /1416.87   - 10.11    /1491.56   *100% ] = [-0.34 % ]         PFElecTkProducer::produce                         

and also something like 7% reduction in AOD size: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5fc39f/20701/profiling/23434.21/products_AOD_sizes_compare_23434.21.txt

Checking process _ step3_sizes_CMSSW_12_2_X_2021-11-22-2300_23434.21.txt and step3_sizes_23434.21.txt with useUnpacked=no (if above 100 or 10%):
Compare packed values
-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
...
 261192.0 ->     30672.5    -230520    -158.0  -3.22     recoSuperClusters_particleFlowSuperClusterHGCal__RECO.
 272140.0 ->     32567.7    -239572    -157.2  -3.35     recoCaloClusters_particleFlowSuperClusterHGCal__RECO.
...
  7148019 ->     6610163    -537856            -7.5     ALL BRANCHES

and 9% in miniAOD: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5fc39f/20701/profiling/23434.21/products_miniAOD_sizes_compare_23434.21.txt

Checking process _ step4_sizes_CMSSW_12_2_X_2021-11-22-2300_23434.21.txt and step4_sizes_23434.21.txt with useUnpacked=no (if above 100 or 10%):
Compare packed values
-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
  15177.1 ->      1174.8     -14002    -171.3  -4.50     patPhotons_slimmedPhotonsHGC__PAT.
  17589.7 ->      2959.4     -14630    -142.4  -4.70     patElectrons_slimmedElectronsHGC__PAT.
-------------------------------------------------------------
   311194 ->      282557     -28637            -9.2     ALL BRANCHES

We see reco changes in HGCal electrons and photons, which look expected, but do we also expect changes in GSF tracks, could you clarify?

https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_12_2_X_2021-11-22-2300+5fc39f/46995/validateJR/all_OldVSNew_TTbar14TeV2026D60wf28234p0/all_OldVSNew_TTbar14TeV2026D60wf28234p0c_log10recoGsfTracks_electronGsfTracks__RECO_obj_p.png

[SohamBhattacharya]

@jpata Thank you for the summary. Indeed, the reco changes for HGCal ele/pho seem as expected.
As for electronGsfTracks, it depends on particleFlowSuperClusterHGCal ([1] shows the dependency), which is affected (reduced) by this PR.

[1]
electronGsfTracks --> electronCkfTrackCandidates --> electronMergedSeeds --> ecalDrivenElectronSeeds --> particleFlowSuperClusterHGCal
https://cmssdt.cern.ch/lxr/source/TrackingTools/GsfTracking/python/GsfElectronGsfFit_cff.py#0008
https://cmssdt.cern.ch/lxr/source/FastSimulation/Tracking/python/electronCkfTrackCandidates_cff.py#0005
https://cmssdt.cern.ch/lxr/source/RecoParticleFlow/PFTracking/python/mergedElectronSeeds_cfi.py#0005
https://cmssdt.cern.ch/lxr/source/RecoEgamma/EgammaElectronProducers/python/ecalDrivenElectronSeeds_cff.py#0019

[jpata]

+reconstruction

• improves the filtering of fake electron, photon seeds in HGCAL
• we expect and observe reco changes in HGCAL-related electrons and photons
• timing and AOD/MINIAOD disk usage are also improved in Phase2

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1