Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PFID with DNNs - updated model trained on Run3Summer21 #36243

Merged
merged 4 commits into from Nov 29, 2021

Conversation

valsdav
Copy link
Contributor

@valsdav valsdav commented Nov 24, 2021

PR description:

This PR updates the PFID DNN models trained on Run3Summer21 samples. A few additional variables have been included for electrons.
The paths of the models have been updated as well as the thresholds defining the default working point.
An additional threshold on the non-isolated background class has been included for electrons in PFEgammaFilters.

This PR comes along two PRs for ElectronIdentification (cms-data/RecoEgamma-ElectronIdentification#24) and PhotonIdentification (cms-data/RecoEgamma-PhotonIdentification#10) data repositories.

PR validation:

For documentation about the changes in the model training and input variables please refer to the slides at: https://indico.cern.ch/event/1096395/#12-egm-updates-for-run-3-prepa.

The correctness of the DNN configuration has been verified. A validation of the physics performance of the PFID WPs is ongoing in the PF and Egamma group.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36243/26878

  • This PR adds an extra 40KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @valsdav (Davide Valsecchi) for master.

It involves the following packages:

  • RecoEgamma/EgammaElectronProducers (reconstruction)
  • RecoEgamma/EgammaPhotonProducers (reconstruction)
  • RecoEgamma/ElectronIdentification (reconstruction)
  • RecoEgamma/PhotonIdentification (reconstruction)
  • RecoParticleFlow/PFProducer (reconstruction)

@jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@mmarionncern, @Sam-Harper, @afiqaize, @jainshilpi, @cbernet, @rovere, @lgray, @sobhatta, @lecriste, @hatakeyamak, @wrtabb, @varuns23, @seemasharmafnal, @ram1123 this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36243/26893

  • This PR adds an extra 40KB to repository

@cmsbuild
Copy link
Contributor

Pull request #36243 was updated. @jpata, @cmsbuild, @slava77 can you please check and sign again.

@@ -57,6 +57,10 @@ class PFEGammaFilters {
float ele_dnnLowPtThr_;
float ele_dnnHighPtBarrelThr_;
float ele_dnnHighPtEndcapThr_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these *Thr_ , as well as the ele_max* and most of the badHcal_* ones, can be made const

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We kept them non-const only because they are read from a group of options in the constructor https://github.com/cms-sw/cmssw/blob/master/RecoParticleFlow/PFProducer/src/PFEGammaFilters.cc#L72 and I felt it would be ugly to put that in the initializer list. How do you suggest to fix this problem? Thanks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhm, yes, it would require some gymnastic indeed...
Ok, don't mind

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36243/26894

  • This PR adds an extra 40KB to repository

@cmsbuild
Copy link
Contributor

Pull request #36243 was updated. @jpata, @cmsbuild, @slava77 can you please check and sign again.

@slava77
Copy link
Contributor

slava77 commented Nov 25, 2021

Copy link
Contributor

@jpata jpata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comment inline. let's run a test with profiling too.

@@ -15,14 +15,22 @@
particleFlowTmp.PFEGammaFiltersParameters.allowEEEinPF = cms.bool(False)

# Thresholds for e/gamma PFID DNN
# Thresholds for electron: Sig_isolated+Sig_nonIsolated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be good to have a reference to an indico presentation here with these numbers

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jpata, It should be this one https://indico.cern.ch/event/1096395/#12-egm-updates-for-run-3-prepa., shown at the last PPD meeting by @akapoorcern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, please put it in the code comment :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok! 👍

@jpata
Copy link
Contributor

jpata commented Nov 26, 2021

@cmsbuild please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d5bc9c/20787/summary.html
COMMIT: 2325db7
CMSSW: CMSSW_12_2_X_2021-11-26-1100/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36243/20787/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 8313 differences found in the comparisons
  • DQMHistoTests: Total files compared: 42
  • DQMHistoTests: Total histograms compared: 3247873
  • DQMHistoTests: Total failures: 8484
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3239367
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 41 files compared)
  • Checked 177 log files, 37 edm output root files, 42 DQM output files
  • TriggerResults: no differences found

@jpata
Copy link
Contributor

jpata commented Nov 27, 2021

@jpata
Copy link
Contributor

jpata commented Nov 29, 2021

  • memory in initialization

    • baseline Screenshot from 2021-11-29 10-53-40
    • this PR Screenshot from 2021-11-29 10-53-58
  • memory in processing

    • baseline Screenshot from 2021-11-29 10-55-18
    • this PR Screenshot from 2021-11-29 10-55-30
  • CPU in processing

    • baseline Screenshot from 2021-11-29 10-57-45
    • this PR Screenshot from 2021-11-29 10-57-19

@jpata
Copy link
Contributor

jpata commented Nov 29, 2021

+reconstruction

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@valsdav
Copy link
Contributor Author

valsdav commented Nov 29, 2021

  • memory in initialization

    * baseline ![Screenshot from 2021-11-29 10-53-40](https://user-images.githubusercontent.com/69717/143836977-379a4f40-839a-40ef-bc41-700afb796826.png)
    * this PR ![Screenshot from 2021-11-29 10-53-58](https://user-images.githubusercontent.com/69717/143836915-36b8c66d-213e-4e9e-91bc-0bd193e025eb.png)
    
    • memory in processing

      • baseline Screenshot from 2021-11-29 10-55-18
      • this PR Screenshot from 2021-11-29 10-55-30
    • CPU in processing

      • baseline Screenshot from 2021-11-29 10-57-45
      • this PR Screenshot from 2021-11-29 10-57-19

Thanks @jpata for the summary.

@jpata
Copy link
Contributor

jpata commented Nov 29, 2021

this is showing just the ::stream modules. The relevant mkFit modules are ::global

Is mkFit here a typo, or was this comment meant for the mkFit PR? GsfElectronProducer is showing up among the stream modules, and in any case, the total CPU/mem looks also fine on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants