Changes to jet pileup subtraction #29948

stepobr · 2020-05-21T20:51:39Z

PR description:

Changes related to pileup subtraction in jets. These existed for a long while and were used by Heavy Ions group and now migrating to central repo. This also fixes a bug in pileup subtraction, that appeared after HE phase 1 upgrade.

PR validation:

Tested using runTheMatrix.py

@bi-ran @mandrenguyen

cmsbuild · 2020-05-21T20:52:04Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-05-21T20:58:03Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15586

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoHI/HiJetAlgos/src/MultipleAlgoIterator.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935
- File RecoHI/HiJetAlgos/src/ParametrizedSubtractor.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937
- File RecoHI/HiJetAlgos/src/ParticleTowerProducer.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935
- File RecoJets/JetProducers/plugins/VirtualJetProducer.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937, [RFC] Test clang-tidy --checks modernize-make-unique #29934
- File RecoJets/JetProducers/src/PileUpSubtractor.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15586/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15586/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15586/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15586/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2020-05-21T21:44:50Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-05-21T21:50:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15587

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoHI/HiJetAlgos/src/MultipleAlgoIterator.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935
- File RecoHI/HiJetAlgos/src/ParametrizedSubtractor.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937
- File RecoHI/HiJetAlgos/src/ParticleTowerProducer.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935
- File RecoJets/JetProducers/plugins/VirtualJetProducer.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937, [RFC] Test clang-tidy --checks modernize-make-unique #29934
- File RecoJets/JetProducers/src/PileUpSubtractor.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937

cmsbuild · 2020-05-21T21:51:16Z

A new Pull Request was created by @stepobr (Stepan Obraztsov) for master.

It involves the following packages:

RecoHI/HiJetAlgos
RecoJets/JetProducers

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @jdolen, @ahinzmann, @schoef, @jazzitup, @jdamgov, @yslai, @nhanvtran, @yenjie, @gkasieczka, @clelange, @kurtejung, @mariadalfonso, @mandrenguyen, @dgulhan, @seemasharmafnal, @yetkinyilmaz this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

slava77 · 2020-05-21T22:20:36Z

@cmsbuild please test

cmsbuild · 2020-05-21T22:20:55Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6491/console Started: 2020/05/22 00:22

cmsbuild · 2020-05-21T23:34:18Z

+1
Tested at: 501a13d
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49ee57/6491/summary.html
CMSSW: CMSSW_11_1_X_2020-05-21-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-06-10T14:54:52Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-10T15:02:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29948/15995

This PR adds an extra 104KB to repository
Found files with invalid states:
- RecoHI/HiJetAlgos/plugins/ParticleTowerProducer.h:
  - Added: 59a7068
  - Modified: 52ceb17, 19624dc
  - Deleted: aa74a4f
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoHI/HiJetAlgos/python/HiRecoPFJets_cff.py modified in PR(s): Flow modulated CS subtraction for heavy ions #30057
- File RecoHI/HiJetAlgos/src/MultipleAlgoIterator.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935
- File RecoHI/HiJetAlgos/src/ParametrizedSubtractor.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937
- File RecoHI/HiJetAlgos/src/ParticleTowerProducer.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935
- File RecoJets/JetProducers/plugins/VirtualJetProducer.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-make-unique #29934, [RFC] Test clang-tidy --checks modernize-use-emplace #29937
- File RecoJets/JetProducers/src/PileUpSubtractor.cc modified in PR(s): [RFC] Test clang-tidy --checks modernize-loop-convert #29935, [RFC] Test clang-tidy --checks modernize-use-emplace #29937

cmsbuild · 2020-06-10T15:02:51Z

Pull request #29948 was updated. @perrotta, @cmsbuild, @slava77 can you please check and sign again.

stepobr · 2020-06-10T15:28:20Z

after adding PFTowers to comparisons, we can see that this PR introduced some differences at the edges of acceptance, based on plots from the 2011 HI workflow 140.53
Is this migration to a larger eta expected?

@slava77 Thank you for the checks, the changes are expected because of the switch from calo geometry to a fixed grid. If I compare the etas for the actual tower centers and for the fixed grid the difference is indeed in the last bins.

slava77 · 2020-06-10T15:53:10Z

@cmsbuild please test

cmsbuild · 2020-06-10T15:53:37Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6937/console Started: 2020/06/10 18:13

cmsbuild · 2020-06-10T17:27:02Z

+1
Tested at: e06ba5b
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49ee57/6937/summary.html
CMSSW: CMSSW_11_2_X_2020-06-09-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-06-10T17:27:06Z

Comparison job queued.

cmsbuild · 2020-06-10T19:15:06Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-49ee57/6937/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 14 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2780497
DQMHistoTests: Total failures: 140
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2780307
DQMHistoTests: Total skipped: 50
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
Checked 152 log files, 16 edm output root files, 36 DQM output files

slava77 · 2020-06-12T02:32:41Z

+1

for #29948 e06ba5b

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show differences only in PFTowers (as described in ) and akPu*PFJets (after a change in the MultipleAlgoIterator::calculateOrphanInput algorithm [overloaded with new method]) in HI workflows
e.g in wf 140.56

@stepobr if there is a link to a presentation or another document that describes expected changes in the MultipleAlgoIterator, it would help to link it in the PR description to improve self-documentation

cmsbuild · 2020-06-12T02:33:04Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-06-14T18:10:20Z

+1

stepobr added 6 commits May 21, 2020 16:27

move ParticleTowerProducer to plugins

59a7068

fix pileup subtraction bug

f4d176c

calculateOrphanInput overwritten in plugin

a4a984d

add minimumTowersFraction to fillDescriptions

d632591

quick update to move ParticleTowerProducer to plugins

2083ce9

small fixes

299c48a

cmsbuild added this to the CMSSW_11_1_X milestone May 21, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 21, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels May 21, 2020

fix code and style checks issues

501a13d

cmsbuild added code-checks-pending and removed code-checks-rejected labels May 21, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels May 21, 2020

cmsbuild added tests-started and removed tests-pending labels May 21, 2020

fix code check warnings

e06ba5b

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jun 10, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 10, 2020

cmsbuild added tests-started and removed tests-pending labels Jun 10, 2020

cmsbuild added tests-approved and removed tests-started labels Jun 10, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jun 10, 2020

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Jun 12, 2020

cmsbuild added orp-approved and removed orp-pending labels Jun 14, 2020

cmsbuild merged commit 69ec082 into cms-sw:master Jun 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changes to jet pileup subtraction #29948

Changes to jet pileup subtraction #29948

stepobr commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

slava77 commented May 21, 2020

cmsbuild commented May 21, 2020 •

edited

cmsbuild commented May 21, 2020

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

stepobr commented Jun 10, 2020

slava77 commented Jun 10, 2020

cmsbuild commented Jun 10, 2020 •

edited

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

slava77 commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

silviodonato commented Jun 14, 2020

Changes to jet pileup subtraction #29948

Changes to jet pileup subtraction #29948

Conversation

stepobr commented May 21, 2020

PR description:

PR validation:

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

cmsbuild commented May 21, 2020

slava77 commented May 21, 2020

cmsbuild commented May 21, 2020 • edited

cmsbuild commented May 21, 2020

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

stepobr commented Jun 10, 2020

slava77 commented Jun 10, 2020

cmsbuild commented Jun 10, 2020 • edited

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

cmsbuild commented Jun 10, 2020

slava77 commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

silviodonato commented Jun 14, 2020

cmsbuild commented May 21, 2020 •

edited

cmsbuild commented Jun 10, 2020 •

edited