Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up HI miniAOD by removing unneeded features #31893

Merged
merged 3 commits into from Nov 5, 2020

Conversation

mandrenguyen
Copy link
Contributor

@mandrenguyen mandrenguyen commented Oct 22, 2020

PR description:

Turns off several unused features in miniAOD:

  • Disable pfPileUpIsoPFBRECO
  • Remove algos from puppi, making puppi do nearly nothing
  • no longer compute ghost tracks for impactParameterTagInfos
  • Turn off area calculation for fat jet collection used for taus
    CPU savings is around 10%

I also cleaned up pp_on_AA era statements in applySubstructure, which no longer do anything since applySubstructure is completely inactive for these eras

PR validation:

tested on 140.5611 and 300

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31893/19298

  • This PR adds an extra 24KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mandrenguyen (Matthew Nguyen) for master.

It involves the following packages:

CommonTools/ParticleFlow
CommonTools/PileupAlgos
PhysicsTools/PatAlgos
RecoBTag/ImpactParameter
RecoTauTag/Configuration

@perrotta, @jpata, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @jdolen, @ahinzmann, @smoortga, @riga, @schoef, @emilbols, @jdamgov, @JyothsnaKomaragiri, @nhanvtran, @gkasieczka, @clelange, @cbernet, @hatakeyamak, @ferencek, @gpetruc, @andrzejnovak, @mariadalfonso, @seemasharmafnal, @mmarionncern this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 22, 2020

The tests are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+1
Tested at: 6424979
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-290783/10199/summary.html
CMSSW: CMSSW_11_2_X_2020-10-21-2300
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-290783/10199/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 35
  • DQMHistoTests: Total histograms compared: 2544110
  • DQMHistoTests: Total failures: 7
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2544081
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
  • Checked 149 log files, 22 edm output root files, 35 DQM output files

@slava77
Copy link
Contributor

slava77 commented Oct 22, 2020

this seems related to #31647; just to tag it here

@santocch
Copy link

+1

Copy link
Contributor

@slava77 slava77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly technical comments
the one for CommonTools/ParticleFlow/python/PFBRECO_cff.py is somewhat more significant than the rest

CommonTools/ParticleFlow/python/PFBRECO_cff.py Outdated Show resolved Hide resolved
RecoTauTag/Configuration/python/boostedHPSPFTaus_cff.py Outdated Show resolved Hide resolved
@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31893/19571

  • This PR adds an extra 24KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

Pull request #31893 was updated. @perrotta, @jpata, @cmsbuild, @santocch, @slava77 can you please check and sign again.

@slava77
Copy link
Contributor

slava77 commented Nov 4, 2020

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

The tests are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

+1
Tested at: 8c48ced
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-290783/10509/summary.html
CMSSW: CMSSW_11_2_X_2020-11-04-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2020

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-290783/10509/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 1 differences found in the comparisons
  • DQMHistoTests: Total files compared: 35
  • DQMHistoTests: Total histograms compared: 2544092
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2544069
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
  • Checked 149 log files, 22 edm output root files, 35 DQM output files

@slava77
Copy link
Contributor

slava77 commented Nov 4, 2020

+1

for #31893 8c48ced

  • after the rebase the updates which were previously applied to pp_on_AA_2018 | pp_on_PbPb_run3 now consistently apply to pp_on_AA, as expected from the refactoring of modifiers introduced by Common pp_on_AA modifier for 2018 and run 3 AA eras #31935
  • results of jenkins tests are the same as in the previous version

@santocch
Copy link

santocch commented Nov 5, 2020

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 5, 2020

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@silviodonato
Copy link
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants