improving performance of PuppiProducer #31164
Conversation
|
@mrodozov @smuzaffar |
|
@slava77 the remote calls to jenkins hanged with proxy error 502 and I'm trying to restart the service but I don't have sudo rights to do it. I'm trying to do it from the java CLI using the keys for cmsbuild |
|
It seems there are other problems at cern today (at least affecting cms computing infrastructure)
On Aug 18, 2020, at 3:33 PM, Mircho Rodozov <notifications@github.com<mailto:notifications@github.com>> wrote:
@slava77<https://github.com/slava77> the remote calls to jenkins hanged with proxy error 502 and I'm trying to restart the service but I don't have sudo rights to do it
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#31164 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGPFQ2VYBUIGTQZJX5IVY3SBJ7LRANCNFSM4QDNGPKA>.
|
|
@davidlange6 do you by any chance happen to have root permissions for the jenkins03 machine ? |
|
No, I don’t - perhaps ask IT to reboot it?
On Aug 18, 2020, at 4:45 PM, Mircho Rodozov <notifications@github.com<mailto:notifications@github.com>> wrote:
@davidlange6<https://github.com/davidlange6> do you by any chance happen to have root permissions for the jenkins03 machine ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#31164 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGPFQ5R5XJMARLSV773IOLSBKHZ3ANCNFSM4QDNGPKA>.
|
|
The code-checks are being triggered in jenkins. |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31164/17796
|
|
A new Pull Request was created by @missirol (Marino Missiroli) for master. It involves the following packages: CommonTools/PileupAlgos @perrotta, @jpata, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
@cmsbuild please test |
|
The tests are being triggered in jenkins.
|
|
+1 |
|
Comparison job queued. |
|
Comparison is ready Comparison Summary:
|
|
Comparison job queued. |
|
Comparison is ready Comparison Summary:
|
|
+1 |
|
+1
curiously enough, there was no actual overlap in the context lines with #31174 |
cmsbuild
added
fully-signed
reconstruction-approved
and removed
pending-signatures
reconstruction-pending
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2) |
|
+1 |
|
@slava77 , one late question from my side: would it be okay to backport this (except for the last commit) to |
sure, but please limit it to the commit range that does not change the results. |
I'm looking into a backport of this PR to |
Thank you. |
|
Backport to |
PR description:
This PR is an attempt to improve the performance (in terms of execution time) of the
PuppiProducerplugin.No changes in the outputs are expected.
Most of these technical changes are not in the producer itself, but in the
PuppiContainerclass; all changes are related (directly or indirectly) to the functionvar_within_R(the "alpha" variable on which the PUPPI weight is based), which appears to be the most time-consuming part of the algorithm.The table below summarizes a few timing estimates for the
puppimodule in the RECO step for a Run-3 and Phase-2 workflow.The numbers are taken from the
FastTimerService, running on 100 events; they suggest that the overall reduction in timing is approx. 55-60%.Here is a short summary of the changes in the PR (below,
goodVarandvar_within_Rare used interchangeably, since the former function is simply a call to the latter):PuppiCandidateis converted to a simplestruct, removing its inheritance fromfastjet::PseudoJet(looking at the PUPPI implementation as a whole, the usage offastjetdoes not seem necessary); this change accounts for most of the speedup (approx. 35-40% speedup);skip the calculation of
goodVarwhen not needed, e.g. for candidates that are assigned a default weight (0, or 1); this seems to bring roughly a 10-15% improvement in performance.another bit of speedup is obtained by taking one condition (
puppi_id != 3) out ofvar_within_R, and using it just once inPuppiContainer::initializeto fill the collections later used as inputs togoodVar; this leads to adding one more data member (avectorofPuppiCandidates) to thePuppiContainerclass.In this PR, there are two spots where things could certainly be improved (but improving them will lead to small changes in the outputs):
in
PuppiContainer::initialize, an instance offastjet::PseudoJet(here) is used to obtain the 4-momenta of the candidates used by PUPPI; this was the only way I could find to maintain the same numbers as pre-PR (this may be due to the fact that the inputs used forreset_PtYPhiMare floats, and thenfastjetrecomputes and stores the p4 using doubles); one way to simplify this (assuming the differences in the outputs are ultimately not significant) would be to removePuppiCandidatealtogether and simply use theRecoObjclass already in use in thePuppiProducer(maybe, changing the floats inRecoObjto doubles).in
goodVar, theDeltaR2between two given candidates is calculated (at most) twice (here), first with rapidity, then with pseudo-rapidity; this is not modified by this PR (otherwise, the outputs would inevitably change), but maybe it’s something that could be simplified.FYI: @ahinzmann @lathomas @kirschen
PR validation:
Checked that the PUPPI weights of all PF candidates are unchanged for 200 events (100 for a Run-3 workflow, and 100 for a Phase-2 workflow), for both
puppi(used for jets) andpuppiNoLep(used for MET).Standard workflows, i.e.
runTheMatrix.py -l limited -i all --ibeos, ran successfully.if this PR is a backport please specify the original PR and why you need to backport that PR:
N/A