Run3-hcx300 Update code for new shower library (Lev & Salavat) #34432
Conversation
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34432/23848
Code check has found code style and quality issues which could be resolved by applying following patch(s)
|
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34432/23849
|
|
A new Pull Request was created by @bsunanda (Sunanda Banerjee) for master. It involves the following packages:
@civanch, @Dr15Jones, @makortel, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @silviodonato, @qliphy, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
@cmsbuild Please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9a3f37/16669/summary.html Comparison SummarySummary:
|
|
+1 |
|
@civanch Please approve |
|
+1 |
|
+1 |
cmsbuild
added
fully-signed
operations-approved
orp-approved
and removed
operations-pending
orp-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged. |
|
belatedly - has someone assessed the performance impact of this new library in the context of the parameters with which the ROOT file was created? |
|
@davidlange6 |
|
Its not really in the shadow is it? Maybe someone has a recent profile
… On Jul 14, 2021, at 6:00 PM, Salavat Abdullin ***@***.***> wrote:
@davidlange6
No. Not yet (for not yet baseline new HF ShowerLibrary).
Well, assuming HF ShowerLibrary performance is in the shadow of real GEANT workflow for the (bulk) of CMS detector.
But expert's help would be appreciated. I recall your optimization done 5 years ago
#16049
Not sure if it still holds (important) nowadays? - I don't remember what was CPU gain back then...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
@davidlange6 , recently we did profile with Run2 shower library in 12_0_0_pre1 and it was not contributing to CPU in a significant way. We should repeat the exercise when this new SL will be integrated. |
|
Anyway, in what way is the new library compressed?
… On Jul 14, 2021, at 8:02 PM, Vladimir Ivantchenko ***@***.***> wrote:
@davidlange6 , recently we did profile with Run2 shower library in 12_0_0_pre1 and it was not contributing to CPU in a significant way. We should repeat the exercise when this new SL will be integrated.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
@davidlange6 David, the situation is as follows: (1) you're right, there seems to be a sizable impact of using new SL (HF Shower Library) on SIM performance: (2) relevant files (their slightly updated versions) used for merging many "FillSim" (Full SImulation of Cherenkov light) components for each particle spice (e,pi) and energy value into the new SL, alas, were not committed to CMSSW (unlike all other codes related to new SL) and were used privately by Lev Kheyn, please find them in his public
NB: new SL is not compressed or optimized in any way, so parameters of root files are default ones (whatever they are). [*] (1) regular/default SL : (2) new SL: |
|
My guess would be that you want
1 a faster compression algorithm (eg, reduce 9 to 4 or perhaps use zstandard).
2 smaller baskets as this file basically gets read as random access.
3 its not obvious that you want this file to be an EDM file.
… On Jul 15, 2021, at 1:05 PM, Salavat Abdullin ***@***.***> wrote:
@davidlange6 David, the situation is as follows:
(1) you're right, there seems to be sizable impact of using new SL (HF SHowerLibrary) on SIM performnce:
~25% slow down of GENSIM step for 11634.0 wf [*];
(2) files used for writing components (electrons, pions for given energies) of new SL and merging them in final SL,
alas, were not committed to CMSSW (unlike all other codes related to new SL) and wer used in private by Lev Kheyn, please find them in his public
/afs/cern.ch/user/k/kheyn/public/For_Sunanda -
(a) HFShowerLibraryAnalyzer.cc - code reading "FullSim" (Full SImulation of Cherenkov light) root files and creating library
(b) writelibraryfile_cfg.py - config
(c) fileList.txt - used list of FullSim files
We'd appreciate your expert's insight - whether something can be done to improve the situation (1)...
[*] Quick and (very) dirty comparison of CPU performance by running simultaneously (in parallel) two jobs w/wo new SL for wf 11634.0 (noPU TTbar 14 TeV 2021, 100 ev step1=GENSIM) on the same lxplus PC, numbers coming out of cms.Service("Timing")
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
@davidlange6 |
|
I’ll let the experts have a look . At least the first two can be found by reading documentation found at
edmPluginHelp -p PoolOutputModule.
… On Jul 15, 2021, at 4:06 PM, Salavat Abdullin ***@***.***> wrote:
@davidlange6
could you, please, convert/translate your consideration into a prescription/recipe
for aforementioned (2) files in /afs/cern.ch/user/k/kheyn/public/For_Sunanda/ ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
@davidlange6 I've published (somewhat modified version of) both code and job config for assembling SL from 32 input components in #34561 and produced yet another version of LS, after many attempts to improve the performance of new SL(s) in TTbar 2021 MC SIM step. Finally the best updated new SL has been found/produced with parameters buffersize=1 and splitlevel=2 for Tree->Branches. Unfortunately we have little idea what did you do 5 years ago (2016), when you've somehow reformatted initial version of HFShowerLibrary_npmt_noatt_eta4_16en_xxx.root (produced by Lev back in 2015) so that it supposedly became faster (no numbers available though...) ? [1] [2] |
|
@davidlange6 -> |
|
Update: Lev has reported (using single-particle gun, shot at HF) that the latest/biggest SL is the best (as evaluated on OscarMTProducer TimeService fragments collected over 100 ev) among three new tried SL's, ~60-70 times faster (per single particle) than aforementioned cmssdt-residing [1] one, but still not at the level of the actual default one... |
|
These sound like the right sort of changes. I understand the new library is 3x bigger than the run 2 library - naturally that will be slower to access (as the code jumps around in the library). Is it already single precision? Beyond that, you might compare the split level of the old file (TTree::splitlevel())
… On Jul 22, 2021, at 8:07 AM, Salavat Abdullin ***@***.***> wrote:
@davidlange6
Hi David, sorry to bug you once again, unfortunately we were misled by the files put by Lev in a rush to the aformentioned public directory, who has produced/assembled new SL [1].
Actually he didn't use PoolOutputModule, but TFileService...
I've published (somewhat modified version of) both code and job config for assembling SL from 32 input components in #34561 and produced yet another version, after many attempts to improve the performance of new SL(s) in TTbar 2021 MC SIM step.
NB: new SL contains twice more number of showers per each energy point, so its size is inevitably bigger than for actual default ().
Finally the best updated new SL has been found/produced with parameters buffersize=1 and splitlevel=2 for Tree->Branches.
This modified/updated version of new SL is ~5% slower for TTbar 2021 MC SIM wrt actual default SL, but not ~25% anymore as for the intial version produced by Lev with ROOT defaults for Tree->Branch buffersize=32000 and splitlevel=99.
Do you see any further optimization possible?
Unfortunately we have little idea what did you do 5 years ago (2016), when you've somehow reformatted initial version of HFShowerLibrary_npmt_noatt_eta4_16en_v4.root (produced by Lev back in 2015) so that it became smaller (~500 MB -> 330 MB) and probably faster (no numbers available though...) ?
[1]
HFShowerLibrary_run3_v5.root, put by Shahzad into https://cmssdt.cern.ch/SDT/data/CMSSW/SimG4CMS/Calo/data/
[2]
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Finally Lev has re-done his OscarMTProducer CPU time eveluations with single 100 GeV pions and it seems that the "latest-greatest" (even thought the biggest...) version of new SL is the best of all compared:
Left (best) to right (worst): (1) green : new SL with added (wrt (3) below) LZ4 compression algo and compression level=4 (2) black: actual default HFShowerLibrary_npmt_noatt_eta4_16en_v4.root (3) blue: my intermediate attempt to re-assemble new SL only using buffersize=1 and splitlevel=2 for Tree->Branches (4) red: HFShowerLibrary_run3_v5(_210713).root - the initial version of new SL committed to CMSSDT |
PR description:
Update code for new shower library (Lev & Salavat), Uses a modifier run3_HFSL to activate the new library. This can be activated only when the corresponding file is saved in the data repository
PR validation:
Use the runTheMatrix test workflow and a new script in SimG4CMS/Calo/test/python/runHF6_cfg.py
if this PR is a backport please specify the original PR and why you need to backport that PR:
Nothing special