MTD simulation: premixing tuning #29039

casarsa · 2020-02-25T15:47:26Z

PR description:

The packing of the PU SimHits information in the premixing library has been tuned to fix a bias in the MTD time resolution when running with premixing. Basically, the width of the word, in which the time and energy information is stored, has been increased from 10 to 16 bits.

The figure below shows a comparison of the BTL resolution between the standard PU (WF 22234.0) and the premixed PU (WF 22234.99) for the current code and the proposed changes.

PR validation:

PU library size:

current:
PMTDSimAccumulator_mix_FTLBarrel_DIGI. 3.46034e+06 2.89781e+06
PMTDSimAccumulator_mix_FTLEndcap_DIGI. 1.33435e+06 788234

new:
PMTDSimAccumulator_mix_FTLBarrel_DIGI. 4.66987e+06 2.82419e+06
PMTDSimAccumulator_mix_FTLEndcap_DIGI. 1.57796e+06 902263

Performance:

The new code affects the steps 2 and 3, it has been tested with WF 22234.99 in CMSSW_11_1_X_2020-02-24-2300.

step 2

MemoryReport> Peak virtual size 4503.98 Mbytes
Key events increasing vsize:
[1] run: 1 lumi: 1 event: 1 vsize = 4119.94 deltaVsize = 0 rss = 2456.23 delta = 0
[2] run: 1 lumi: 1 event: 2 vsize = 4119.95 deltaVsize = 0.015625 rss = 2674.35 delta = 218.117
[4] run: 1 lumi: 1 event: 4 vsize = 4503.96 deltaVsize = 384.008 rss = 2787.09 delta = 112.746
[6] run: 1 lumi: 1 event: 6 vsize = 4503.98 deltaVsize = 0.0195312 rss = 2787.36 delta = 0.265625
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[8] run: 1 lumi: 1 event: 8 vsize = 4503.98 deltaVsize = 0 rss = 2860.68 delta = 73.3164
[7] run: 1 lumi: 1 event: 7 vsize = 4503.98 deltaVsize = 0 rss = 2831.74 delta = 44.3789
[6] run: 1 lumi: 1 event: 6 vsize = 4503.98 deltaVsize = 0.0195312 rss = 2787.36 delta = 0.265625
TimeReport> Time report complete in 844.182 seconds
Time Summary:

Min event: 61.3985
Max event: 116.356
Avg event: 74.3752
Total loop: 823.047
Total init: 21.0407
Total job: 844.182
EventSetup Lock: 7.72476e-05
EventSetup Get: 79.1781
Event Throughput: 0.01215 ev/s
CPU Summary:
Total loop: 685.092
Total init: 9.92849
Total extra: 0
Total job: 695.103
Processing Summary:
Number of Events: 10
Number of Global Begin Lumi Calls: 1
Number of Global Begin Run Calls: 1

step 3

MemoryReport> Peak virtual size 4815.11 Mbytes
Key events increasing vsize:
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[1] run: 1 lumi: 1 event: 1 vsize = 4035.09 deltaVsize = 0 rss = 2976.07 delta = 0
[2] run: 1 lumi: 1 event: 2 vsize = 4431.09 deltaVsize = 396 rss = 3095.2 delta = 119.129
[3] run: 1 lumi: 1 event: 3 vsize = 4815.11 deltaVsize = 384.02 rss = 3062.52 delta = -32.6836
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[5] run: 1 lumi: 1 event: 5 vsize = 4815.11 deltaVsize = 0 rss = 3238.1 delta = 175.582
[4] run: 1 lumi: 1 event: 4 vsize = 4815.11 deltaVsize = 0 rss = 2982.25 delta = -80.2656
[3] run: 1 lumi: 1 event: 3 vsize = 4815.11 deltaVsize = 384.02 rss = 3062.52 delta = -32.6836
TimeReport> Time report complete in 655.984 seconds
Time Summary:

Min event: 40.0881
Max event: 145.567
Avg event: 58.967
Total loop: 636.154
Total init: 19.2085
Total job: 655.984
EventSetup Lock: 8.89301e-05
EventSetup Get: 78.7761
Event Throughput: 0.0157195 ev/s
CPU Summary:
Total loop: 582.007
Total init: 12.7771
Total extra: 0
Total job: 594.932
Processing Summary:
Number of Events: 10
Number of Global Begin Lumi Calls: 1
Number of Global Begin Run Calls: 1

cmsbuild · 2020-02-25T15:47:53Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-02-25T15:54:47Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29039/13905

This PR adds an extra 20KB to repository

cmsbuild · 2020-02-25T15:55:16Z

A new Pull Request was created by @casarsa (Massimo Casarsa) for master.

It involves the following packages:

DataFormats/FTLDigi
SimFastTiming/FastTimingCommon

@cmsbuild, @civanch, @kpedro88, @mdhildreth can you please review it and eventually sign? Thanks.
@rovere this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

kpedro88 · 2020-02-25T16:00:41Z

test parameters

workflows = 20661.97,20634.99,23434.1001

kpedro88 · 2020-02-25T16:01:08Z

please test

cmsbuild · 2020-02-25T16:01:21Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4873/console Started: 2020/02/25 17:09

cmsbuild · 2020-02-25T17:17:44Z

+1
Tested at: 2358a53
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-de2d39/4873/summary.html
CMSSW: CMSSW_11_1_X_2020-02-25-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-02-25T17:17:54Z

Comparison job queued.

cmsbuild · 2020-02-25T18:43:21Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-de2d39/4873/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2695371
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2695051
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

civanch · 2020-02-26T10:55:53Z

+1

silviodonato · 2020-02-26T15:55:12Z

abort

cmsbuild · 2020-02-26T15:55:40Z

Jenkins tests are aborted.

silviodonato · 2020-02-26T15:59:48Z

please test workflow 20661.97,20634.99,23434.1001,23434.1001
@kpedro88 I cannot find the results of the wf you launched (20661.97,20634.99,23434.1001)
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-de2d39/4873/runTheMatrix-results/

cmsbuild · 2020-02-26T16:01:20Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 23434.1001,20634.99,20661.97
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4894/console Started: 2020/02/26 17:02

silviodonato · 2020-02-26T17:04:59Z

@casarsa , as you might know, we need to keep a compatibility between 11_0_0 and 11_1_X in order to re-run the reconstruction starting from the GEN-SIM-DIGI-RAW produced in 11_0_0_patch1 for the HLT TDR.
Do you think this PR can break this compatibility? In other words, if we merge this PR and we read the MTD digi produced in 11_0_0_patch1, should we get meaningful values?

silviodonato · 2020-02-26T17:13:21Z

FYI: @fwyzard @thiago: you might be interested in this issue with MTD and premixing (not only for the possible compatibility issue)

casarsa · 2020-02-26T17:14:15Z

@silviodonato This PR changes the format of the premixing library. The MTD digis are not affected.

fwyzard · 2020-02-26T17:55:44Z

@silviodonato thanks for the heads up; according to @casarsa this should be fine.

Premixing will be good to have in the long run, unfortunately not for the HLT TDR !

cmsbuild · 2020-02-26T18:07:04Z

+1
Tested at: 2358a53
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-de2d39/4894/summary.html
CMSSW: CMSSW_11_1_X_2020-02-25-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-02-26T18:07:07Z

Comparison job queued.

cmsbuild · 2020-02-26T19:39:16Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-de2d39/4894/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/20634.99_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D41PU_GenSimHLBeamSpotFull14+PREMIX_2026D41PU_PremixHLBeamSpotFull14PU_Premix+DigiFullTriggerPUPRMXCombined_2026D41PU+RecoFullGlobalPUPRMX_2026D41PU+HARVESTFullGlobalPU_2026D41PU
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/20661.97_PREMIXUP26D41_PU25+PREMIX_2026D41PU_PremixHLBeamSpotFullPU_Premix
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/23234.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D49_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D49+RecoFullGlobal_2026D49+HARVESTFullGlobal_2026D49
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-de2d39/23434.1001_TTbar_14TeV+RecoFullGlobalPU_TestOldDigi_2026D49PU+HARVESTFullGlobalPU_2026D49PU

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2679706
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2679385
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

silviodonato · 2020-02-27T08:15:00Z

+1

casarsa added 2 commits February 19, 2020 15:06

Tuning of the MTD premixing configuration

7c97972

Data field extended to 16 bits

2358a53

cmsbuild added this to the CMSSW_11_1_X milestone Feb 25, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures simulation-pending tests-pending upgrade-pending labels Feb 25, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 25, 2020

cmsbuild added tests-started and removed tests-pending labels Feb 25, 2020

cmsbuild added tests-approved and removed tests-started labels Feb 25, 2020

cmsbuild added comparison-available and removed comparison-pending labels Feb 25, 2020

cmsbuild added simulation-approved and removed simulation-pending labels Feb 26, 2020

cmsbuild added comparison-pending tests-pending tests-started and removed comparison-available tests-approved tests-pending labels Feb 26, 2020

cmsbuild added tests-pending and removed tests-started labels Feb 26, 2020

cmsbuild added tests-started and removed tests-pending labels Feb 26, 2020

cmsbuild added tests-approved and removed tests-started labels Feb 26, 2020

cmsbuild added comparison-available and removed comparison-pending labels Feb 26, 2020

cmsbuild added orp-approved and removed orp-pending labels Feb 27, 2020

cmsbuild merged commit b09f9d0 into cms-sw:master Feb 27, 2020

casarsa deleted the mc-simMTD-premixTuning branch March 2, 2020 10:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MTD simulation: premixing tuning #29039

MTD simulation: premixing tuning #29039

casarsa commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

kpedro88 commented Feb 25, 2020 •

edited

kpedro88 commented Feb 25, 2020

cmsbuild commented Feb 25, 2020 •

edited

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

civanch commented Feb 26, 2020

silviodonato commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

silviodonato commented Feb 26, 2020

cmsbuild commented Feb 26, 2020 •

edited

silviodonato commented Feb 26, 2020

silviodonato commented Feb 26, 2020

casarsa commented Feb 26, 2020

fwyzard commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

silviodonato commented Feb 27, 2020

MTD simulation: premixing tuning #29039

MTD simulation: premixing tuning #29039

Conversation

casarsa commented Feb 25, 2020

PR description:

PR validation:

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

kpedro88 commented Feb 25, 2020 • edited

kpedro88 commented Feb 25, 2020

cmsbuild commented Feb 25, 2020 • edited

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

civanch commented Feb 26, 2020

silviodonato commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

silviodonato commented Feb 26, 2020

cmsbuild commented Feb 26, 2020 • edited

silviodonato commented Feb 26, 2020

silviodonato commented Feb 26, 2020

casarsa commented Feb 26, 2020

fwyzard commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

silviodonato commented Feb 27, 2020

kpedro88 commented Feb 25, 2020 •

edited

cmsbuild commented Feb 25, 2020 •

edited

cmsbuild commented Feb 26, 2020 •

edited