MTD validation update #27607

casarsa · 2019-07-25T11:58:20Z

PR description:

This is an update of the MTD validation package:

the CrossingFrames for the BTL and ETL SimHits have been switched on in order to add the PU SimHits to the SIM monitoring plots;
energy and time resolution histograms have been added for BTL;
an occupancy plot vs hit energy has been added for BTL.

PR validation:

The updated code has been compiled in the release CMSSW_11_0_X_2019-07-24-2300 and tested with the 2026D41 WFs 20434 and 20634 with PU200.

The total size of the MTD monitoring plots is:
476.28 KiB MTD/ETL
237.66 KiB MTD/BTL

WF 20434 report (step 3):

MemoryReport> Peak virtual size 4673.77 Mbytes
Key events increasing vsize:
[5] run: 1 lumi: 1 event: 5 vsize = 4641.77 deltaVsize = 0.0351562 rss = 3226.27 delta = 29.5234
[2] run: 1 lumi: 1 event: 2 vsize = 4641.74 deltaVsize = 0.304688 rss = 3196.74 delta = 1.77734
[7] run: 1 lumi: 1 event: 7 vsize = 4673.77 deltaVsize = 32 rss = 3273.65 delta = 76.9062
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[9] run: 1 lumi: 1 event: 9 vsize = 4673.77 deltaVsize = 0 rss = 3287.06 delta = 13.4141
[8] run: 1 lumi: 1 event: 8 vsize = 4673.77 deltaVsize = 0 rss = 3287.99 delta = 14.3438
[7] run: 1 lumi: 1 event: 7 vsize = 4673.77 deltaVsize = 32 rss = 3273.65 delta = 47.3828
TimeReport> Time report complete in 506.789 seconds
Time Summary:

Min event: 1.74964
Max event: 12.5869
Avg event: 3.23466
Total loop: 385.479
Total init: 111.365
Total job: 506.789
EventSetup Lock: 0.000208378
EventSetup Get: 84.7993
Event Throughput: 0.259417 ev/s
CPU Summary:
Total loop: 367.605
Total init: 143.27
Total extra: 0
Total job: 520.702
Processing Summary:
Number of Events: 100
Number of Global Begin Lumi Calls: 1
Number of Global Begin Run Calls: 1

WF 20634 report (step 3):

MemoryReport> Peak virtual size 9869.82 Mbytes
Key events increasing vsize:
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[1] run: 1 lumi: 1 event: 1 vsize = 8169.59 deltaVsize = 0 rss = 6721.41 delta = 0
[2] run: 1 lumi: 1 event: 2 vsize = 9833.69 deltaVsize = 1664.11 rss = 6793.38 delta = 71.9688
[7] run: 1 lumi: 1 event: 7 vsize = 9865.82 deltaVsize = 32 rss = 6828.83 delta = 35.4531
[10] run: 1 lumi: 1 event: 10 vsize = 9869.82 deltaVsize = 4 rss = 6832.15 delta = 3.31641
[8] run: 1 lumi: 1 event: 8 vsize = 9865.82 deltaVsize = 0 rss = 6842.48 delta = 13.6445
[7] run: 1 lumi: 1 event: 7 vsize = 9865.82 deltaVsize = 32 rss = 6828.83 delta = -288.48
[10] run: 1 lumi: 1 event: 10 vsize = 9869.82 deltaVsize = 4 rss = 6832.15 delta = 3.31641
TimeReport> Time report complete in 2705.11 seconds
Time Summary:

Min event: 137.084
Max event: 372.577
Avg event: 236.689
Total loop: 2488.53
Total init: 212.354
Total job: 2705.11
EventSetup Lock: 0.000253677
EventSetup Get: 106.802
Event Throughput: 0.00401843 ev/s
CPU Summary:
Total loop: 2186.06
Total init: 115.821
Total extra: 0
Total job: 2306
Processing Summary:
Number of Events: 10
Number of Global Begin Lumi Calls: 1
Number of Global Begin Run Calls: 1

…tograms of BTL energy and time resolution, and BTL occupancy vs hit energy

cmsbuild · 2019-07-25T11:58:52Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-07-25T12:05:06Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27607/11047

This PR adds an extra 24KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27607/11047/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27607/11047/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2019-07-25T12:18:37Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-07-25T12:24:27Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27607/11048

This PR adds an extra 24KB to repository

cmsbuild · 2019-07-25T12:24:57Z

A new Pull Request was created by @casarsa (Massimo Casarsa) for master.

It involves the following packages:

SimGeneral/MixingModule
Validation/MtdValidation

@andrius-k, @kmaeshima, @schneiml, @civanch, @mdhildreth, @cmsbuild, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@makortel this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

civanch · 2019-07-25T16:16:46Z

please test

cmsbuild · 2019-07-25T16:17:11Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1641/console Started: 2019/07/25 18:18

cmsbuild · 2019-07-25T18:07:19Z

+1
Tested at: fe5a34f
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-946727/1641/summary.html

cmsbuild · 2019-07-25T18:07:22Z

Comparison job queued.

cmsbuild · 2019-07-25T20:05:33Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-946727/1641/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 2628546
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2628227
DQMHistoTests: Total skipped: 317
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 18.258 KiB( 31 files compared)
DQMHistoSizes: changed ( 20034.0,... ): 6.086 KiB MTD/BTL
Checked 133 log files, 14 edm output root files, 32 DQM output files

jfernan2 · 2019-07-25T20:24:04Z

@casarsa I do see plots (6) in BTL but not in ETL like you said in the description. This is happening in the 3 Phase2 workflows. Is this expected?
E.g. https://tinyurl.com/y5po36vr

casarsa · 2019-07-26T07:10:51Z

@jfernan2 For the time being those checks have been implemented only for BTL.

jfernan2 · 2019-07-26T08:51:33Z

@casarsa OK, I was confused by your statement:
"The total size of the MTD monitoring plots is:
476.28 KiB MTD/ETL "

jfernan2 · 2019-07-26T08:52:47Z

+1

civanch · 2019-07-27T16:02:00Z

+1

cmsbuild · 2019-07-27T16:02:27Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-07-27T20:55:45Z

Validation/MtdValidation/plugins/BuildFile.xml

@@ -4,4 +4,5 @@
 <use name="Geometry/Records"/>
 <use name="Geometry/MTDGeometryBuilder"/>
 <use name="DQMServices/Core"/>
+<use name="hepmc"/>


@casarsa why this dependence?

That surprised me too. It seems it's required by SimDataFormats/CrossingFrame/interface/CrossingFrame.h, that includes HepMCProduct.h.

By quick look it looks like the include of HepMCProduct.h could easily be removed from SimDataFormats/CrossingFrame/interface/CrossingFrame.h.

Regardless of the possible cleanup of SimDataFormats/CrossingFrame/interface/CrossingFrame.h, the proper dependence to be added here is SimDataFormats/CrossingFrame. That should make the explicit dependence in hepmc uneecessary (as SimDataFormats/CrossingFrame depends on that).

@makortel let's open a separate issue for that

fabiocos · 2019-07-27T20:57:02Z

Validation/MtdValidation/plugins/BtlRecHitsValidation.cc

+  MixCollection<PSimHit> btlSimHits(btlSimHitsHandle.product());
+
+  // --- Loop over the BLT SIM hits
+  std::unordered_map<uint32_t, MTDHit> m_btlSimHits;


@casarsa this energy accumulation is equivalent to that used in the digi - rechit construction, am I correct?

Yes, you are. In both cases I'm starting from the SimHits and am accumulating the energy released in the same BTL crystal.

kpedro88 · 2019-07-29T17:03:45Z

+1

casarsa added 2 commits July 25, 2019 11:19

Switched on the CrossingFrames for BTL and ETL SimHits

cd7ce54

Added MixingModule SimHits to the BTL/ETL monitoring plots; added his…

db3a5c8

…tograms of BTL energy and time resolution, and BTL occupancy vs hit energy

cmsbuild added this to the CMSSW_11_0_X milestone Jul 25, 2019

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures simulation-pending tests-pending labels Jul 25, 2019

cmsbuild added code-checks-rejected and removed code-checks-pending labels Jul 25, 2019

Applied code-format patch

fe5a34f

cmsbuild added code-checks-pending and removed code-checks-rejected labels Jul 25, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 25, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 25, 2019

cmsbuild added tests-approved and removed tests-started labels Jul 25, 2019

cmsbuild added comparison-available and removed comparison-pending labels Jul 25, 2019

cmsbuild added dqm-approved and removed dqm-pending labels Jul 26, 2019

cmsbuild added fully-signed simulation-approved and removed pending-signatures simulation-pending labels Jul 27, 2019

fabiocos reviewed Jul 27, 2019

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Jul 29, 2019

cmsbuild merged commit 0bbb845 into cms-sw:master Jul 29, 2019

casarsa deleted the mc-valMTD-update branch August 21, 2019 07:43

casarsa mentioned this pull request Aug 28, 2019

BTL validation update #27891

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MTD validation update #27607

MTD validation update #27607

casarsa commented Jul 25, 2019 •

edited

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

civanch commented Jul 25, 2019

cmsbuild commented Jul 25, 2019 •

edited

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

jfernan2 commented Jul 25, 2019

casarsa commented Jul 26, 2019

jfernan2 commented Jul 26, 2019

jfernan2 commented Jul 26, 2019

civanch commented Jul 27, 2019

cmsbuild commented Jul 27, 2019

fabiocos Jul 27, 2019

casarsa Jul 28, 2019

makortel Jul 29, 2019

makortel Jul 29, 2019

kpedro88 Jul 29, 2019

fabiocos Jul 27, 2019

casarsa Jul 28, 2019

kpedro88 commented Jul 29, 2019

MTD validation update #27607

MTD validation update #27607

Conversation

casarsa commented Jul 25, 2019 • edited

PR description:

PR validation:

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

civanch commented Jul 25, 2019

cmsbuild commented Jul 25, 2019 • edited

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

cmsbuild commented Jul 25, 2019

jfernan2 commented Jul 25, 2019

casarsa commented Jul 26, 2019

jfernan2 commented Jul 26, 2019

jfernan2 commented Jul 26, 2019

civanch commented Jul 27, 2019

cmsbuild commented Jul 27, 2019

fabiocos Jul 27, 2019

Choose a reason for hiding this comment

casarsa Jul 28, 2019

Choose a reason for hiding this comment

makortel Jul 29, 2019

Choose a reason for hiding this comment

makortel Jul 29, 2019

Choose a reason for hiding this comment

kpedro88 Jul 29, 2019

Choose a reason for hiding this comment

fabiocos Jul 27, 2019

Choose a reason for hiding this comment

casarsa Jul 28, 2019

Choose a reason for hiding this comment

kpedro88 commented Jul 29, 2019

casarsa commented Jul 25, 2019 •

edited

cmsbuild commented Jul 25, 2019 •

edited