Diamond sampic dqm

[ChrisMisan]

Overview:
Goal of this PR is to deliver a new DQM module for the diamond sampic flow. Said module contains offline and online dqm plots which are switched using configuration parameter.
Existing pixel dqm was serving as a reference for online/offline switch:
https://github.com/cms-sw/cmssw/blob/master/DQM/CTPPS/plugins/CTPPSPixelDQMSource.cc

Commented plots in DQM module might be re-activated in the future and therefore were not deleted.

Testing
Due to lack of data (see #34759) automatic testing of this code is not possible. Manual test is available at RecoPPS/test directory.

Memory performance
This is memory consumption for cmsRun process running the DQM module.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35445/25593

• This PR adds an extra 24KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35445/25593/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35445/25593/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35445/25595

• This PR adds an extra 28KB to repository

[cmsbuild]

A new Pull Request was created by @ChrisMisan (Christopher) for master.

It involves the following packages:

• DQM/CTPPS (dqm)
• RecoPPS/Configuration (reconstruction)
• RecoPPS/Local (reconstruction)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @slava77, @jpata, @pmandrik, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@fabferro, @jan-kaspar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[jfernan2]

Hi @ChrisMisan
In order to keep track of the DQM developers, can you please add yourself along with your github username to the appropiated e-group here:
https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMContacts#CT_PPS
?
Thanks in advance

[jfernan2]

please test

[cmsbuild]

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3cf7ab/19208/summary.html
COMMIT: f64638f
CMSSW: CMSSW_12_1_X_2021-09-28-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/35445/19208/install.sh to create a dev area with all the needed externals and cmssw changes.

CMS StaticAnalyzer warnings: There are 3 EventSetupRecord::get warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3cf7ab/19208/llvm-analysis/esrget-sa.txt for details.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOfflineConfiguration_150 had ERRORS
---> test TestDQMOfflineConfiguration_170 had ERRORS
---> test TestDQMOfflineConfiguration_100 had ERRORS
---> test TestDQMOfflineConfiguration_160 had ERRORS
and more ...

RelVals

• 136.8523136.8523_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2/step2_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2.log
• 136.7611136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log
• 136.8311136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log

Expand to see more relval errors ...

• 136.88811
• 1325.81
• 158.01
• 140.56
• 4.53
• 136.731
• 136.793
• 136.874
• 135.4
• 1000.0
• 1001.0
• 5.1
• 1306.0
• 312.0
• 1330.0
• 9.0
• 25.0
• 10042.0
• 10024.0
• 11634.0
• 11634.7
• 10824.0
• 25202.0
• 12434.0
• 11634.912
• 11634.911
• 10224.0
• 23234.0
• 34634.0
• 28234.0
• 250202.181
• 34834.999

AddOn Tests

• fastsim2

----- Begin Fatal Exception 28-Sep-2021 19:48:30 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 1
   [1] Running path 'AODSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'AODSIMoutput'
   [3] Calling method for module TotemTimingRecHitProducer/'totemTimingRecHits'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edm::DetSetVector<TotemTimingDigi>
Looking for module label: totemTimingRawToDigi
Looking for productInstanceName: TotemTiming

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-35445/25851

• This PR adds an extra 96KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DQM/CTPPS/python/ctppsDQM_cff.py modified in PR(s): Diamond dqm #35454

[cmsbuild]

Pull request #35445 was updated. @perrotta, @pmandrik, @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @slava77, @jpata, @qliphy, @rvenditti, @fabiocos, @pbo0, @davidlange6 can you please check and sign again.

[jpata]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3cf7ab/19514/summary.html
COMMIT: 12147e0
CMSSW: CMSSW_12_1_X_2021-10-08-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35445/19514/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 22 differences found in the comparisons
• DQMHistoTests: Total files compared: 40
• DQMHistoTests: Total histograms compared: 2770950
• DQMHistoTests: Total failures: 5
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 2770922
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -10588.948 KiB( 39 files compared)
• DQMHistoSizes: changed ( 10024.0,... ): -3151.653 KiB CTPPS/TimingFastSilicon
• DQMHistoSizes: changed ( 10024.0,... ): 2360.820 KiB CTPPS/DiamondSampic
• DQMHistoSizes: changed ( 11634.0,... ): 2441.273 KiB CTPPS/DiamondSampic
• DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
• Checked 169 log files, 37 edm output root files, 40 DQM output files
• TriggerResults: no differences found

[jfernan2]

+1

[jpata]

+reconstruction

• totemTiming replaced with diamondSampic in recoCTPPSTask, corresponding keep statement added
• in reco, we see the totemTimingLocalTracks collection disappear, and diamondSampicLocalTracks appear
• additional workflows for ctpps 136.8562, 136.8561 were also checked and run successfully

[perrotta]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.