Omit calibration channels and invalid data in HCAL TP emulation

[christopheralanwest]

This PR excludes two types of data from TP emulation:

• Calibration channels in HBHE and HF
• Invalid QIE10 data, which should not be considered valid for the purpose of TP emulation

[cmsbuild]

A new Pull Request was created by @christopheralanwest for master.

It involves the following packages:

SimCalorimetry/HcalTrigPrimAlgos

@cmsbuild, @rekovic, @mulhearn, @davidlange6 can you please review it and eventually sign? Thanks.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

[kpedro88]

For future-proofing, it would be better to let this function accept HcalBarrel DetIds as well.

[christopheralanwest]

Thanks, good idea. I've made the change.

[kpedro88]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/20380/console Started: 2017/06/06 23:39
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/20381/console Started: 2017/06/07 03:38

[cmsbuild]

+1
Tested at: 78e17e3
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19105/20381/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19105/20381/summary.html

Comparison Summary:

• You potentially added 162 lines to the logs
• Reco comparison results: 1717 differences found in the comparisons
• DQMHistoTests: Total files compared: 23
• DQMHistoTests: Total histograms compared: 1837639
• DQMHistoTests: Total failures: 42716
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 1794750
• DQMHistoTests: Total skipped: 173
• DQMHistoTests: Total Missing objects: 0
• Checked 94 log files, 14 edm output root files, 23 DQM output files

[kpedro88]

@rekovic please review/sign

[christopheralanwest]

I've included some more details about this PR below.

Without this fix, the HCAL trigger primitive emulation crashes when run on data in which HCAL calibration channels have been read out. To reproduce a crash, use the following file from run 294745 in which the calibration channels are read out:

cmsrel CMSSW_9_2_1
cd CMSSW_9_2_1/src
cmsenv
cmsDriver.py step2 --datatier FEVTDEBUGHLT --conditions 92X_dataRun2_HLT_v2 -s L1REPACK:FullSimTP --process reTPs --era Run2_2017 --eventcontent FEVTDEBUGHLT -n 10 --filein root://eoscms.cern.ch//store/user/cawest/run294745_ls0010_streamDQM_mrg-c2f12-23-01.dat --no_exec
sed -i 's/PoolSource/NewEventStreamFileReader/' step2_L1REPACK.py
sed -i 's/secondaryFileNames = cms.untracked.vstring()//' step2_L1REPACK.py
cmsRun step2_L1REPACK.py

The exactly command is irrelevant--the point is to run the simHcalTriggerPrimitiveDigis module. These events crashed the HLT as well and so they do not show up in standard datasets. I need to use a DQM streamer file copied by hand from P5 to provoke the crash. Including this PR eliminates the crash.

To see the effect of filtering bad data on data versus emulator mismatches, I've attached plots of mismatches in run 295376 before and after this PR. In this run, the low voltage for HFM07 (the yellow blocks at ieta < 29) was off. The remaining discrepancies in HF are known to be due to a firmware bug.

run295376_mismatch.pdf
run295376_mismatch_omit_bad_data.pdf

As simulation does not include invalid data or calibration channels, this PR should have no effect on simulation or pre-2016 data. This expectation is confirmed in the comparisons with the baseline, in which the only comparison with RECO changes (in RelVal 25202.0) is a consequence of different files being used for pileup simulation (see https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_9_2_X_2017-06-06-1100+19105/20457/validateJR/logRootQA.log) for the baseline versus the baseline+PR19105.

[rekovic]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

[davidlange6]

+1