SiPixelQuality : fix tagging of the stuckTBM channels

[tocheng]

Backport of #23267 to CMSSW_10_1_X

[cmsbuild]

A new Pull Request was created by @tocheng (Tongguang) for CMSSW_10_1_X.

It involves the following packages:

CalibTracker/SiPixelQuality
Configuration/PyReleaseValidation
Configuration/StandardSequences

@prebello, @arunhep, @kpedro88, @fabozzi, @cmsbuild, @franzoni, @cerminar, @GurpreetSinghChahal, @lpernie, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @tocheng, @VinInn, @Martin-Grunewald, @dkotlins, @rovere, @mmusich, @dgulhan this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[tocheng]

Hello, @tvami, @tsusa, @veszpv, this is something you want to follow.

[lpernie]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28092/console Started: 2018/05/21 22:59

[cmsbuild]

+1
Tested at: d174cf7
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23268/28092/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23268/28092/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 29
• DQMHistoTests: Total histograms compared: 2500840
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2500663
• DQMHistoTests: Total skipped: 176
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 28 files compared)
• Checked 119 log files, 9 edm output root files, 29 DQM output files

[cmsbuild]

Pull request #23268 was updated. @arunhep, @cerminar, @cmsbuild, @franzoni, @lpernie, @fabiocos, @davidlange6 can you please check and sign again.

[prebello]

please test workflow 136.8391, 1040.1, 1001.2, 1040

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28416/console Started: 2018/06/04 12:51

[cmsbuild]

+1
Tested at: dbc7147
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23268/28416/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23268/28416/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-23268/1001.2_RunZeroBias2017F+RunZeroBias2017F+TIER0EXPRUN2+ALCAEXPRUN2+ALCAHARVDSIPIXELCAL
• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-23268/1040.0_RunZeroBias2017F+RunZeroBias2017F+TIER0RAWSIPIXELCAL+ALCASPLITSIPIXELCAL+ALCAHARVDSIPIXELCAL
• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-23268/1040.1_RunExpressPhy2017F+RunExpressPhy2017F+TIER0EXPSIPIXELCAL+ALCASPLITSIPIXELCAL+ALCAHARVDSIPIXELCAL
• /build/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-23268/136.8391_RunExpressPhy2017F+RunExpressPhy2017F+HLTDR2_2017+RECODR2_2017reHLTSiPixelCalZeroBias_Prompt+HARVEST2017

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 29
• DQMHistoTests: Total histograms compared: 2498020
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2497843
• DQMHistoTests: Total skipped: 176
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 28 files compared)
• Checked 119 log files, 9 edm output root files, 29 DQM output files

[lpernie]

+1

[fabiocos]

@tocheng @lpernie as far as I can see this PR is not tested in RelVals, as it refers to PCL run in Express, am I correct? Can you confirm it?
BTW, it might be useful to update the tests in the test area to 2018, anyway the step3 adapted to 2018 works, while the step4 complains, which GT should be used? How was this practically validated?

[tocheng]

@fabiocos
Yes, it is refers to PCL run in Express.
Step 4 complains because step4 uses 100X Express GT
https://github.com/cms-sw/cmssw/blob/CMSSW_10_1_X/Configuration/PyReleaseValidation/python/relval_steps.py#L1543
Latest GT 101X_dataRun2_Express_v7 should be fine to remove the complaints.
https://cms-conddb.cern.ch/cmsDbBrowser/list/Prod/gts/101X_dataRun2_Express_v7

We currently validate the payloads manually. We look at the bad components in the payloads and compare with the pixel map and FED error plots in the offline tracker DQM GUI to make sure the payloads make sense.

If the PR leads to changes in harvester (step4), we run the harvester manually using the latest Express GT.
If the PR leads to changes in ALCARECO, we run the production of ALCARECO and harvester manually and compare payloads with the ones before the changes.

[lpernie]

@fabiocos something else is needed here?

[fabiocos]

+operations

[fabiocos]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_10_2_X is complete. This pull request will be automatically merged.