[13.1.X] Introduce missing hits recovery in the SiStripHitEfficiency PCL and CalibTree workflows

[mmusich]

backport of #42175
backport of #41970

PR description:

These changes implement the recovery of missing hits for the mkFit trajectories. Without this, the hit efficiency is overestimated in single-sided layers.
Indico page for results : https://indico.cern.ch/event/1271487/ (Update on hit efficiency)
This PR also to extend the recovery of missing hits for the mkFit trajectories to the DQM-based SiStripHitEfficiency workflow

PR validation:

see master PR: #42175

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

verbatim backport of #42175 and #41970

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_13_1_X.

It involves the following packages:

• CalibTracker/SiStripHitEfficiency (alca)
• Calibration/TkAlCaRecoProducers (alca)

@perrotta, @consuegs, @cmsbuild, @saumyaphor4252, @francescobrivio, @tvami can you please review it and eventually sign? Thanks.
@echabert, @robervalwalsh, @gbenelli, @tocheng, @mmusich, @threus this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from Introduce missing hits recovery in the SiStripHitEfficiency PCL workflow to be in synch with CalibTree based one #42175

[mmusich]

test parameters:

• workflow = 1001.2, 1001.3

[mmusich]

@cmsbuild, please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f8f8a0/33570/summary.html
COMMIT: e68a53c
CMSSW: CMSSW_13_1_X_2023-07-06-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42200/33570/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

• You potentially removed 11 lines from the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3237726
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3237701
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 218 log files, 161 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[tvami]

@cmsbuild , please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f8f8a0/33580/summary.html
COMMIT: e68a53c
CMSSW: CMSSW_13_1_X_2023-07-06-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42200/33580/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 10 lines from the logs
• Reco comparison results: 189 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3237726
• DQMHistoTests: Total failures: 5093
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3232611
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 218 log files, 161 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[francescobrivio]

+1

• all differences come from 12434.7 which is known from spurious differences in outputs of Run-3 wfs *.7 #39803

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[rappoccio]

+1