Added skimming filter for disappearing muons

[revering]

PR description:

Created a skim filter used for a search for disappearing muons (https://icms.cern.ch/tools-api/restplus/relay/piggyback/notes/AN/2020/214/files/3/download). The filter uses a tag-and-probe method to select high-quality muons which pair with probe tracks to form an invariant mass near the Z-peak, then extrapolates the probe track to the muon chambers and selects events with missing or low energy standalone muons nearby. Options are included to also select for control regions with same-sign tag/probe pairs or invariant masses away from the Z mass.

In the full analysis we need hit level data from the HCAL to fully establish our signal region, so this skim selection is used to choose a subset of interesting RECO events to keep the complete HCAL hit information in. This selection is designed to apply to AOD level files so that in existing runs we can choose events to reconstruct from RAW data without needing to re-reco the entire dataset. With the default options, roughly 0.5% of events in the single muon 2018 UL dataset are selected.

For an additional check of the expected event rate and data size, we ran ran the filter on a run 3 relVal RECO MC dataset /RelValZMM_14/CMSSW_13_0_0_pre3-PU_130X_mcRun3_2022_realistic_v2-v1/GEN-SIM-RECO. This DY->mumu process should be the most significant source of events passing our filter.
After keeping the full hbhe and ho hit collections, the relVal file had 2574 events in 1.7 GB and the post-filter file had 77 events in 60 MB, corresponding to a pass rate of 3% and an expected dataset size of 10 TB at 300 fb^-1. As an additional cross-check, we ran the skim on run 2 AOD data and confirmed a 0.5% pass rate, likely due to the inclusion of non-DY processes in the overall singlemuon dataset with lower acceptance rates than Z->mumu.

PR validation:

I ran the skim on a set of DY MC files and plotted each cut to confirm that the selections are correctly applied. I also ran our signal analyzer on the skimmed and non-skimmed MC to verify that there were no events cut by the filter which would have otherwise passed our signal or control region selections.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34006

• This PR adds an extra 16KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34006/code-checks.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34006/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34006/code-format.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34006/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34007

• This PR adds an extra 16KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34007/code-checks.patch
  e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34007/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34008

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @revering for master.

It involves the following packages:

• Configuration/Skimming (pdmv)

@cmsbuild, @bbilin, @sunilUIET, @kskovpen can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @fabiocos, @youyingli this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[perrotta]

please test

[cmsbuild]

-1

Failed Tests: ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5397a2/30409/summary.html
COMMIT: d005f43
CMSSW: CMSSW_13_0_X_2023-02-05-0000/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40666/30409/install.sh to create a dev area with all the needed externals and cmssw changes.

Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40666/34595

• This PR adds an extra 16KB to repository

• Found files with invalid states:

  • Configuration/Skimming/python/DisappearingMuonsSkim_cfi.py:
    • Added: d005f43
    • Deleted: ef202ab

[cmsbuild]

Pull request #40666 was updated. @cmsbuild, @bbilin, @sunilUIET, @kskovpen can you please check and sign again.

[perrotta]

please test with 10805.31
(that workflow is not really needed here, but I would like to verify that workflow in a context different from cms-data/RecoEgamma-EgammaPhotonProducers#3)

[youyingli]

Hi @perrotta , could you please test again? The bot seems not to receive this command.

[perrotta]

please test workflow 10805.31

[perrotta]

Hi @perrotta , could you please test again? The bot seems not to receive this command.

Thank you @youyingli : it was my fault, indeed, as I wrote a wrong syntax for the command

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5397a2/31289/summary.html
COMMIT: 8ee8512
CMSSW: CMSSW_13_1_X_2023-03-14-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40666/31289/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 8 lines from the logs
• Reco comparison results: 18 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3653342
• DQMHistoTests: Total failures: 18
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3653302
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 219 log files, 169 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[kskovpen]

+pdmv

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1