Remove trigger evaluation in ExoticaDQM module and related variables

[CeliaFernandez]

PR description:

This PR removes the requirement of the Offline EXO PAG validation code for the events to pass the selected triggers. Currently a set of HLT paths was evaluated and the histograms were filled only if the events passed at least one of them.

Phase-2 HLT menu development (see #40412) would reduce the statistics in the EXO DQM histograms as shown in [1]. To avoid this decrease and provided that trigger evaluation do not provide significant insight into the validation of offline objects, we have converged into removing the trigger requirement.

Impact in Phase-2 can be studied by testing together with #40412
@srimanob

[1] https://indico.cern.ch/event/1243344/

PR validation:

This PR passed all unit tests and the full standard battery of runTheMatrix workflows:
runTheMatrix.py -l limited -i all --ibeos

It was checked that the current HLT menu let all the events of the relval samples pass and fill the histograms. No changes are expected in the validation plots with respect to the previous EXO DQM configuration. The number of events that fired these triggers is shown in the attached plots for three different relval samples that are often used in EXO validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

It is not a backport

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40665/34005

• This PR adds an extra 28KB to repository

[cmsbuild]

A new Pull Request was created by @CeliaFernandez (Celia Fernández Madrazo) for master.

It involves the following packages:

• DQM/Physics (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @syuvivida, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@rappoccio this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[srimanob]

test parameters:

• pull_request = Enable Phase2 HLT with Digi and L1T step #40412
• workflow = 23634.0

[srimanob]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6bf368/30342/summary.html
COMMIT: d8a5cc4
CMSSW: CMSSW_13_0_X_2023-01-31-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40665/30342/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 50 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 47 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3649835
• DQMHistoTests: Total failures: 294
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3649519
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 216 log files, 166 edm output root files, 50 DQM output files
• TriggerResults: found differences in 6 / 48 workflows

[srimanob]

The failure in comparison comes from FastSim workflow (13234.0, 13434.0, 135.4, 2018.1, 5.1). This should be expected since we have no HLT in FastSim, so no event passed to DQM before this PR is introduced.

One more workflow which failed in comparison is 20834.76. It is a Phase-2 workflow where HLT runs together with DIGI. Since the workflow is introduced before, so the baseline will have less events. So the change we see is expected.

[srimanob]

This PR should be good to go with #40412

[emanueleusai]

@srimanob thank you very much for the explanation of the differences, but I am still confused.

WFs 13234.0,13434.0,135.4,2018.1,20834.76,5.1 present differences in Physics/Exotica, as expected from your explanation above.

However I also see differences in WFs 20834.0,20834.75,20896.0,20900.0,21034.999,23634.0 in Egamma/Electrons/Ele1_General/ele6_triggers that i cannot explain from the modified code.
I would assume this is coming from the IB, but the fact that an "ele6_triggers" plot (and only that) makes me suspicious.

Do you have an idea of what might be going on here?

[srimanob]

Thanks @emanueleusai
I don't expect that change, I will need to have a look.

[perrotta]

@cmsbuild please test

• @srimanob @emanueleusai let retest this and compare with an IB which includes already Enable Phase2 HLT with Digi and L1T step #40412

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6bf368/30373/summary.html
COMMIT: d8a5cc4
CMSSW: CMSSW_13_0_X_2023-02-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40665/30373/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 15 lines from the logs
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3649835
• DQMHistoTests: Total failures: 473
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3649340
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 218 log files, 168 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[srimanob]

Thanks @perrotta
But it is possible that the PR that we merge effect this? So we don't see difference when we use baseline that PR is merged. We don't see difference now in ele trigger.

[perrotta]

Thanks @perrotta But it is possible that the PR that we merge effect this? So we don't see difference when we use baseline that PR is merged. We don't see difference now in ele trigger.

@srimanob I am not sure that I understand what you mean...

Anyhow:

• in the first comparison we compared the combined effect of this PR and Enable Phase2 HLT with Digi and L1T step #40412 with respect to a baseline which did not contain any of them
• In the second comparison we compared the effect of this PR only with respect to a baseline which does not contain it

Therefore, the previous extra differences are expected to come from the already merged #40412

[emanueleusai]

ah yes I see the ele6_trigger differences in #40412

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

[srimanob]

Thanks @emanueleusai @perrotta
I've opened the git issue to follow up the change in ele_trigger at #40700 (not from this PR, but from #40412)