[12.4.X] introduce `DiMuonMassBiasMonitor` and `DiMuonMassBiasClient` #39166

mmusich · 2022-08-24T07:32:17Z

PR description:

The purpose of this PR is to expand on the concept introduced at #38425, about adding the DQM monitoring for the ALCARECOTkAlDiMuonAndVertex stream and adds a new DQM Analyzer (DiMuonMassBiasMonitor) to monitor the bias of the di-muon mass system as a function of the component track kinematics.
In addition it adds a client DQMEDHarvester to create profiles of the mass bias vs track kinematics, based on the 2D MonitorElements introduced.

PR validation:

Run the (augmented) unit tests of this package and also run manually on 2022C data obtaining the fits at this link.
The resulting DQM file has been uploaded to a private GUI that is available by:

 ssh -NL 8060:localhost:8060 <USER>@lxplus724.cern.ch

and visiting:

https://tinyurl.com/2pwdox97

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

verbatim backport of #39148 and #39128

…asClient, update sequence run in alcareco production, update unit tests

mmusich · 2022-08-24T07:32:29Z

type trk, new-feature

cmsbuild · 2022-08-24T07:32:43Z

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_12_4_X.

It involves the following packages:

DQMOffline/Alignment (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@rociovilar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mmusich · 2022-08-24T07:34:40Z

@cmsbuild, please test

cmsbuild · 2022-08-24T11:31:22Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a5b591/27044/summary.html
COMMIT: c5d5f23
CMSSW: CMSSW_12_4_X_2022-08-23-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39166/27044/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test testDiMuonVertexMonitor had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3677242
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3677211
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 56.999 KiB( 49 files compared)
DQMHistoSizes: changed ( 1000.0 ): 56.983 KiB AlCaReco/TkAlDiMuonAndVertex
DQMHistoSizes: changed ( 1000.0 ): 0.008 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 1000.0 ): 0.008 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

tvami · 2022-08-24T11:46:29Z

@mmusich given that this is the DQM part, I assume it's not needed for the ReReco (of 2022A/2022B). Is that correct?

tvami · 2022-08-24T11:47:58Z

I guess I should also ask, independently of #39166 (comment) would you like this to be part of the next 12_4_X that is supposed to be cut today/tomorrow?

mmusich · 2022-08-24T11:48:35Z

would you like this to be part of the next 12_4_X that is supposed to be cut today/tomorrow?

yes.

tvami · 2022-08-24T11:49:35Z

urgent

cmsbuild · 2022-08-24T12:01:30Z

Pull request #39166 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

mmusich · 2022-08-24T12:01:58Z

@cmsbuild, please test

mmusich · 2022-08-24T12:19:16Z

DQM updates as usual can be applied also on top of the existing online releases.

this applies only to online DQM, and not offline DQM that runs concurrently with the reconstruction at Tier-0.

Because of that: do you really think that we should wait building 12_4_8 because of this PR? And, if so, why?

It's not needed for re-reco, but it will be useful to get it for the upcoming Prompt Reco of the rest of 2022 data.
So if it's in the plans to use 12_4_8 at Tier-0 it would be great to have.

perrotta · 2022-08-24T12:41:16Z

It's not needed for re-reco, but it will be useful to get it for the upcoming Prompt Reco of the rest of 2022 data. So if it's in the plans to use 12_4_8 at Tier-0 it would be great to have.

Given the pause in the LHC runs, we will still have several days (weeks?) without beam. Let build the release for the re-reco now, as planned, then. Another 12_4_X release for the MC production will follow. If this (and possibly additional) PRs will be needed at T0 before that, we can still make a patch release with them,

cmsbuild · 2022-08-24T15:58:37Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a5b591/27053/summary.html
COMMIT: e03333c
CMSSW: CMSSW_12_4_X_2022-08-24-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39166/27053/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 4114 lines to the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3677242
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 2
DQMHistoTests: Total successes: 3677205
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 57.003 KiB( 49 files compared)
DQMHistoSizes: changed ( 1000.0 ): 56.983 KiB AlCaReco/TkAlDiMuonAndVertex
DQMHistoSizes: changed ( 1000.0 ): 0.008 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 1000.0 ): 0.008 KiB MessageLogger/Warnings
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

…rt concurrent lumis

cmsbuild · 2022-08-24T16:59:38Z

Pull request #39166 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

mmusich · 2022-08-24T17:00:01Z

enable threading

mmusich · 2022-08-24T17:00:34Z

please test

cmsbuild · 2022-08-24T21:08:27Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a5b591/27063/summary.html
COMMIT: cae3ec2
CMSSW: CMSSW_12_4_X_2022-08-24-1100/el8_amd64_gcc10
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39166/27063/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3677242
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3677211
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 56.991 KiB( 49 files compared)
DQMHistoSizes: changed ( 1000.0 ): 56.983 KiB AlCaReco/TkAlDiMuonAndVertex
DQMHistoSizes: changed ( 1000.0 ): 0.004 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 1000.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

emanueleusai · 2022-08-25T05:19:44Z

+1

cmsbuild · 2022-08-25T05:20:09Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mmusich · 2022-08-25T16:24:28Z

last IB relvals look clean after the last bugfix was pushed in master. Any residual objection to merge this PR @cms-sw/orp-l2 ?

perrotta · 2022-08-25T17:34:37Z

+1

backport of introduce DiMuonMassBiasClient and use in ALCARECOTkAlDiMuonAndVertexDQM #39148 , Introduce DiMuonMassBiasMonitor for Tk Alignment ALCARECO monitoring purposes #39128 and DiMuonMassBiasClient: add protection for empty input histogram in the fitting function #39173

mmusich added 5 commits August 24, 2022 09:14

update DiMuonVertexMonitor

040a746

introduce DiMuonMassBiasMonitor

44c0a50

introduce DiMuonMassBiasClient

b0d02a1

protect vertex tool compatibility failures with try/catch

3894421

consume DQM token produced in DiMuonMassBiasMonitor from DiMuonMassBi…

c5d5f23

…asClient, update sequence run in alcareco production, update unit tests

cmsbuild added this to the CMSSW_12_4_X milestone Aug 24, 2022

cmsbuild added dqm-pending new-feature orp-pending pending-signatures tests-pending trk labels Aug 24, 2022

cmsbuild added tests-started and removed tests-pending labels Aug 24, 2022

cmsbuild added tests-rejected and removed tests-started labels Aug 24, 2022

cmsbuild added the urgent label Aug 24, 2022

add protection for empty input histogram in the fitting function

e03333c

cmsbuild added tests-pending and removed tests-rejected labels Aug 24, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 24, 2022

tvami mentioned this pull request Aug 24, 2022

DiMuonMassBiasClient does not support concurrent lumis #39180

Closed

remove ALCARECOTkAlDiMuonMassBiasClient from being run: doesn't suppo…

cae3ec2

…rt concurrent lumis

cmsbuild added tests-pending and removed tests-approved labels Aug 24, 2022

cmsbuild added tests-started and removed tests-pending labels Aug 24, 2022

cmsbuild added tests-approved and removed tests-started labels Aug 24, 2022

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Aug 25, 2022

cmsbuild added orp-approved and removed orp-pending labels Aug 25, 2022

cmsbuild merged commit d481bf9 into cms-sw:CMSSW_12_4_X Aug 25, 2022

mmusich deleted the DiMuonMassBiasClient_12_4_X branch August 25, 2022 17:44

mmusich mentioned this pull request Sep 7, 2022

[12.4.X] Add Tracker Alignment di-muon monitoring to other resonances #39332

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[12.4.X] introduce `DiMuonMassBiasMonitor` and `DiMuonMassBiasClient` #39166

[12.4.X] introduce `DiMuonMassBiasMonitor` and `DiMuonMassBiasClient` #39166

mmusich commented Aug 24, 2022

mmusich commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

mmusich commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

tvami commented Aug 24, 2022

tvami commented Aug 24, 2022

mmusich commented Aug 24, 2022

tvami commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

mmusich commented Aug 24, 2022

mmusich commented Aug 24, 2022 •

edited

perrotta commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

mmusich commented Aug 24, 2022

mmusich commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

emanueleusai commented Aug 25, 2022

cmsbuild commented Aug 25, 2022

mmusich commented Aug 25, 2022

perrotta commented Aug 25, 2022

[12.4.X] introduce DiMuonMassBiasMonitor and DiMuonMassBiasClient #39166

[12.4.X] introduce DiMuonMassBiasMonitor and DiMuonMassBiasClient #39166

Conversation

mmusich commented Aug 24, 2022

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

mmusich commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

mmusich commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

Unit Tests

Comparison Summary

tvami commented Aug 24, 2022

tvami commented Aug 24, 2022

mmusich commented Aug 24, 2022

tvami commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

mmusich commented Aug 24, 2022

mmusich commented Aug 24, 2022 • edited

perrotta commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

Comparison Summary

cmsbuild commented Aug 24, 2022

mmusich commented Aug 24, 2022

mmusich commented Aug 24, 2022

cmsbuild commented Aug 24, 2022

Comparison Summary

emanueleusai commented Aug 25, 2022

cmsbuild commented Aug 25, 2022

mmusich commented Aug 25, 2022

perrotta commented Aug 25, 2022

[12.4.X] introduce `DiMuonMassBiasMonitor` and `DiMuonMassBiasClient` #39166

[12.4.X] introduce `DiMuonMassBiasMonitor` and `DiMuonMassBiasClient` #39166

mmusich commented Aug 24, 2022 •

edited