[RPC] Changing the RPCDcsInfo* about an issue of the DQM legacy module #31056

jeongsumin · 2020-08-05T09:11:20Z

PR description:

The RPCDcsInfo module has a problem because of using DQMOneLumiEDAnalyzer.
(This problem was already issued in the #25090 )

This module gets the RPC DCS bit per Lumi-Section using 'scalersRawToDigi'.
So in the harvesting step, RPCDcsInfoClient also affected by this problem.
But DCS bit already contained in the reportSummaryMap which is histogramming at the DQMProvInfo module.
Therefore, we do not need to use the RPCDcsInfo module and the luminosity block (dqmEndLuminosityBlock) in the RPCDcsInfoClient.

From this change, RPC does not have an issue with this.

At the bottom, I attach the RPC HV status plot which affected by RPC DCS bit.
before & after changing the module:

Fixing the bug: the HV status bit has filled at the "lumi_number+1"th bin. So I change that.
** HV status histogram: If the scalersRawToDigi (DCS bit) not found, then HV status is 0 (HV was off). If not, then HV status is 1 (HV was on)
*** This data has 576 LumiSection

PR validation:

Tested by runTheMatrix 138.1 (RunCosmics2020, MWGR data) with multi-thread and single-thread

if this PR is a backport please specify the original PR and why you need to backport that PR:

@andresib @jhgoh
I make the new PR about changing the RPC DQM legacy module.

…PC DCS bits that saved at the DQMProvInfo

cmsbuild · 2020-08-05T09:11:50Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-05T09:19:02Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17567

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/RPCMonitorClient/python/RPCTier0Source_cff.py modified in PR(s): DQM: Remove concurrent-lumi blocking DQM Modules. #30111

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17567/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17567/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2020-08-06T05:54:25Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-06T05:59:47Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17593

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/RPCMonitorClient/python/RPCTier0Source_cff.py modified in PR(s): DQM: Remove concurrent-lumi blocking DQM Modules. #30111

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17593/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17593/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2020-08-06T06:04:45Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-06T06:09:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31056/17594

This PR adds an extra 16KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DQM/RPCMonitorClient/python/RPCTier0Source_cff.py modified in PR(s): DQM: Remove concurrent-lumi blocking DQM Modules. #30111

cmsbuild · 2020-08-06T06:09:45Z

A new Pull Request was created by @jeongsumin (jeongsumin) for master.

It involves the following packages:

DQM/RPCMonitorClient
DQM/RPCMonitorDigi

@kmaeshima, @cmsbuild, @andrius-k, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@acimmino this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2020-08-06T14:02:10Z

DQM/RPCMonitorDigi/src/SealModule.cc

@@ -7,5 +7,3 @@ DEFINE_FWK_MODULE(RPCMonitorDigi);
 DEFINE_FWK_MODULE(RPCRecHitProbability);
 #include "DQM/RPCMonitorDigi/interface/RPCTTUMonitor.h"
 DEFINE_FWK_MODULE(RPCTTUMonitor);
-#include "DQM/RPCMonitorDigi/interface/RPCDcsInfo.h"
-DEFINE_FWK_MODULE(RPCDcsInfo);


If you are making it so the module can never be loaded, then shouldn't RPCDcsInfo.h and RPCDcsInfo.cc both be removed from CMSSW?

I forgot to update the removal... Thank you for mentioning that!

cmsbuild · 2020-08-17T16:35:32Z

Comparison job queued.

cmsbuild · 2020-08-17T18:02:24Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3cf0ea/8789/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2608220
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2608197
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.852 KiB( 34 files compared)
DQMHistoSizes: changed ( 1000.0,... ): -0.042 KiB RPC/DCSInfo
DQMHistoSizes: changed ( 10024.0,... ): -0.030 KiB RPC/DCSInfo
Checked 149 log files, 22 edm output root files, 35 DQM output files

jfernan2 · 2020-08-18T07:57:00Z

+1
Thanks @jeongsumin

cmsbuild · 2020-08-18T15:51:04Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2020-08-18T15:57:18Z

+1

mrodozov · 2020-08-19T09:24:31Z

After this PR was merged, this unit test started failing in all IBs
https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc900/CMSSW_11_2_X_2020-08-18-2300/unitTestLogs/DQM/Integration#/
with:

9-Aug-2020 03:52:23 CEST  Successfully opened file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/express/Commissioning2019/ExpressCosmics/FEVT/Express-v1/000/334/393/00000/D0F052ED-9CA5-F547-BA73-2AA370D51AE8.root
19-Aug-2020 03:52:32 CEST  Closed file root://eoscms.cern.ch//eos/cms/store/user/cmsbuild/store/express/Commissioning2019/ExpressCosmics/FEVT/Express-v1/000/334/393/00000/D0F052ED-9CA5-F547-BA73-2AA370D51AE8.root
----- Begin Fatal Exception 19-Aug-2020 03:52:32 CEST-----------------------
An exception of category 'PluginNotFound' occurred while
   [0] Constructing the EventProcessor
Exception Message:
Unable to find plugin 'RPCDcsInfo' in category 'CMS EDM Framework Module'. Please check spelling of name.
----- End Fatal Exception -------------------------------------------------

jfernan2 · 2020-08-19T09:58:24Z

@jeongsumin I believe it has to do with the Online client in
https://github.com/cms-sw/cmssw/blob/master/DQM/Integration/python/clients/rpc_dqm_sourceclient-live_cfg.py#L159
after the removal of the plugin:
#31056 (comment)
Can you please have a look ASAP?

jeongsumin · 2020-08-19T10:22:04Z

@jfernan2 Thanks for checking that. I will see that right now.

jeongsumin · 2020-08-19T16:55:23Z

@jfernan2 @mrodozov Sorry for the late reply.
When I checked the rpc_dqm_sourceclient-live_cfg.py that remove the rpcDcsInfo at #L159 and https://github.com/cms-sw/cmssw/blob/master/DQM/Integration/python/clients/rpc_dqm_sourceclient-live_cfg.py#L105,
there was no fatal exception about RPCDcsInfo.
But I saw the error message from RPCMonitorDigi that might be occurred when the muon input tag is not valid.
https://github.com/cms-sw/cmssw/blob/master/DQM/RPCMonitorDigi/src/RPCMonitorDigi.cc#L182

Have you ever seen that message when running the unit test? I don't know the reason why this error can be occurred for now.

jfernan2 · 2020-08-19T18:38:04Z

@jeongsumin you removed the plugin in:
https://github.com/cms-sw/cmssw/pull/31056/files#diff-1fd1af75a5154bfd453f1cd93273aad4L10
hence Online DQM client cannot find the plugin: the crash happens at runing even if the cfi defines it

jeongsumin · 2020-08-20T10:02:32Z

@jfernan2 I forgot to do the unit test by using the before merged version yesterday. I'm so sorry about that.
(before merged version: RPCDcsInfo module exist in the RPCMonitorDigi)

The "[RPCMonitorDigi]: Muons - Product not valid for event" error also occurs at the before merged version when doing the unit test using by RPC online client. I'm not sure, but It seems to some option that the muon information can be used when the unit test.
@mrodozov Is there any special option about using the muon tag when running the RPC online client? I would very appreciate your answer.

So I think if the RPCDcsInfo process is removed from RPC online client, there will be no more test failing in all IBs.
And If it is no problem to open the new PR about the modification of RPC online client, then I would make that after your answer.

jfernan2 · 2020-08-20T12:44:51Z

@jeongsumin if RPC is fine removing the module from Online DQM, we are fine too :-)

jeongsumin · 2020-08-20T13:56:42Z

@jfernan2 Thanks a lot. Then I will make the PR about this right now.

Removing the RPCDcsInfo module & Modifying the DcsInfoClient to use R…

afac9be

…PC DCS bits that saved at the DQMProvInfo

cmsbuild added this to the CMSSW_11_2_X milestone Aug 5, 2020

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures tests-pending labels Aug 5, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Aug 5, 2020

clang format

8828d89

cmsbuild added code-checks-pending and removed code-checks-rejected labels Aug 6, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Aug 6, 2020

clang format

0c5d9e7

cmsbuild added code-checks-pending and removed code-checks-rejected labels Aug 6, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 6, 2020

Dr15Jones reviewed Aug 6, 2020

View reviewed changes

remove DcsInfo from RPC digi

3d60e16

cmsbuild removed the code-checks-approved label Aug 6, 2020

cmsbuild added tests-approved and removed tests-started labels Aug 17, 2020

cmsbuild added comparison-available and removed comparison-pending labels Aug 17, 2020

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Aug 18, 2020

cmsbuild added orp-approved and removed orp-pending labels Aug 18, 2020

cmsbuild merged commit efae9ca into cms-sw:master Aug 18, 2020

This was referenced Aug 18, 2020

Apply necessary changes to work with python3 #31166

Merged

Fix undefined behavior in EgammaTowerIsolation #31170

Merged

jeongsumin mentioned this pull request Aug 20, 2020

[RPC] Removing the RPCDcsInfo module from RPC online client #31193

Merged

schneiml mentioned this pull request Sep 4, 2020

DQM: Remove concurrent-lumi blocking DQM Modules. #30111

Closed

jfernan2 mentioned this pull request Sep 5, 2020

Remove legacy dqm #31369

Closed

Somhammer mentioned this pull request Feb 28, 2021

[RPC] Bug fix DQM/RPCMonitorClient crash due to type conversion -1 to unsigned int #33017

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RPC] Changing the RPCDcsInfo* about an issue of the DQM legacy module #31056

[RPC] Changing the RPCDcsInfo* about an issue of the DQM legacy module #31056

jeongsumin commented Aug 5, 2020 •

edited

cmsbuild commented Aug 5, 2020

cmsbuild commented Aug 5, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

Dr15Jones Aug 6, 2020

jeongsumin Aug 6, 2020

cmsbuild commented Aug 17, 2020

cmsbuild commented Aug 17, 2020

jfernan2 commented Aug 18, 2020

cmsbuild commented Aug 18, 2020

qliphy commented Aug 18, 2020

mrodozov commented Aug 19, 2020

jfernan2 commented Aug 19, 2020

jeongsumin commented Aug 19, 2020

jeongsumin commented Aug 19, 2020

jfernan2 commented Aug 19, 2020

jeongsumin commented Aug 20, 2020

jfernan2 commented Aug 20, 2020

jeongsumin commented Aug 20, 2020

[RPC] Changing the RPCDcsInfo* about an issue of the DQM legacy module #31056

[RPC] Changing the RPCDcsInfo* about an issue of the DQM legacy module #31056

Conversation

jeongsumin commented Aug 5, 2020 • edited

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Aug 5, 2020

cmsbuild commented Aug 5, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

cmsbuild commented Aug 6, 2020

Dr15Jones Aug 6, 2020

Choose a reason for hiding this comment

jeongsumin Aug 6, 2020

Choose a reason for hiding this comment

cmsbuild commented Aug 17, 2020

cmsbuild commented Aug 17, 2020

jfernan2 commented Aug 18, 2020

cmsbuild commented Aug 18, 2020

qliphy commented Aug 18, 2020

mrodozov commented Aug 19, 2020

jfernan2 commented Aug 19, 2020

jeongsumin commented Aug 19, 2020

jeongsumin commented Aug 19, 2020

jfernan2 commented Aug 19, 2020

jeongsumin commented Aug 20, 2020

jfernan2 commented Aug 20, 2020

jeongsumin commented Aug 20, 2020

jeongsumin commented Aug 5, 2020 •

edited