L1T DQM Offline Stage 2 upgrades #18085

cbbrainerd · 2017-03-25T23:49:46Z

Changed L1T DQM Offline sequences to use L1T Stage 2 modules and to produce the same histograms as L1T DQM Online sequences.

Based on and supersedes #16095, #16639, #17739, and #18025.

Work by @cbbrainerd and @z4027163.

cmsbuild · 2017-03-25T23:50:05Z

A new Pull Request was created by @cbbrainerd (Christopher Brainerd) for master.

It involves the following packages:

DQM/L1TMonitorClient
DQMOffline/L1Trigger

@dmitrijus, @cmsbuild, @rekovic, @vanbesien, @mulhearn, @davidlange6 can you please review it and eventually sign? Thanks.
@kreczko, @rociovilar this is something you requested to watch as well.
@Muzaffar, @davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here

rekovic · 2017-03-28T14:10:39Z

please test

cmsbuild · 2017-03-28T14:10:51Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18758/console Started: 2017/03/28 16:12

cmsbuild · 2017-03-28T15:55:52Z

+1
Tested at: d337ce5
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18085/18758/summary.html

cmsbuild · 2017-03-28T15:55:55Z

Comparison job queued.

cmsbuild · 2017-03-28T18:10:28Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18085/18758/summary.html

rekovic · 2017-04-03T10:06:43Z

+1

dmitrijus · 2017-04-03T11:49:40Z

+1

cmsbuild · 2017-04-03T11:49:58Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @Muzaffar, @davidlange6, @smuzaffar

rekovic · 2017-05-05T09:57:02Z

+1

dmitrijus · 2017-05-09T08:12:56Z

+1

cmsbuild · 2017-05-09T08:13:14Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

davidlange6 · 2017-05-09T10:23:33Z

+1

thomreis · 2017-05-09T12:17:38Z

Hi @cbbrainerd @z4027163 ,
we would need a backport of this to 91x.

thomreis · 2017-05-09T17:24:50Z

Scratch that. We are going to do the backport of this PR and #18616 together in one. @astakia will take care of that.

davidlange6 · 2017-05-11T10:41:23Z

I see that I missed the firestorm of new printouts introduced by this PR. Please fix those asap. Run for example workflow 1325

%MSG-e L1TCaloLayer1RawToDigi: L1TCaloLayer1RawToDigi:l1tCaloLayer1Digis 11-May-2017 06:30:54 CEST Run: 1 Event: 110
Could not load FED data for 1354, putting empty collections!
%MSG
%MSG-e L1TCaloLayer1RawToDigi: L1TCaloLayer1RawToDigi:l1tCaloLayer1Digis 11-May-2017 06:30:54 CEST Run: 1 Event: 110
Could not load FED data for 1356, putting empty collections!
%MSG
%MSG-e L1TCaloLayer1RawToDigi: L1TCaloLayer1RawToDigi:l1tCaloLayer1Digis 11-May-2017 06:30:54 CEST Run: 1 Event: 110
Could not load FED data for 1358, putting empty collections!
%MSG

davidlange6 · 2017-05-11T10:42:26Z

or these

%MSG-w MonitorElement: L1TStage2MuonComp:l1tdeStage2uGMT@streamBeginRun 11-May-2017 12:32:02 CEST Run: 1 Stream: 6
*** MonitorElement: WARNING:setBinLabel: attempting to set label of non-existent bin number for ME: L1TEMU/L1TdeStage2uGMT/data_vs_emulator_comparison/summary

%MSG
%MSG-w MonitorElement: L1TStage2MuonComp:l1tdeStage2uGMT@streamBeginRun 11-May-2017 12:32:02 CEST Run: 1 Stream: 6
*** MonitorElement: WARNING:setBinLabel: attempting to set label of non-existent bin number for ME: L1TEMU/L1TdeStage2uGMT/data_vs_emulator_comparison/summary

%MSG
%MSG-w MonitorElement: L1TStage2MuonComp:l1tdeStage2uGMT@streamBeginRun 11-May-2017 12:32:02 CEST Run: 1 Stream: 0
*** MonitorElement: WARNING:setBinLabel: attempting to set label of non-existent bin number for ME: L1TEMU/L1TdeStage2uGMT/data_vs_emulator_comparison/summary

thomreis · 2017-05-11T10:51:02Z

@davidlange6 the warning messages are fixed in #18662 .

rekovic

@davidlange6 Regarding the error from the CaloLayer1 unpacker.
I will turn it to LogInfo and turn it off after 5 occurrences.

rekovic · 2017-05-11T14:54:13Z

DQMOffline/L1Trigger/python/L1TriggerDqmOffline_cff.py

+from DQM.L1TMonitor.L1TStage2_cff import *
+
+stage2UnpackPath = cms.Sequence(
+     l1tCaloLayer1Digis +


This should not be ran in case of MC, since we don't pack CaloLayer1.

Or we put in the CaloLayer1 packer in the DIGI2RAW, which is the right thing to do.

For the moment, leave the packer in the sequence, and turn the LogError into LogInfo in case of no FED found, silenced after 5 occurrences.

packer...to say unpacker.

davidlange6 · 2017-05-11T15:13:51Z

why do you expect the error message to happen at all?

…

On May 11, 2017, at 5:10 PM, rekovic ***@***.***> wrote: @rekovic commented on this pull request. In DQMOffline/L1Trigger/python/L1TriggerDqmOffline_cff.py: > + +from L1Trigger.L1TGlobal.hackConditions_cff import * +from L1Trigger.L1TMuon.hackConditions_cff import * +from L1Trigger.L1TCalorimeter.hackConditions_cff import * + +from DQMOffline.L1Trigger.L1TStage2CaloLayer2Offline_cfi import * +l1tStage2CaloLayer2OfflineDQMEmu.stage2CaloLayer2JetSource=cms.InputTag("valCaloStage2Layer2Digis") +l1tStage2CaloLayer2OfflineDQMEmu.stage2CaloLayer2EtSumSource=cms.InputTag("valCaloStage2Layer2Digis") +from DQMOffline.L1Trigger.L1TEGammaOffline_cfi import * +l1tEGammaOfflineDQMEmu.stage2CaloLayer2EGammaSource=cms.InputTag("valCaloStage2Layer2Digis") + +from Configuration.StandardSequences.Eras import eras +from DQM.L1TMonitor.L1TStage2_cff import * + +stage2UnpackPath = cms.Sequence( + l1tCaloLayer1Digis + For the moment, leave the packer in the sequence, and turn the LogError into LogInfo in case of no FED found, silenced after 5 occurrences. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Dr15Jones · 2017-05-11T15:23:34Z

As a reminder, what ever mechanism you use to keep track of the number of messages sent, it must be thread safe.

rekovic · 2017-05-11T15:33:48Z

DQMOffline/L1Trigger/python/L1TriggerDqmOffline_cff.py

+#
+
+Stage2l1TriggerOnline = cms.Sequence(
+                               stage2UnpackPath


We simply do not need the stage2UnpackPath when we run DQMOffline....so this should just go !

rekovic · 2017-05-11T15:36:29Z

@davidlange6

why do you expect the error message to happen at all?

You are right...should not be there.
The problem was the new unpacker sequence which should not be there when we run Offline DQM.
https://github.com/cms-sw/cmssw/pull/18085/files#r116023664.
All needed should be done in ~~DIGI2RAW~~ RAW2DIGI step.

davidlange6 · 2017-05-11T16:05:09Z

great - since this problem is already pushed back to 91x, we need a minimal fix soon (as to see how much of the problem is fixed)

…

On May 11, 2017, at 5:36 PM, rekovic ***@***.***> wrote: @davidlange6 why do you expect the error message to happen at all? You are right...should not be there. The problem was the new unpacker sequence which should not be there when we run Offline DQM. https://github.com/cms-sw/cmssw/pull/18085/files#r116023664. All needed should be done in DIGI2RAW step. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

rekovic · 2017-05-11T23:01:48Z

@davidlange6
Here is a recap.

The error messages reported in #18085 (comment) only happen when L1T DQMOffline is ran on MC.

This PR invoked the L1TDQM from Online, which in itself has the unpacker sequence, unpacking every stage of L1T, runs the emulator, and then does the comparison of unpacked L1T and emulated L1T and stores it in the DQM histograms. As L1T DQM Online is designed to run on DATA, running
it as part of the DQM Offline on DATA runs without a problem.

However, if one runs on MC RAW, there are some parts of L1T which are not packed in RAW.
So, some of the L1T unpackers complain, and also some parts of L1T DQM.
These are now adjusted properly to handle this in #18703.

The moral of the story: we need to develop the missing packers. We are working on it, and hopefully will make it in 92X.

The warning messages reported in #18085 (comment) are fixed in #18662, as mentioned by @thomreis.

cmsbuild added this to the Next CMSSW_9_1_X milestone Mar 25, 2017

cmsbuild added comparison-pending dqm-pending l1-pending orp-pending pending-signatures tests-pending labels Mar 25, 2017

cbbrainerd mentioned this pull request Mar 25, 2017

L1T DQM Offline Stage 2 upgrades #17739

Closed

cbbrainerd changed the title ~~Implemented usage of Offline L1TStage2 modules and produce same histo…~~ L1T DQM Offline Stage 2 upgrades Mar 25, 2017

cmsbuild added tests-started and removed tests-pending labels Mar 28, 2017

cmsbuild added tests-approved and removed tests-started labels Mar 28, 2017

cmsbuild added comparison-available and removed comparison-pending labels Mar 28, 2017

cmsbuild added l1-approved and removed l1-pending labels Apr 3, 2017

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Apr 3, 2017

cmsbuild added comparison-available and removed comparison-pending labels May 4, 2017

cmsbuild added l1-approved and removed l1-pending labels May 5, 2017

astakia mentioned this pull request May 8, 2017

L1T offline DQM efficiency plots for the muons #18616

Merged

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels May 9, 2017

cmsbuild added orp-approved and removed orp-pending labels May 9, 2017

cmsbuild merged commit 4f15b52 into cms-sw:master May 9, 2017

astakia mentioned this pull request May 9, 2017

L1T offline DQM efficiency plots for the muons (9_1_X) #18644

Closed

rekovic reviewed May 11, 2017

View reviewed changes

rekovic mentioned this pull request May 11, 2017

Pr92x L1T UnpackerCaloLayer1 & DQMCaloLayer1 Adjust for Non-present FED data in MC #18703

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

L1T DQM Offline Stage 2 upgrades #18085

L1T DQM Offline Stage 2 upgrades #18085

cbbrainerd commented Mar 25, 2017

cmsbuild commented Mar 25, 2017 •

edited by smuzaffar

rekovic commented Mar 28, 2017

cmsbuild commented Mar 28, 2017 •

edited

cmsbuild commented Mar 28, 2017

cmsbuild commented Mar 28, 2017

cmsbuild commented Mar 28, 2017

rekovic commented Apr 3, 2017

dmitrijus commented Apr 3, 2017

cmsbuild commented Apr 3, 2017

rekovic commented May 5, 2017

dmitrijus commented May 9, 2017

cmsbuild commented May 9, 2017

davidlange6 commented May 9, 2017

thomreis commented May 9, 2017

thomreis commented May 9, 2017

davidlange6 commented May 11, 2017

davidlange6 commented May 11, 2017

thomreis commented May 11, 2017

rekovic left a comment

rekovic May 11, 2017

rekovic May 11, 2017

rekovic May 11, 2017

rekovic May 11, 2017

davidlange6 commented May 11, 2017 via email

Dr15Jones commented May 11, 2017

rekovic May 11, 2017

rekovic commented May 11, 2017 •

edited

davidlange6 commented May 11, 2017 via email

rekovic commented May 11, 2017

L1T DQM Offline Stage 2 upgrades #18085

L1T DQM Offline Stage 2 upgrades #18085

Conversation

cbbrainerd commented Mar 25, 2017

cmsbuild commented Mar 25, 2017 • edited by smuzaffar

rekovic commented Mar 28, 2017

cmsbuild commented Mar 28, 2017 • edited

cmsbuild commented Mar 28, 2017

cmsbuild commented Mar 28, 2017

cmsbuild commented Mar 28, 2017

rekovic commented Apr 3, 2017

dmitrijus commented Apr 3, 2017

cmsbuild commented Apr 3, 2017

rekovic commented May 5, 2017

dmitrijus commented May 9, 2017

cmsbuild commented May 9, 2017

davidlange6 commented May 9, 2017

thomreis commented May 9, 2017

thomreis commented May 9, 2017

davidlange6 commented May 11, 2017

davidlange6 commented May 11, 2017

thomreis commented May 11, 2017

rekovic left a comment

Choose a reason for hiding this comment

rekovic May 11, 2017

Choose a reason for hiding this comment

rekovic May 11, 2017

Choose a reason for hiding this comment

rekovic May 11, 2017

Choose a reason for hiding this comment

rekovic May 11, 2017

Choose a reason for hiding this comment

davidlange6 commented May 11, 2017 via email

Dr15Jones commented May 11, 2017

rekovic May 11, 2017

Choose a reason for hiding this comment

rekovic commented May 11, 2017 • edited

davidlange6 commented May 11, 2017 via email

rekovic commented May 11, 2017

cmsbuild commented Mar 25, 2017 •

edited by smuzaffar

cmsbuild commented Mar 28, 2017 •

edited

rekovic commented May 11, 2017 •

edited