Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond #17657

vkhristenko · 2017-02-28T22:02:15Z

Updates for HCAL DQM

HE Upgrade of 1 RBX
HF dual anode readout
update constants
tested with CMSSW_9_0_0_pre5 and run 288236 runs fine

VK

vkhristenko · 2017-02-28T22:02:25Z

@DryRun

cmsbuild · 2017-02-28T22:02:31Z

A new Pull Request was created by @vkhristenko (Viktor Khristenko) for CMSSW_9_0_X.

It involves the following packages:

DQM/HcalCommon
DQM/HcalTasks
DQM/Integration

@cmsbuild, @dmitrijus, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@threus, @batinkov, @deguio this is something you requested to watch as well.
@davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

dmitrijus · 2017-03-01T02:01:18Z

Please register it in https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMP5TagCollector

cmsbuild · 2017-03-01T08:01:12Z

Pull request #17657 was updated. @cmsbuild, @dmitrijus, @vanbesien, @davidlange6 can you please check and sign again.

cmsbuild · 2017-03-01T08:05:57Z

Pull request #17657 was updated. @cmsbuild, @dmitrijus, @vanbesien, @davidlange6 can you please check and sign again.

cmsbuild · 2017-03-01T08:22:37Z

Pull request #17657 was updated. @cmsbuild, @dmitrijus, @vanbesien, @davidlange6 can you please check and sign again.

vkhristenko · 2017-03-01T08:29:02Z

@dmitrijus , I updated the tag collector - but that is for 900_pre5

for today, either @DryRun cherry picks it or we run as is

VK

deguio · 2017-03-01T08:32:02Z

@dmitrijus

@DryRun is working on the 830 version of this PR. he can update the tag collector when ready
in the meantime could you open a 840 session as well? most likely we will move to it today at p5

davidlange6 · 2017-03-02T15:47:26Z

DQM/HcalCommon/interface/Constants.h

@@ -32,17 +32,25 @@ namespace hcaldqm
 			24, 0, 20, 0, 21, 0, 25, 0, 31, 0,
 			35, 0, 37, 0, 34, 0, 30, 0, 22,22,
 			29,29, 32,32, 0, 0, 0, 0, 0, 0,
-			0, 0, 36, 0, 0, 0, 0, 0, 0, 0,
+			0, 0, 38, 0, 0, 0, 0, 0, 0, 0,
 			0, 0, 0, 0, 0, 0, 0, 0, 0, 0
 		};
 		uint16_t const CRATE2FED[50] = {


Hi @vkhristenko @deguio- ok, so these are indeed not constant constants (as we had worried about last summer). So, are these only for online DQM? I guess the answer is no, as the tier0 sees problems. If not, how do we support 2016 data?

dmitrijus · 2017-03-02T15:49:18Z

+1

cmsbuild · 2017-03-02T15:49:40Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18077/console Started: 2017/03/02 16:52
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18097/console Started: 2017/03/03 01:52

cmsbuild · 2017-03-02T15:49:41Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_9_0_X IBs after it passes the integration tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

vkhristenko · 2017-03-02T17:23:47Z

@davidlange6

David, I requested from Hcal that somehow this mapping is obtainable from within cmssw. HCAL Status is generated per FED and all of the monitoring is done in that at cDAQ, therefore when I was writing this I thought that it's quite prudent to see the status in a similar fashion without thinking of how to convert.

However, there is a limitation of having to deal with this mapping.
I believe this is a bit short-sided that we can not perform this kind of transformations on the coordinates of our system. So far, my request went unanswered. Therefore, I see 2 possible solutions:

HCAL DPG adds this record, similar to the electronics map @deguio. A list of tuples or something like that. Similar to electronics map, this map changes only when you are upgrading something and is part of conditions and you can make your applications customizable.
HCAL DQM has to make a rewrite of that to change the status generation part from electronics to detector coordinates.

What to do - is up to @deguio, @hatakeyamak and Hcal Operations.

VK

deguio · 2017-03-02T23:45:28Z

point taken. we will find a way to accomodate new and old detector together.
(not for 840 though)

cmsbuild · 2017-03-03T02:40:48Z

-1

Tested at: b83b0f3

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17657/18097/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following worklfows:
136.731 step4

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step4_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

10021.0 step5

runTheMatrix-results/10021.0_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step5_TenMuE_0_200+TenMuE_0_200_pythia8_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log

10024.0 step5

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017/step5_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+ALCAFull_2017+HARVESTFull_2017.log

10824.0 step5

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018/step5_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+ALCAFull_2018+HARVESTFull_2018.log

cmsbuild · 2017-03-03T02:40:50Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

davidlange6 · 2017-03-03T09:20:14Z

Hum - problem is that it seems this pr breaks also 2017 mc? David On 3 Mar 2017, at 00:46, Federico De Guio <notifications@github.com<mailto:notifications@github.com>> wrote: point taken. we will find a way to accomodate new and old detector together. (not for 840 though) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17657 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEzyw03xTKG1xYhY2CbXjDzk8GRM5uKTks5rh1SYgaJpZM4MO8U3>.

vkhristenko · 2017-03-03T09:26:47Z

yes. With exactly the same problem as for 2016. Because of this mapping of crate2fed and fed2crate

VK

davidlange6 · 2017-03-03T09:59:04Z

what a mess. this of course means that even the MC, the right information is available via the GT...

but anyway, I would suggest that we just leave out the hcal dqm in the tier0 instead of breaking it everywhere else.

davidlange6 · 2017-03-03T10:44:08Z

I should have clarified, that the reverse (disable everywhere but tier0) is also fine in case that matches the priorities better.

vkhristenko · 2017-03-03T10:46:48Z

we need only online dqm - the rest can be switched off (offline workflows) for now until hcal sorts out the issue.

and regarding MC and GT - it's not a question of GT. It's a question of what FEDs/Crates are at cDAQ for that MC

VK

davidlange6 · 2017-03-03T12:36:15Z

"FEDs/Crates are at cDAQ for that MC" : is that not defined for MC in a GT or in code (we certainly don't get the info from P5 via a phone call / email:) ) ?

vkhristenko · 2017-03-03T12:40:36Z

David, we can discuss forever.

Bottom line, what I mean is that I can not customize my application to account for a conversion from FED2Crate and from Crate2FED. May be I don't understand the joke.... all I'm trying to say is that I'm not able to customize that part, period. If hcal dpg decides they do not need that part - we will rewrite removing these dependencies completely. I hope we can close on this subject here.

VK

davidlange6 · 2017-03-03T12:56:50Z

I guess I simply don't understand why beyond just not getting the needed small help from your conditions people. but ok, its silly for me to spend time getting hcal people to talk to other hcal people.

…

On Mar 3, 2017, at 1:40 PM, Viktor Khristenko ***@***.***> wrote: David, we can discuss forever. Bottom line, what I mean is that I can not customize my application to account for a conversion from FED2Crate and from Crate2FED. May be I don't understand the joke.... all I'm trying to say is that I'm not able to customize that part, period. If hcal dpg decides they do not need that part - we will rewrite removing these dependencies completely. I hope we can close on this subject here. VK — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

davidlange6 · 2017-03-03T12:57:05Z

will come back in a different form. Closing until that happens.

DryRun and others added 10 commits February 27, 2017 17:38

Changes for plan-1

2a5a05e

Changes for plan-1, move arrays to maps

ab2e867

HFDigiCollection --> QIE10DigiCollection for upgrade

4b1e503

HFDigiCollection --> QIE10DigiCollection for upgrade

cef33dd

ADC to 256 for QIE10/11

a13fe77

FIBER_uTCA_MIN1=0, not 1

650a64d

Update crate-FED mappings, laser moved from 36 to 38

53a1b0f

Add some brackets

4f0adc6

Force to use offline GT 90X_dataRun2_HLT_v1

dd5b4bf

Merged CMSSW_9_0_0_pre5_HCALDQM from repository DryRun

89d4f1c

cmsbuild added this to the Next CMSSW_9_0_X milestone Feb 28, 2017

cmsbuild added comparison-pending dqm-pending orp-pending pending-signatures tests-pending labels Feb 28, 2017

fixing proper initialization for qie11 and peroper config for hcal2

a576eb1

fixing fed_sqm_.... client

5add507

making select the default GT

b83b0f3

DryRun mentioned this pull request Mar 1, 2017

Backport 9_0_0_pre5 changes to 8_3_0 #17661

Closed

deguio mentioned this pull request Mar 1, 2017

Prep hcal 2017 #17663

Merged

vkhristenko changed the title ~~Preping Online HCAL DQM Applications for MWGR2 and beyond~~ Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond Mar 2, 2017

cmsbuild mentioned this pull request Mar 2, 2017

Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond #17698

Closed

davidlange6 reviewed Mar 2, 2017

View reviewed changes

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Mar 2, 2017

cmsbuild added tests-started and removed tests-pending labels Mar 2, 2017

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Mar 3, 2017

davidlange6 closed this Mar 3, 2017

DryRun mentioned this pull request Apr 13, 2017

Restore HCAL DQM to offline workflow #18353

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond #17657

Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond #17657

vkhristenko commented Feb 28, 2017

vkhristenko commented Feb 28, 2017

cmsbuild commented Feb 28, 2017

dmitrijus commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

vkhristenko commented Mar 1, 2017

deguio commented Mar 1, 2017

davidlange6 Mar 2, 2017

dmitrijus commented Mar 2, 2017

cmsbuild commented Mar 2, 2017 •

edited

cmsbuild commented Mar 2, 2017

vkhristenko commented Mar 2, 2017 •

edited

deguio commented Mar 2, 2017

cmsbuild commented Mar 3, 2017

cmsbuild commented Mar 3, 2017

davidlange6 commented Mar 3, 2017 via email

vkhristenko commented Mar 3, 2017

davidlange6 commented Mar 3, 2017

davidlange6 commented Mar 3, 2017

vkhristenko commented Mar 3, 2017 •

edited

davidlange6 commented Mar 3, 2017

vkhristenko commented Mar 3, 2017

davidlange6 commented Mar 3, 2017 via email

davidlange6 commented Mar 3, 2017

Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond #17657

Preping Online/Offline HCAL DQM Applications for MWGR2 and beyond #17657

Conversation

vkhristenko commented Feb 28, 2017

vkhristenko commented Feb 28, 2017

cmsbuild commented Feb 28, 2017

dmitrijus commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

vkhristenko commented Mar 1, 2017

deguio commented Mar 1, 2017

davidlange6 Mar 2, 2017

Choose a reason for hiding this comment

dmitrijus commented Mar 2, 2017

cmsbuild commented Mar 2, 2017 • edited

cmsbuild commented Mar 2, 2017

vkhristenko commented Mar 2, 2017 • edited

deguio commented Mar 2, 2017

cmsbuild commented Mar 3, 2017

cmsbuild commented Mar 3, 2017

davidlange6 commented Mar 3, 2017 via email

vkhristenko commented Mar 3, 2017

davidlange6 commented Mar 3, 2017

davidlange6 commented Mar 3, 2017

vkhristenko commented Mar 3, 2017 • edited

davidlange6 commented Mar 3, 2017

vkhristenko commented Mar 3, 2017

davidlange6 commented Mar 3, 2017 via email

davidlange6 commented Mar 3, 2017

cmsbuild commented Mar 2, 2017 •

edited

vkhristenko commented Mar 2, 2017 •

edited

vkhristenko commented Mar 3, 2017 •

edited