Backport - RPD in 80X #16308

cferraio · 2016-10-21T16:59:02Z

Backport of PR #16150 for RPD integration into 80X

cmsbuild · 2016-10-21T16:59:31Z

A new Pull Request was created by @cferraio (Chris Ferraioli) for CMSSW_8_0_X.

It involves the following packages:

CalibCalorimetry/HcalPlugins
CalibFormats/HcalObjects
CaloOnlineTools/HcalOnlineDb
CondFormats/HcalObjects
DataFormats/HcalDetId
EventFilter/HcalRawToDigi
Geometry/ForwardGeometry
RecoHI/Configuration
RecoLocalCalo/Configuration
RecoLocalCalo/HcalRecProducers

@ghellwig, @civanch, @Dr15Jones, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @franzoni, @cerminar, @slava77, @ggovi, @mmusich, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @echapon, @apfeiffer1, @MiheeJo, @jazzitup, @richard-cms, @Martin-Grunewald, @yenjie, @tocheng, @kurtejung, @mariadalfonso, @mandrenguyen, @dgulhan, @yetkinyilmaz this is something you requested to watch as well.
@slava77, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

slava77 · 2016-10-21T20:30:05Z

@cmsbuild please test

cmsbuild · 2016-10-21T20:30:19Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/15896/console

slava77 · 2016-10-21T20:30:21Z

backport of #16150

cmsbuild · 2016-10-21T22:31:28Z

+1
Tested at: a0a9d61
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16308/15896/summary.html

cmsbuild · 2016-10-21T22:31:31Z

Comparison job queued.

cmsbuild · 2016-10-21T23:19:04Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16308/15896/summary.html

mmusich · 2016-10-24T11:42:40Z

+1

slava77 · 2016-10-24T20:07:41Z

A status update for the review:

waiting for resolution of the discussion in RPD implemented in 81X #16150 (it's not obvious yet that no changes will be necessary in the code)

ggovi · 2016-10-25T09:06:28Z

+1

mdhildreth · 2016-10-25T15:17:27Z

+1

abdoulline · 2016-10-25T20:57:45Z

@slava77, @davidlange6

I was told HCAL is now including "dummy" (yet without new FE QIE10 cards installed, until TS3) ZDC/RPD "QIE10" (via uTCA FEDs) in RAW. So we could try/validate extended emap with 80X, provided that this PR is merged.
Would be god to have extended emap validated and included in 80X GTs in advance, even before pA preparatory period (a ~week right after TS3).

cferraio · 2016-10-26T20:09:39Z

update - pr #16150, identical to this PR in 81X, has merged.

we have been working with a data file produced by Dick and Quan at 904, and the 904 emap produced by Dick. from what we can tell, everything is unpacking fine. we're now going to write an analyzer to produce easy to read histos of time slices, charge, and ADC for each channel.

abdoulline · 2016-10-28T12:21:04Z

@cferraio, @BetterWang,

HcalElectronicsMap_ZDC_RPD_QIE10

test tag (all the regular HCAL channels + ZDC&RPD QIE10 channels) is now available for P5 can be used in any local P5 tests.
(using the first recipe example for using it on top of any regular GT from
https://twiki.cern.ch/twiki/bin/viewauth/CMS/HcalMixESSources).

abdoulline · 2016-10-30T21:11:51Z

@slava77, @davidlange6
I guess this PR is not supposed to wait till it's 81X (#16150, merged) sibling will formally go through RelVals (as there is nothing really to validate in the central workflows), otherwise it might miss pA campaign beginning.
It's needed to be in 80X release to validate extended (+ ZDC/RPD QIE10) emap via regular AlCa procedures.

slava77 · 2016-10-30T21:25:15Z

On 10/30/16 2:11 PM, Salavat Abdullin wrote:

@slava77 https://github.com/slava77, @davidlange6
https://github.com/davidlange6
I guess this PR will /not/ supposed to wait till it's 81X (#16150
#16150, merged) sibling will
formally go through RelVals (as there is nothing really to validate in
the central workflows), otherwise it might miss pA campaign beginning.
It's needed to be in 80X release to validate extended (+ ZDC/RPD QIE10)
emap via regular AlCa procedures.

If I understand correctly, there is some raw data with RPD (and ZDC?) in
QIE10
which can be used as a check of this PR.
Please post (or ask someone) some copy-paste style instructions to check
it, including a test emap.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16308 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AEdcbnAwK2vf432XIAQnuuB0lgpIPqCLks5q5QgYgaJpZM4Kda3k.

abdoulline · 2016-10-30T21:32:08Z

I'd expect Chris and Quan to contribute

@cferraio, @BetterWang

On Sun, 30 Oct 2016, Slava Krutelyov wrote:

On 10/30/16 2:11 PM, Salavat Abdullin wrote:

@slava77 https://github.com/slava77, @davidlange6
https://github.com/davidlange6
I guess this PR will /not/ supposed to wait till it's 81X (#16150
#16150, merged) sibling will
formally go through RelVals (as there is nothing really to validate in
the central workflows), otherwise it might miss pA campaign beginning.
It's needed to be in 80X release to validate extended (+ ZDC/RPD QIE10)
emap via regular AlCa procedures.

If I understand correctly, there is some raw data with RPD (and ZDC?) in
QIE10
which can be used as a check of this PR.
Please post (or ask someone) some copy-paste style instructions to check
it, including a test emap.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16308 (comment), or
mute the thread
<https://github.com/notifications/unsubscribe-auth/AEdcbnAwK2vf432XIAQnuuB0lgpIPqCLks5q5QgYgaJpZM4Kda3
k>.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02pdqR9DNNJ4fxQogeElLZFWczi4zks5q5Qs_gaJpZM4Kda3k.gif]

BetterWang · 2016-10-30T21:39:51Z

Hi @slava77 @abdoulline , you can find a working emap at
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
If you merge topic this PR, and run
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py
you will get QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO filled.
This works for me on 81X release, and should be working on 80X.
@cferraio can cross check.

abdoulline · 2016-10-31T06:28:10Z

@cferraio, @BetterWang

what about P5 "dummy" data with extended emap tag
HcalElectronicsMap_ZDC_RPD_QIE10 (mentioned 3 day ago in this thread)?

On Sun, 30 Oct 2016, Quan Wang wrote:

Hi @slava77 @abdoulline , you can find a working emap at
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
If you merge topic this PR, and run
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py
you will get QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO filled.
This works for me on 81X release, and should be working on 80X.
@cferraio can cross check.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02p694wwPxa4pThLkKfc9mGbJgZdMks5q5Q6rgaJpZM4Kda3k.gif]

abdoulline · 2016-10-31T12:48:46Z

Please, lay down step-by-step ("copy-paste" as Slava has asked) recipe to
just execute right away.

On Sun, 30 Oct 2016, Quan Wang wrote:

Hi @slava77 @abdoulline , you can find a working emap at
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
If you merge topic this PR, and run
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py
you will get QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO filled.
This works for me on 81X release, and should be working on 80X.
@cferraio can cross check.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02p694wwPxa4pThLkKfc9mGbJgZdMks5q5Q6rgaJpZM4Kda3k.gif]

cferraio · 2016-10-31T13:42:58Z

Hi @abdoulline,

First (of course) set up the PR in an 80X release.

Next copy the two files Quan pointed you to.
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py

Finally, run rpd.py with a "cmsRun rpd.py", open the created root file (raw.root), and inspect QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO.

Quan has many other runs in /afs/cern.ch/user/q/qwang/work/public/ which can be analyzed by changing the run number in rpd.py.

Since I've just set up myself, you can directly copy-paste this:
cmsrel CMSSW_8_0_X_2016-10-30-0000
cd CMSSW_8_0_X_2016-10-30-0000/src/
cmsenv
git cms-addpkg CalibCalorimetry/HcalPlugins
git cms-addpkg CalibFormats/HcalObjects
git cms-addpkg CaloOnlineTools/HcalOnlineDb
git cms-addpkg CondFormats/HcalObjects
git cms-addpkg DataFormats/HcalDetId
git cms-addpkg EventFilter/HcalRawToDigi
git cms-addpkg Geometry/ForwardGeometry
git cms-addpkg RecoHI/Configuration
git cms-addpkg RecoLocalCalo/Configuration
git cms-addpkg RecoLocalCalo/HcalRecProducers
git pull https://github.com/cferraio/cmssw RPDin80X
scram b -j 20
cp /afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt .
cp /afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py .
cmsRun rpd.py
root raw.root

-Chris

On Oct 31, 2016, at 13:48, Salavat Abdullin notifications@github.com wrote:

Please, lay down step-by-step ("copy-paste" as Slava has asked) recipe to
just execute right away.

On Sun, 30 Oct 2016, Quan Wang wrote:

Hi @slava77 @abdoulline , you can find a working emap at
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
If you merge topic this PR, and run
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py
you will get QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO filled.
This works for me on 81X release, and should be working on 80X.
@cferraio can cross check.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02p694wwPxa4pThLkKfc9mGbJgZdMks5q5Q6rgaJpZM4Kda3k.gif]

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

abdoulline · 2016-10-31T13:51:42Z

OK, thank you, Chris. I'm just a bit confused to see all those git
commands and not a single PR of interest

git cms-merge-topic 16308

on top of 8_1_2X

On Mon, 31 Oct 2016, Chris Ferraioli wrote:

Hi @abdoulline,

First (of course) set up the PR in an 80X release.

Next copy the two files Quan pointed you to.
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py

Finally, run rpd.py with a "cmsRun rpd.py", open the created root file (raw.root), and inspect
QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO.

Quan has many other runs in /afs/cern.ch/user/q/qwang/work/public/ which can be analyzed by
changing the run number in rpd.py.

Since I've just set up myself, you can directly copy-paste this:
cmsrel CMSSW_8_0_X_2016-10-30-0000
cd CMSSW_8_0_X_2016-10-30-0000/src/
cmsenv
git cms-addpkg CalibCalorimetry/HcalPlugins
git cms-addpkg CalibFormats/HcalObjects
git cms-addpkg CaloOnlineTools/HcalOnlineDb
git cms-addpkg CondFormats/HcalObjects
git cms-addpkg DataFormats/HcalDetId
git cms-addpkg EventFilter/HcalRawToDigi
git cms-addpkg Geometry/ForwardGeometry
git cms-addpkg RecoHI/Configuration
git cms-addpkg RecoLocalCalo/Configuration
git cms-addpkg RecoLocalCalo/HcalRecProducers
git pull https://github.com/cferraio/cmssw RPDin80X
scram b -j 20
cp /afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt .
cp /afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py .
cmsRun rpd.py
root raw.root

-Chris

On Oct 31, 2016, at 13:48, Salavat Abdullin notifications@github.com wrote:

Please, lay down step-by-step ("copy-paste" as Slava has asked) recipe to
just execute right away.

On Sun, 30 Oct 2016, Quan Wang wrote:

Hi @slava77 @abdoulline , you can find a working emap at
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txt
If you merge topic this PR, and run
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.py
you will get QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO filled.
This works for me on 81X release, and should be working on 80X.
@cferraio can cross check.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02p694wwPxa4pThLkKfc9mGbJgZdMks5q5Q6rgaJpZM4Kda3k.gif]

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02ldBO7hP6Inr9BcLDLTqqwySP3uQks5q5fBmgaJpZM4Kda3k.gif]

cferraio · 2016-10-31T13:54:33Z

Ah well that seems an easier way about it than what I usually do.

Anyway, the copy/paste and cmsrun commands should work out of the box.

Sent from my iPhone

On Oct 31, 2016, at 14:51, Salavat Abdullin <notifications@github.com mailto:notifications@github.com> wrote:

OK, thank you, Chris. I'm just a bit confused to see all those git
commands and not a single PR of interest

git cms-merge-topic 16308

on top of 8_1_2X

On Mon, 31 Oct 2016, Chris Ferraioli wrote:

Hi @abdoulline,

First (of course) set up the PR in an 80X release.

Next copy the two files Quan pointed you to.
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txthttp://cern.ch/user/q/qwang/work/public/904_runs/emap.txt
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.pyhttp://cern.ch/user/q/qwang/work/public/904_runs/rpd.py

Finally, run rpd.py with a "cmsRun rpd.py", open the created root file (raw.root), and inspect
QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO.

Quan has many other runs in /afs/cern.ch/user/q/qwang/work/public/http://cern.ch/user/q/qwang/work/public/ which can be analyzed by
changing the run number in rpd.py.

Since I've just set up myself, you can directly copy-paste this:
cmsrel CMSSW_8_0_X_2016-10-30-0000
cd CMSSW_8_0_X_2016-10-30-0000/src/
cmsenv
git cms-addpkg CalibCalorimetry/HcalPlugins
git cms-addpkg CalibFormats/HcalObjects
git cms-addpkg CaloOnlineTools/HcalOnlineDb
git cms-addpkg CondFormats/HcalObjects
git cms-addpkg DataFormats/HcalDetId
git cms-addpkg EventFilter/HcalRawToDigi
git cms-addpkg Geometry/ForwardGeometry
git cms-addpkg RecoHI/Configuration
git cms-addpkg RecoLocalCalo/Configuration
git cms-addpkg RecoLocalCalo/HcalRecProducers
git pull https://github.com/cferraio/cmssw RPDin80X
scram b -j 20
cp /afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txthttp://cern.ch/user/q/qwang/work/public/904_runs/emap.txt .
cp /afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.pyhttp://cern.ch/user/q/qwang/work/public/904_runs/rpd.py .
cmsRun rpd.py
root raw.root

-Chris

On Oct 31, 2016, at 13:48, Salavat Abdullin <notifications@github.com mailto:notifications@github.com> wrote:

Please, lay down step-by-step ("copy-paste" as Slava has asked) recipe to
just execute right away.

On Sun, 30 Oct 2016, Quan Wang wrote:

Hi @slava77 @abdoulline , you can find a working emap at
/afs/cern.ch/user/q/qwang/work/public/904_runs/emap.txthttp://cern.ch/user/q/qwang/work/public/904_runs/emap.txt
If you merge topic this PR, and run
/afs/cern.ch/user/q/qwang/work/public/904_runs/rpd.pyhttp://cern.ch/user/q/qwang/work/public/904_runs/rpd.py
you will get QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC_RECO filled.
This works for me on 81X release, and should be working on 80X.
@cferraio can cross check.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02p694wwPxa4pThLkKfc9mGbJgZdMks5q5Q6rgaJpZM4Kda3k.gif]

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the
thread.[AEx02ldBO7hP6Inr9BcLDLTqqwySP3uQks5q5fBmgaJpZM4Kda3k.gif]

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com//pull/16308#issuecomment-257298631, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AE0vVOkf7slxoNnkAfDN9fJYEopPlJPrks5q5fJzgaJpZM4Kda3k.

kpedro88 · 2016-10-31T16:04:22Z

Even if a PR isn't submitted yet, you can always do:

git cms-merge-topic cferraio:RPDin80X

(add the -u option if you want to disable automatic checkdeps, though sometimes this can break things)

slava77 · 2016-10-31T22:36:38Z

+1

for #16308 a0a9d61

backport of QIE10 unpacking code for ZDC RPD implemented in 81X #16150. The ZDC digis will be unpacked only when the hcal emap is updated. Once unpacked, they remain in the even products, but there are no consumers. So, no immediate downstream effect.
jenkins tests pass; comparisons show no differences as expected.
tests in 81X confirmed that the new product QIE10DataFrameHcalDataFrameContainer_hcalDigis_ZDC appears and it remains empty without relevant inputs (data and emap)
tests suggested in Backport - RPD in 80X #16308 (comment) run and ZDC data becomes filled with the provided emap. I ran on 100K events without problems.

@davidlange6
this PR overlaps with #16394
I prefer that this one goes in first.
Then #16394 will need a quick update to save the QIE10 ZDC digis, and then features from #16394 can be consistently applicable to 80X and 81X. If this PR is left pending, the 81X and 80X versions of #16394 will be inconsistent at least temporarily.

cmsbuild · 2016-10-31T22:37:01Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @smuzaffar

davidlange6 · 2016-11-01T17:55:27Z

+1

first commit for backport PR

a0a9d61

cmsbuild added this to the Next CMSSW_8_0_X milestone Oct 21, 2016

cmsbuild added alca-pending comparison-pending db-pending geometry-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Oct 21, 2016

cmsbuild added backport tests-started and removed tests-pending labels Oct 21, 2016

cmsbuild added tests-approved and removed tests-started labels Oct 21, 2016

cmsbuild added comparison-available and removed comparison-pending labels Oct 21, 2016

cmsbuild added alca-approved and removed alca-pending labels Oct 24, 2016

cmsbuild added the geometry-approved label Oct 25, 2016

cmsbuild added simulation-approved and removed simulation-pending labels Oct 25, 2016

abdoulline mentioned this pull request Oct 26, 2016

RPD implemented in 81X #16150

Merged

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Oct 31, 2016

slava77 mentioned this pull request Nov 1, 2016

new T0 scenario ppEra_Run2_2016_pA; migration of customise_PPwithHI to eras #16394

Merged

deguio mentioned this pull request Nov 1, 2016

add zdc task #16408

Closed

cmsbuild added orp-approved and removed orp-pending labels Nov 1, 2016

cmsbuild merged commit 9de1abc into cms-sw:CMSSW_8_0_X Nov 1, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport - RPD in 80X #16308

Backport - RPD in 80X #16308

cferraio commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

slava77 commented Oct 21, 2016

cmsbuild commented Oct 21, 2016 •

edited

slava77 commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

mmusich commented Oct 24, 2016

slava77 commented Oct 24, 2016

ggovi commented Oct 25, 2016

mdhildreth commented Oct 25, 2016

abdoulline commented Oct 25, 2016

cferraio commented Oct 26, 2016

abdoulline commented Oct 28, 2016

abdoulline commented Oct 30, 2016 •

edited

slava77 commented Oct 30, 2016

abdoulline commented Oct 30, 2016

BetterWang commented Oct 30, 2016

abdoulline commented Oct 31, 2016

abdoulline commented Oct 31, 2016

cferraio commented Oct 31, 2016

abdoulline commented Oct 31, 2016

cferraio commented Oct 31, 2016

kpedro88 commented Oct 31, 2016

slava77 commented Oct 31, 2016

cmsbuild commented Oct 31, 2016

davidlange6 commented Nov 1, 2016

Backport - RPD in 80X #16308

Backport - RPD in 80X #16308

Conversation

cferraio commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

slava77 commented Oct 21, 2016

cmsbuild commented Oct 21, 2016 • edited

slava77 commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

mmusich commented Oct 24, 2016

slava77 commented Oct 24, 2016

ggovi commented Oct 25, 2016

mdhildreth commented Oct 25, 2016

abdoulline commented Oct 25, 2016

cferraio commented Oct 26, 2016

abdoulline commented Oct 28, 2016

abdoulline commented Oct 30, 2016 • edited

slava77 commented Oct 30, 2016

abdoulline commented Oct 30, 2016

BetterWang commented Oct 30, 2016

abdoulline commented Oct 31, 2016

abdoulline commented Oct 31, 2016

cferraio commented Oct 31, 2016

abdoulline commented Oct 31, 2016

cferraio commented Oct 31, 2016

kpedro88 commented Oct 31, 2016

slava77 commented Oct 31, 2016

cmsbuild commented Oct 31, 2016

davidlange6 commented Nov 1, 2016

cmsbuild commented Oct 21, 2016 •

edited

abdoulline commented Oct 30, 2016 •

edited