Enable packing/unpacking for Phase2 HCAL #20920

kpedro88 · 2017-10-13T21:21:28Z

With #20758, we have a Phase2 GT that contains a functioning emap with all the HB upgrade cells.

This means that we no longer have to skip packing/unpacking for HB in Phase2 workflows. Among other things, this will enable premixing to work for the Phase2 HB.

(These changes were also propagated to 2019/post-LS2, but had to be disabled because there isn't a 2019 GT queue yet.)

cmsbuild · 2017-10-13T21:23:07Z

The code-checks are being triggered in jenkins.

cmsbuild · 2017-10-13T21:28:25Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-20920/1455

cmsbuild · 2017-10-13T21:29:06Z

A new Pull Request was created by @kpedro88 (Kevin Pedro) for master.

It involves the following packages:

CalibCalorimetry/HcalPlugins
Configuration/Eras
RecoLocalCalo/HcalRecProducers

@perrotta, @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @slava77, @lpernie, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @tocheng, @Martin-Grunewald, @ebrondol, @mmusich, @mariadalfonso this is something you requested to watch as well.
@davidlange6, @slava77 you are the release manager for this.

cms-bot commands are listed here

kpedro88 · 2017-10-13T21:29:43Z

please test

cmsbuild · 2017-10-13T21:30:01Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/23764/console Started: 2017/10/13 23:30

cmsbuild · 2017-10-14T02:54:32Z

-1

Tested at: 5e075ae

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
25793ba
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23764/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23764/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23764/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

The relvals timed out after 2 hours.

AddOn:

I found errors in the following addon tests:

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
25793ba
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23764/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23764/git-merge-result

cmsbuild · 2017-10-14T02:54:34Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

kpedro88 · 2017-10-14T03:39:25Z

please test

cmsbuild · 2017-10-14T03:39:45Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/23765/console Started: 2017/10/14 07:31

cmsbuild · 2017-10-16T16:58:34Z

+1
Tested at: c2427cf
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23788/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
8c0f84b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23788/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23788/git-merge-result

cmsbuild · 2017-10-16T16:58:37Z

Comparison job queued.

cmsbuild · 2017-10-16T18:39:26Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-20920/23788/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 26
DQMHistoTests: Total histograms compared: 2767139
DQMHistoTests: Total failures: 105
DQMHistoTests: Total nulls: 2
DQMHistoTests: Total successes: 2766861
DQMHistoTests: Total skipped: 171
DQMHistoTests: Total Missing objects: 0
Checked 107 log files, 10 edm output root files, 26 DQM output files

kpedro88 · 2017-10-16T19:34:52Z

No differences in 2019 comparison (after most recent commit)
1 difference in 2023 comparisons: plot that involves emap-related quantities, change is expected since the emap is different
reco comparisons show missing ZDC collections. Previously, the ZDC digi collection was being passed through the fake HCAL unpacker, but there are no ZDC entries in the emap for the real unpacker. Right now ZDC is unpacked via the Castor unpacker, but Castor is removed in the phase2 detector. Probably the ZDC unpacking needs to be moved back into HCAL (maybe in conjunction with moving to QIE10 readout); I'm discussing this with experts now, and can post a GitHub issue to track the progress. @slava77 @perrotta please advise

slava77 · 2017-10-16T20:23:17Z

On 10/16/17 12:35 PM, Kevin Pedro wrote: Probably the ZDC unpacking needs to be moved back into HCAL (maybe in conjunction with moving to QIE10 readout); I'm discussing this with experts now, and can post a GitHub issue to track the progress. @slava77 <https://github.com/slava77> @perrotta <https://github.com/perrotta> please advise

let's wait for clarification here from ZDC before proceeding with this PR.

abdoulline · 2017-10-16T20:45:50Z

My understanding is that HCAL emap contained entries for old ZDC detector, which doesn't exist anymore (since unsuccessful attempt to use it for the very last time in 2016 for HI data taking).
Old ZDC unpacking has been moved to CASTOR (following ZDC FEDs move to it) early in 2016 :

8395990#diff-0b2dd580fc15df181abe8beefafaae43

Instead, ZDC group has been developed a hodoscope ("RPD") detector, for which HcalZDCDetId has been extended
https://github.com/cms-sw/cmssw/commits/master/DataFormats/HcalDetId/interface/HcalZDCDetId.h
but since that time there were no more published developments (SIM/RECO) to my knowledge...

perrotta · 2017-10-23T10:00:17Z

@kpedro88 : my understanding after the reco meeting was that the ZDC group would have taken care in due time of the packing/unpacking of their detector inside HCal, and in the meanwhile we can go on with this PR and remain without the ZDC collections for Phase2 data.
Is this summary correct? Can we therefore ask for merging this PR now? Do you have some further info to be reported here after your inquiry (anticipated in #20920 (comment)) with the ZDC experts?

kpedro88 · 2017-10-23T14:29:37Z

@perrotta yes, this summary is correct. We should go forward with this PR. Here are some excerpts from the email discussion with @BetterWang:

ZDC will be part of HCAL (ideally HF, since we use the same
electronics, readout chain) for 2018 run and Phase2;

ZDC will be unpacked and calibrated by HCAL software; the current
ZDC unpacker should be discarded to give way to the HCAL unpacker.

ZDC will not be simulated in any simulations (pp, or HI) at the
moment (might update for HI in long term, but not before LS2), if has
to, maybe just use a dummy packer?

ZDC/RPD EMAP history can be found at [1]. We might want to double check it.
[1] http://cmsdoc.cern.ch/cms/HCAL/document/Mapping/ZDC/

If the [ZDC] simulation is used in the future, it will need to be upgraded to use QIE10 readout instead of QIE8, in order to be consistent with data. (Hopefully it can just be packed like HF QIE10 digis to reduce duplication of effort.)

There was already some work done in the HCAL unpacker to support ZDC w/ QIE10, see #16150.

ZDC was read out from CASTOR in 2016 pPb run. It works really fine during data taking.
We couldn't use QIE10 to read out from HF due to the radiation damage to the fibers.
In 2018, we will reinstall new fibers before the HI run, and read out from HCal.

perrotta · 2017-10-23T14:46:31Z

Thank you @kpedro88 for the confirmation

perrotta · 2017-10-23T14:49:03Z

+1

HCal packing/unpacking enabled for Phase2
ZDC collections are now missing, but ZDC will not be simulated in any simulations (pp, or HI) at the
moment (Enable packing/unpacking for Phase2 HCAL #20920 (comment))

kpedro88 · 2017-10-24T14:46:35Z

@ghellwig, @arunhep, @cerminar, @franzoni, @lpernie please sign

lpernie · 2017-10-24T15:05:16Z

+1

davidlange6 · 2017-10-26T07:38:09Z

merge

…remixing_cff as unnecessary Note that HCAL phase2 packing+unpacking was enabled in cms-sw#20920, so there is no need to remove that for phase2 premixing (which anyway is not being tested or working at the moment). Also note that in L1TDigiToRaw_cff we have to reset the rawDataCollector.RawCollectionList as if the L1TDigiToRaw_cff would not have been loaded. This was not needed earlier because the file was explicitly not loaded.

…zation Note that HCAL phase2 packing+unpacking was enabled in cms-sw#20920, so there is no need to remove that for phase2 premixing (which anyway is not being tested or working at the moment). Also note that in L1TDigiToRaw_cff we have to reset the rawDataCollector.RawCollectionList as if the L1TDigiToRaw_cff would not have been loaded. This was not needed earlier because the file was explicitly not loaded.

kpedro88 added 3 commits October 13, 2017 11:54

use real emap for HB upgrade

272709c

only use castor fallback if collection is valid

428e6dc

skip packer in 2019 until there is a GT queue

5e075ae

cmsbuild added this to the CMSSW_9_4_X milestone Oct 13, 2017

cmsbuild added alca-pending code-checks-pending comparison-pending operations-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 13, 2017

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 13, 2017

cmsbuild added tests-started and removed tests-pending labels Oct 13, 2017

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Oct 14, 2017

cmsbuild removed the comparison-notrun label Oct 14, 2017

cmsbuild added the tests-started label Oct 16, 2017

cmsbuild added tests-approved and removed tests-started labels Oct 16, 2017

cmsbuild added comparison-available and removed comparison-pending labels Oct 16, 2017

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 23, 2017

cmsbuild added alca-approved and removed alca-pending labels Oct 24, 2017

cmsbuild added orp-approved and removed orp-pending labels Oct 26, 2017

cmsbuild merged commit c4cd8ff into cms-sw:master Oct 26, 2017

kpedro88 mentioned this pull request Nov 28, 2017

Use HCAL packer for 2019 #21475

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable packing/unpacking for Phase2 HCAL #20920

Enable packing/unpacking for Phase2 HCAL #20920

kpedro88 commented Oct 13, 2017

cmsbuild commented Oct 13, 2017

cmsbuild commented Oct 13, 2017

cmsbuild commented Oct 13, 2017

kpedro88 commented Oct 13, 2017

cmsbuild commented Oct 13, 2017 •

edited

cmsbuild commented Oct 14, 2017

cmsbuild commented Oct 14, 2017

kpedro88 commented Oct 14, 2017

cmsbuild commented Oct 14, 2017 •

edited

cmsbuild commented Oct 16, 2017

cmsbuild commented Oct 16, 2017

cmsbuild commented Oct 16, 2017

kpedro88 commented Oct 16, 2017

slava77 commented Oct 16, 2017 via email

abdoulline commented Oct 16, 2017 •

edited

perrotta commented Oct 23, 2017

kpedro88 commented Oct 23, 2017

perrotta commented Oct 23, 2017

perrotta commented Oct 23, 2017

kpedro88 commented Oct 24, 2017

lpernie commented Oct 24, 2017

davidlange6 commented Oct 26, 2017

Enable packing/unpacking for Phase2 HCAL #20920

Enable packing/unpacking for Phase2 HCAL #20920

Conversation

kpedro88 commented Oct 13, 2017

cmsbuild commented Oct 13, 2017

cmsbuild commented Oct 13, 2017

cmsbuild commented Oct 13, 2017

kpedro88 commented Oct 13, 2017

cmsbuild commented Oct 13, 2017 • edited

cmsbuild commented Oct 14, 2017

cmsbuild commented Oct 14, 2017

kpedro88 commented Oct 14, 2017

cmsbuild commented Oct 14, 2017 • edited

cmsbuild commented Oct 16, 2017

cmsbuild commented Oct 16, 2017

cmsbuild commented Oct 16, 2017

kpedro88 commented Oct 16, 2017

slava77 commented Oct 16, 2017 via email

abdoulline commented Oct 16, 2017 • edited

perrotta commented Oct 23, 2017

kpedro88 commented Oct 23, 2017

perrotta commented Oct 23, 2017

perrotta commented Oct 23, 2017

kpedro88 commented Oct 24, 2017

lpernie commented Oct 24, 2017

davidlange6 commented Oct 26, 2017

cmsbuild commented Oct 13, 2017 •

edited

cmsbuild commented Oct 14, 2017 •

edited

abdoulline commented Oct 16, 2017 •

edited