bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) #12829

bsunanda · 2015-12-18T16:46:30Z

Trying to use data model evolution capability for HcalDetId

…d Run2-hcx58 (12726)

cmsbuild · 2015-12-18T16:46:53Z

A new Pull Request was created by @bsunanda for CMSSW_8_0_X.

It involves the following packages:

CalibCalorimetry/CaloMiscalibTools
CalibCalorimetry/CastorCalib
CalibCalorimetry/HcalPlugins
DataFormats/HcalDetId
Geometry/CaloTopology
Geometry/HcalTowerAlgo
RecoHI/HiJetAlgos
RecoParticleFlow/PFClusterProducer
SimCalorimetry/HcalSimProducers
Validation/GlobalRecHits

@civanch, @diguida, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @franzoni, @Dr15Jones, @cerminar, @deguio, @slava77, @mmusich, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @mmarionncern, @yslai, @tocheng, @lgray, @richard-cms, @yenjie, @jazzitup, @kurtejung, @istaslis, @mandrenguyen, @dgulhan, @bachtis, @yetkinyilmaz this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

Following commands in first line of a comment are recognized

+1|approve[d]|sign[ed]: L1/L2's to approve it
-1|reject[ed]: L1/L2's to reject it
assign <category>[,<category>[,...]]: L1/L2's to request signatures from other categories
unassign <category>[,<category>[,...]]: L1/L2's to remove signatures from other categories
hold: L1/all L2's/release manager to mark it as on hold
unhold: L1/user who put this PR on hold
merge: L1/release managers to merge this request
[@cmsbuild,] please test: L1/L2 and selected users to start jenkins tests
[@cmsbuild,] please test with cms-sw/cmsdist#<PR>: L1/L2 and selected users to start jenkins tests using externals from cmsdist

bsunanda · 2015-12-18T16:47:28Z

@cmsbuild Please test

cmsbuild · 2015-12-18T16:47:44Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10358/console

cmsbuild · 2015-12-18T19:49:51Z

+1
Tested at: 7e51ab4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12829/10358/summary.html

cmsbuild · 2015-12-18T21:19:55Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12829/10358/summary.html

slava77 · 2015-12-21T20:53:03Z

RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h

@@ -6,6 +6,7 @@
 #include "Geometry/Records/interface/HcalRecNumberingRecord.h"

 #include <iostream>
+using namespace std;


this should be removed

slava77 · 2015-12-21T21:13:01Z

-1

Things to be followed up:

digitizer couts shouuld be cleaned up
"using namespace" in a header file
reading of old SIM for mixing appear to be broken. There are very large differences in jenkins comparisons of workflows with pileup. E.g. in 50202 in jenkins comparisons

Also,

HIJetValidation plots have small changes in 2011 data processing (wf 140.53)
e.g.

this may be unrelated to the format change.
Still, something to check next time an update is made.

bsunanda · 2015-12-30T15:36:51Z

@cmsbuild Please test
@slava77 @kpedro88 I see the SimHits get the right format at the Digitizer i/p. So I cannot guess what could be wrong

cmsbuild · 2015-12-30T15:37:09Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/10405/console

slava77 · 2015-12-30T16:52:28Z

@bsunanda please clarify which tests you have run to do the check at the digitizer?
Did you use the setup in jenkins from a sample with pileup (new hard scatter SIM + old minibias SIM to mix)?

cmsbuild · 2015-12-30T18:13:41Z

+1
Tested at: bf75713
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12829/10405/summary.html

cmsbuild · 2015-12-30T19:34:17Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12829/10405/summary.html

slava77 · 2015-12-30T21:40:40Z

I see something similar to the previous iteration
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_8_0_X_2015-12-27-0900+12829/11901/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/CaloTowersD_CaloTowersTask.html

This may all be coming from HF (that's unless the plotter HcalDigiMonitor is buggy itself)
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_8_0_X_2015-12-27-0900+12829/11901/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/Hcal_DigiMonitor_Hcal_digi_info_HF.html

bsunanda · 2015-12-30T22:00:34Z

@slava77 I looked at 25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT. I shall look now in 25202. Thanks

From: Slava Krutelyov [notifications@github.com]
Sent: 30 December 2015 22:40
To: cms-sw/cmssw
Cc: Sunanda Banerjee
Subject: Re: [cmssw] bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) (#12829)

I see something similar to the previous iteration
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_8_0_X_2015-12-27-0900+12829/11901/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/CaloTowersD_CaloTowersTask.html

This may all be coming from HF (that's unless the plotter HcalDigiMonitor is buggy itself)
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_8_0_X_2015-12-27-0900+12829/11901/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/Hcal_DigiMonitor_Hcal_digi_info_HF.html

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/12829#issuecomment-168078687.

slava77 · 2015-12-30T22:02:52Z

The issue is visible only in workflows with pileup. 25.0 doesn't have any.

bsunanda · 2015-12-31T12:58:33Z

@slava77 I tried with 25202 and there too I saw all hits input to HcalDigitizer have new format. In any case if there is some with old ones they can be changed to new format in the digitizaer code.

From: Slava Krutelyov [notifications@github.com]
Sent: 30 December 2015 23:02
To: cms-sw/cmssw
Cc: Sunanda Banerjee
Subject: Re: [cmssw] bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) (#12829)

The issue is visible only in workflows with pileup. 25.0 doesn't have any.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/12829#issuecomment-168081238.

slava77 · 2015-12-31T13:15:35Z

On 12/31/15 4:58 AM, bsunanda wrote:

@slava77 I tried with 25202 and there too I saw all hits input to
HcalDigitizer have new format. In any case if there is some with old
ones they can be changed to new format in the digitizaer code.

Just to be sure, were they the same for HF?

From: Slava Krutelyov [notifications@github.com]
Sent: 30 December 2015 23:02
To: cms-sw/cmssw
Cc: Sunanda Banerjee
Subject: Re: [cmssw] bsunanda:Run2-hcx59New Expand Hcal DetId replacing
Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) (#12829)

The issue is visible only in workflows with pileup. 25.0 doesn't have any.

—
Reply to this email directly or view it on
GitHubhttps://github.com//pull/12829#issuecomment-168081238.

—
Reply to this email directly or view it on GitHub
#12829 (comment).

bsunanda · 2015-12-31T13:36:04Z

Yes it is. It is for all hcal subdetectors: hb, he, ho and hf

Sent from Samsung Mobile

Slava Krutelyov notifications@github.com wrote:
On 12/31/15 4:58 AM, bsunanda wrote:

@slava77 I tried with 25202 and there too I saw all hits input to
HcalDigitizer have new format. In any case if there is some with old
ones they can be changed to new format in the digitizaer code.

Just to be sure, were they the same for HF?

From: Slava Krutelyov [notifications@github.com]
Sent: 30 December 2015 23:02
To: cms-sw/cmssw
Cc: Sunanda Banerjee
Subject: Re: [cmssw] bsunanda:Run2-hcx59New Expand Hcal DetId replacing
Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) (#12829)

The issue is visible only in workflows with pileup. 25.0 doesn't have any.

—
Reply to this email directly or view it on
GitHubhttps://github.com//pull/12829#issuecomment-168081238.

—
Reply to this email directly or view it on GitHub
#12829 (comment).

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/12829#issuecomment-168192184.

slava77 · 2015-12-31T15:27:12Z

@bsunanda
I looked at HFDataFramesSorted_simHcalDigis__HLT in 25202 step2 outputs from jenkins and plotted interactively HFDataFramesSorted_simHcalDigis__HLT.obj.obj.data_.adc() for adc()!=0: the baseline has adc() average of 17.4, while with this PR the mean is 3.4.
This drop seems consistent with the drop in towers.
This suggests to me that the problem is in the digitizer/mixing.
Even if the format is the same, according to your observations, something is breaking the outputs.

slava77 · 2015-12-31T15:31:00Z

here come merge conflicts

kpedro88 · 2016-01-06T23:00:53Z

@slava77 - I made a new PR that hopefully resolves the problems. See #12883.

slava77 · 2016-01-07T12:50:51Z

-1
superseded

Expand Hcal DetId (supersedes #12829)

Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) an…

7e51ab4

…d Run2-hcx58 (12726)

cmsbuild added this to the Next CMSSW_8_0_X milestone Dec 18, 2015

cmsbuild added reconstruction-pending simulation-pending dqm-pending alca-pending geometry-pending pending-signatures tests-pending orp-pending comparison-pending labels Dec 18, 2015

cmsbuild added tests-started and removed tests-pending labels Dec 18, 2015

cmsbuild added tests-approved and removed tests-started labels Dec 18, 2015

cmsbuild added comparison-available and removed comparison-pending labels Dec 18, 2015

slava77 reviewed Dec 21, 2015
View reviewed changes

cmsbuild added reconstruction-rejected and removed reconstruction-pending labels Dec 21, 2015

Take care of some of the comments of Slava

bf75713

cmsbuild removed comparison-available reconstruction-rejected labels Dec 21, 2015

cmsbuild added tests-pending comparison-pending and removed comparison-available tests-approved labels Dec 30, 2015

cmsbuild added tests-started and removed tests-pending labels Dec 30, 2015

cmsbuild added tests-approved and removed tests-started labels Dec 30, 2015

cmsbuild added comparison-available and removed comparison-pending labels Dec 30, 2015

kpedro88 mentioned this pull request Jan 6, 2016

Expand Hcal DetId (supersedes #12829) #12883

Merged

cmsbuild added reconstruction-rejected and removed reconstruction-pending labels Jan 7, 2016

davidlange6 closed this Jan 7, 2016

davidlange6 added a commit that referenced this pull request Jan 12, 2016

Merge pull request #12883 from kpedro88/Run2-hcx59new2

72978da

Expand Hcal DetId (supersedes #12829)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) #12829

bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) #12829

bsunanda commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

bsunanda commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

slava77 Dec 21, 2015

slava77 commented Dec 21, 2015

bsunanda commented Dec 30, 2015

cmsbuild commented Dec 30, 2015

slava77 commented Dec 30, 2015

cmsbuild commented Dec 30, 2015

cmsbuild commented Dec 30, 2015

slava77 commented Dec 30, 2015

bsunanda commented Dec 30, 2015

slava77 commented Dec 30, 2015

bsunanda commented Dec 31, 2015

slava77 commented Dec 31, 2015

bsunanda commented Dec 31, 2015

slava77 commented Dec 31, 2015

slava77 commented Dec 31, 2015

kpedro88 commented Jan 6, 2016

slava77 commented Jan 7, 2016

bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) #12829

bsunanda:Run2-hcx59New Expand Hcal DetId replacing Run2-hcx29 (11427), Run2-hcx48 (12406) and Run2-hcx58 (12726) #12829

Conversation

bsunanda commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

bsunanda commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

cmsbuild commented Dec 18, 2015

slava77 Dec 21, 2015

Choose a reason for hiding this comment

slava77 commented Dec 21, 2015

bsunanda commented Dec 30, 2015

cmsbuild commented Dec 30, 2015

slava77 commented Dec 30, 2015

cmsbuild commented Dec 30, 2015

cmsbuild commented Dec 30, 2015

slava77 commented Dec 30, 2015

bsunanda commented Dec 30, 2015

slava77 commented Dec 30, 2015

bsunanda commented Dec 31, 2015

slava77 commented Dec 31, 2015

bsunanda commented Dec 31, 2015

slava77 commented Dec 31, 2015

slava77 commented Dec 31, 2015

kpedro88 commented Jan 6, 2016

slava77 commented Jan 7, 2016