Update pixelCluster to support PhaseII

[VinInn]

Update pixelCluster to support PhaseII

to test reading old reco I refit an event and the result is the same as refitting a new reco file...

in principle one may observe regressions due to the change in maxsize of clusters from 64 to 128

once this is validated and integrated PhaseII wf have to be modified to use standard Run2 or PhaseII sequences (and all specific PhaseII pixel code removed for good)

[cmsbuild]

A new Pull Request was created by @VinInn (Vincenzo Innocente) for CMSSW_8_1_X.

It involves the following packages:

DataFormats/SiPixelCluster

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel this is something you requested to watch as well.
@slava77, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

[VinInn]

@emiglior @boudoul @ebrondol @dkotlins @atricomi fyi

[VinInn]

@cmsbuild , please test

[cmsbuild]

The tests are being triggered in jenkins.

[cmsbuild]

Pull request #15318 was updated. @cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please check and sign again.

[VinInn]

@cmsbuild , please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/14267/console

[VinInn]

so the iorule was wrong, apparently not affecting refit
and testing rereco from pixel cluster is impossible because of cluster spitting...
so testing correctness of reco reading seems to be tricky

[cmsbuild]

+1
Tested at: bf41e59
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15318/14267/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15318/14267/summary.html

[slava77]

I took a look:

• no comments on the code
• jenkins shows differences, higher stat tests will need to be run locally

[cmsbuild]

-1

Tested at: ecee728

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15318/14352/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

• RelVals:

When I ran the RelVals I found an error in the following worklfows:
140.53 step1

DAS Error

1001.0 step1

DAS Error

1003.0 step1

DAS Error

[slava77]

it looks like we are back to pre-jenkins times for code output checkout at run time.

[slava77]

last fixes definitely made things better.
more detailed report and probably signoff are coming soon.

[slava77]

@cmsbuild please test
should work now

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/14395/console

[cmsbuild]

+1
Tested at: ecee728
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15318/14395/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-15318/14395/summary.html

[slava77]

+1

for #15318 ecee728

• changes are in line with expected to support PhaseII
• jenkins tests pass and comparisons with baseline show only small differences
• higher stat tests, including the previously problematic wf 1313 show very small changes (at the level of 0.01% or less per general track); timing shows no significant difference (using wf 25202).

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @smuzaffar

[davidlange6]

+1
did we look at the memory change in reco/hlt?

[slava77]

On 8/6/16 3:04 AM, David Lange wrote:

+1
did we look at the memory change in reco/hlt?

I didn't really think it would matter.

Looking at per-job totals, there is a reduction in run2 between 50 and
80 MB in MemoryCheck-based RSS (looking at PU35 step3 MC without DQM and
Validation; and at step2 and step3 of 136.722 2016B data reco).
This is on a single thread.
I can guess that removal of 2 float baskets/branches for root storage
and addition of 2 uint8 values with their corresponding branches is a
net reduction, but maybe observed change in memory is unrelated.

On disk in RECO in 25202 (70 events) there is ~2% increase (0.1% of RECO
size), but AOD doesn't have the clusters.

94572.8 -> 96273.2 1700 1.8 0.08
SiPixelClusteredmNewDetSetVector_siPixelClusters__RECO.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#15318 (comment), or
mute the thread
https://github.com/notifications/unsubscribe-auth/AEdcbtFzLC7AC_iYv-ur1REBHHU2ZOZUks5qdEAqgaJpZM4JXPtk.

[schneiml]

so testing correctness of reco reading seems to be tricky

Is it possible that this breaks processing current (phase0) DQM stream data using 8_1_X? This scenario usually does not exist (probably for good reason), but it would be useful for phase1 online DQM development (since there is no phase1 online yet, and phase1 DQM only lives in 8_1_0).

When I try to process run 281727 (available in lookarea on the tracker online DQM development machine) with 8_1_0_pre12, I get this error (almost) all the time:

29-Sep-2016 16:13:40 CEST  Can't deserialize event or registry data: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=TBufferFile::CheckByteCount
object of class vector<SiPixelCluster> read too many bytes: 209000 instead of 193708

(almost, since in rare cases one event goes through before the crash)

Note that however I can (!) process 272022 from /fff/BU0/output/interestingruns/ without errors. (This also means I can live with this issue if there is no easy fix/workaround, but 19 LS is a bit limiting now and then...)

Update: 8_1_0_pre9 (before this was merged) works.

(also @dmitrijus @vanbesien )

[VinInn]

reco data-format shall be considered transient. please start from raw. onlyTracker reco is fully supported and it is faster than reading reco