Customise function to increase cluster number check and seeding limits (80X) #16214

makortel · 2016-10-14T15:19:58Z

This PR adds a customise function to increase cluster number check limits by

2x for the "box"
adjust coefficient for "pixel < N + k*strip" from k=1/10 to k=1/7

and maxElement for PixelTripletLargeTipGenerator and MultiHitGeneratorFromChi2 by 5x. These should be sufficient to silence "TooManyClusters" and "TooManyTriplets" errors for the high pileup data.

Tested in 8_0_20 with

cmsDriver.py step3 --conditions 80X_dataRun2_Prompt_v14 -s RAW2DIGI,L1Reco,RECO \
  --runUnscheduled --process reRECO --data --era Run2_2016 --eventcontent RECO \
  --scenario pp --datatier RECO \
  --customise Configuration/DataProcessing/RecoTLR.customisePostEra_Run2_2016 \
  -n 5 --nThreads=5--fileout file:step3.root --filein <your-preferred-file> \
  --customise RecoTracker/Configuration/customiseClusterCheckForHighPileup.customiseClusterCheckForHighPileup

with a random file from ZeroBiasIsolatedBunch0 RAW.

@rovere @VinInn @mtosi

…itGeneratorFromChi2

cmsbuild · 2016-10-14T15:20:13Z

A new Pull Request was created by @makortel (Matti Kortelainen) for CMSSW_8_0_X.

It involves the following packages:

RecoTracker/Configuration

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @GiacomoSguazzoni, @rovere, @VinInn, @mschrode, @gpetruc, @dgulhan this is something you requested to watch as well.
@slava77, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

VinInn · 2016-10-14T15:21:15Z

@cmsbuild ,
please test

makortel · 2016-10-14T15:24:20Z

@VinInn You have a line break in your message, maybe they need to be on a single line? (since the bot hasn't acted yet)

VinInn · 2016-10-14T15:25:26Z

@cmsbuild , please test

cmsbuild · 2016-10-14T15:25:40Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/15745/console

slava77 · 2016-10-14T15:46:45Z

@makortel we should have a PR in 81X
Please submit one.
Thanks.

cmsbuild · 2016-10-14T17:11:58Z

+1
Tested at: 336f5b4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16214/15745/summary.html

cmsbuild · 2016-10-14T18:21:12Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16214/15745/summary.html

makortel · 2016-10-14T19:58:19Z

@slava77 There you go #16219

makortel · 2016-10-14T19:58:49Z

backport of #16219

slava77 · 2016-10-16T19:45:56Z

@makortel @VinInn
Are there good justifications for the increased values?
It looks more like selective increase of all fixed values by a factor of 10.

As discussed in https://hypernews.cern.ch/HyperNews/CMS/get/recoTracking/1653/1/1/1/1.html
it looks like we are primarily dealing with Npixel.vs.Nstrip relationship and (maybe also as a result of VFP change) this cut needs an update.
If that's accepted as a more appropriate change, I think the updated relationship for "pixel < a + b*strip" cut should become a default in tracking selections for 2017 and later.
For 80X we will then have this enabled via configuration just for this high PU run.

Changes wrt. default - 2x for the box - lower diagonal coefficient from 1/10 to 1/7

makortel · 2016-10-19T12:52:16Z

@slava77 With maxElement=500k only detachedTripletStep was giving TooManyTriplets errors, setting maxElement=1M there are no TooMany* errors (with 100 events for both).

slava77 · 2016-10-19T12:57:04Z

@makortel
thank you
which file(s) where you using for the highest PU test? (IsolatedBunch PD or something else)
I wanted to see on ~fewK events where is the memory peak (possibly try 4, 8, 16 thread jobs).

makortel · 2016-10-19T13:14:27Z

I took a random file from ZeroBiasIsolatedBunch0. Are you going to run further tests or you'd like me to run?

slava77 · 2016-10-19T13:42:36Z

@makortel
I may not get to the memory tests until tomorrow.
So, if you have time today, please check.
Thank you.

makortel · 2016-10-19T13:45:06Z

@slava77 Ok, running (but few kevents will take a while).

makortel · 2016-10-20T07:10:47Z

@slava77 I was able to run ~1300 events with 8 threads until the machine I was running on got booted. Maximum RSS (using SimpleMemoryCheck service) was 9.7 GB. The average time per event per thread was 4-5 min (on a 2.27 GHz Westmere, likely interfered with other activity).

In addition, within these ~1300 events, there were no TooMany* errors.

slava77 · 2016-10-20T07:44:50Z

@makortel thank you.
This should be useful for the Friday ops meeting discussion.

slava77 · 2016-10-20T18:26:28Z

@cmsbuild please test

cmsbuild · 2016-10-20T18:26:48Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/15860/console

cmsbuild · 2016-10-20T20:41:57Z

+1
Tested at: 4987532
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16214/15860/summary.html

cmsbuild · 2016-10-20T20:42:00Z

Comparison job queued.

cmsbuild · 2016-10-20T21:58:58Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-16214/15860/summary.html

venturia · 2016-10-21T06:02:46Z

We should make these limits configurable with a condition object and a database tag, eventually

slava77 · 2016-10-21T07:24:09Z

+1

for #16214 4987532

change are in line with description; the same as in the "parent" PR Customise function to increase cluster number check and seeding limits (81X) #16219
local tests on HLTPhysicsIsolatedBunch 283171 roughly confirm observations posted earlier by Matti.

cmsbuild · 2016-10-21T07:24:25Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @smuzaffar

slava77 · 2016-10-21T08:12:36Z

I guess some of these were seen already, but still

(red is 8_0_20_patch1 out of the box) LS60 in HLTPhysicsIsolatedBunch

davidlange6 · 2016-10-27T15:00:29Z

+1

makortel added 2 commits October 14, 2016 16:51

Add customiseClusterCheckForHighPileup.py

4cf3e12

Increase also maxElement for PixelTripletLargeTipGenerator and MultiH…

336f5b4

…itGeneratorFromChi2

cmsbuild added this to the Next CMSSW_8_0_X milestone Oct 14, 2016

cmsbuild added comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 14, 2016

cmsbuild added tests-started and removed tests-pending labels Oct 14, 2016

cmsbuild added tests-approved and removed tests-started labels Oct 14, 2016

cmsbuild added comparison-available and removed comparison-pending labels Oct 14, 2016

makortel mentioned this pull request Oct 14, 2016

Customise function to increase cluster number check and seeding limits (81X) #16219

Merged

makortel changed the title ~~Customise function to increase cluster number check and seeding limits~~ Customise function to increase cluster number check and seeding limits (80X) Oct 14, 2016

cmsbuild added the backport label Oct 14, 2016

Tune cuts

5119787

Changes wrt. default - 2x for the box - lower diagonal coefficient from 1/10 to 1/7

cmsbuild removed the comparison-available label Oct 17, 2016

cmsbuild added tests-started and removed tests-pending labels Oct 20, 2016

cmsbuild added tests-approved and removed tests-started labels Oct 20, 2016

cmsbuild added comparison-available and removed comparison-pending labels Oct 20, 2016

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Oct 21, 2016

cmsbuild added orp-approved and removed orp-pending labels Oct 27, 2016

cmsbuild merged commit 87a741f into cms-sw:CMSSW_8_0_X Oct 27, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Customise function to increase cluster number check and seeding limits (80X) #16214

Customise function to increase cluster number check and seeding limits (80X) #16214

makortel commented Oct 14, 2016 •

edited

cmsbuild commented Oct 14, 2016

VinInn commented Oct 14, 2016

makortel commented Oct 14, 2016

VinInn commented Oct 14, 2016

cmsbuild commented Oct 14, 2016 •

edited

slava77 commented Oct 14, 2016

cmsbuild commented Oct 14, 2016

cmsbuild commented Oct 14, 2016

makortel commented Oct 14, 2016

makortel commented Oct 14, 2016

slava77 commented Oct 16, 2016

makortel commented Oct 19, 2016

slava77 commented Oct 19, 2016

makortel commented Oct 19, 2016

slava77 commented Oct 19, 2016

makortel commented Oct 19, 2016

makortel commented Oct 20, 2016

slava77 commented Oct 20, 2016

slava77 commented Oct 20, 2016

cmsbuild commented Oct 20, 2016 •

edited

cmsbuild commented Oct 20, 2016

cmsbuild commented Oct 20, 2016

cmsbuild commented Oct 20, 2016

venturia commented Oct 21, 2016

slava77 commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

slava77 commented Oct 21, 2016

davidlange6 commented Oct 27, 2016

Customise function to increase cluster number check and seeding limits (80X) #16214

Customise function to increase cluster number check and seeding limits (80X) #16214

Conversation

makortel commented Oct 14, 2016 • edited

cmsbuild commented Oct 14, 2016

VinInn commented Oct 14, 2016

makortel commented Oct 14, 2016

VinInn commented Oct 14, 2016

cmsbuild commented Oct 14, 2016 • edited

slava77 commented Oct 14, 2016

cmsbuild commented Oct 14, 2016

cmsbuild commented Oct 14, 2016

makortel commented Oct 14, 2016

makortel commented Oct 14, 2016

slava77 commented Oct 16, 2016

makortel commented Oct 19, 2016

slava77 commented Oct 19, 2016

makortel commented Oct 19, 2016

slava77 commented Oct 19, 2016

makortel commented Oct 19, 2016

makortel commented Oct 20, 2016

slava77 commented Oct 20, 2016

slava77 commented Oct 20, 2016

cmsbuild commented Oct 20, 2016 • edited

cmsbuild commented Oct 20, 2016

cmsbuild commented Oct 20, 2016

cmsbuild commented Oct 20, 2016

venturia commented Oct 21, 2016

slava77 commented Oct 21, 2016

cmsbuild commented Oct 21, 2016

slava77 commented Oct 21, 2016

davidlange6 commented Oct 27, 2016

makortel commented Oct 14, 2016 •

edited

cmsbuild commented Oct 14, 2016 •

edited

cmsbuild commented Oct 20, 2016 •

edited