Jet core step from710pre8 fixed #4162

cerati · 2014-06-09T15:17:10Z

New PR replacing #4113.
without the iteration-based priority logic in the track merging but with double weight for pixel hits.
@arizzi

Conflicts: RecoTracker/CkfPattern/src/TrajectorySegmentBuilder.cc RecoTracker/FinalTrackSelectors/python/MergeTrackCollections_cff.py RecoTracker/IterativeTracking/python/iterativeTk_cff.py

Conflicts: RecoTracker/FinalTrackSelectors/python/MergeTrackCollections_cff.py RecoTracker/IterativeTracking/python/iterativeTk_cff.py

…t runs

This reverts commit a6a6ffd.

…racking input

…rger to prefer tracks with more pixel hits

… to tell SeedFromConsecutiveHitsCreator to recompute the initial seed kinematic taking into account the direction of the possible eta-phi region used to limit the seeding area

This reverts commit 55f1b90.

arizzi · 2014-06-11T09:13:16Z

This error looks not related with this PR
Il 11/giu/2014 11:11 "cmsbuild" notifications@github.com ha scritto:

-1
Tested at: f7e6c67
f7e6c67
When I ran the RelVals I found an error in the following worklfows:
8.0 step1

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step1_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

----- Begin Fatal Exception 11-Jun-2014 11:04:11 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:04:11 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

9.0 step1

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

----- Begin Fatal Exception 11-Jun-2014 11:04:18 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:04:18 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

25.0 step1

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log

----- Begin Fatal Exception 11-Jun-2014 11:04:18 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:04:18 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

1306.0 step1

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step1_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

----- Begin Fatal Exception 11-Jun-2014 11:04:21 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 18 failed at Wed Jun 11 11:04:21 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

4.22 step2

runTheMatrix-results/4.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

----- Begin Fatal Exception 11-Jun-2014 11:04:39 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 17 failed at Wed Jun 11 11:04:39 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

101.0 step1

runTheMatrix-results/101.0_SingleElectronE120EHCAL+SingleElectronE120EHCAL/step1_SingleElectronE120EHCAL+SingleElectronE120EHCAL.log

----- Begin Fatal Exception 11-Jun-2014 11:04:42 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:04:42 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

----- Begin Fatal Exception 11-Jun-2014 11:05:08 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:05:08 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

----- Begin Fatal Exception 11-Jun-2014 11:05:28 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 17 failed at Wed Jun 11 11:05:28 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

----- Begin Fatal Exception 11-Jun-2014 11:05:31 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 17 failed at Wed Jun 11 11:05:31 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD.log

----- Begin Fatal Exception 11-Jun-2014 11:05:32 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 17 failed at Wed Jun 11 11:05:32 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

401.0 step1

runTheMatrix-results/401.0_TTbarNewMix+TTbarFSPU2+HARVESTFS/step1_TTbarNewMix+TTbarFSPU2+HARVESTFS.log

----- Begin Fatal Exception 11-Jun-2014 11:05:36 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:05:36 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

50101.0 step2

runTheMatrix-results/50101.0_SingleMuPt10+SingleMuPt10FSIdINPUT+SingleMuPt10FS_ID/step2_SingleMuPt10+SingleMuPt10FSIdINPUT+SingleMuPt10FS_ID.log

----- Begin Fatal Exception 11-Jun-2014 11:05:54 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 66 on chan 19 failed at Wed Jun 11 11:05:53 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: ORA-00600: internal error code, arguments: [kqlmlktype:type], [NB], [], [], [], [], [], [], [], [], [], [] at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:456)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT.log

----- Begin Fatal Exception 11-Jun-2014 11:10:50 CEST-----------------------

An exception of category 'StdException' occurred while

[0] Constructing the EventProcessor

[1] Constructing ESSource: class=PoolDBESSource label='GlobalTag'

Exception Message:

A std::exception was thrown.

Can not get data (Additional Information: [frontier.c:1111]: No more proxies. Last error was: Request 60 on chan 17 failed at Wed Jun 11 11:10:50 2014: -8 [payload.c:105]: Server signalled payload error 1: FrontierPrep java.sql.SQLException: Listener refused the connection with the following error: ORA-12514, TNS:listener does not currently know of service requested in connect descriptor at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:512)) ( CORAL : "coral::FrontierAccess::Statement::execute" from "CORAL/RelationalPlugins/frontier" )

----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:

https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4162/85/summary.html

—
Reply to this email directly or view it on GitHub
#4162 (comment).

ktf · 2014-06-11T09:17:03Z

Indeed, it looks like some frontier server issue.

cmsbuild · 2014-06-12T10:49:48Z

+1
Tested at: f7e6c67
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4162/99/summary.html

cmsbuild · 2014-06-12T13:22:06Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4162/99/summary.html

deguio · 2014-06-12T13:23:50Z

+1
DQM code trivially affected.

slava77 · 2014-06-12T17:14:37Z

Here is a bit of a summary, for the record. Overall, changes are as expected and I don't see anything bad on the physics side.

dijet sample (flat 15 GeV to 3 TeV)
- has shorter gsfElectron tracks;
- yields in pfCandidates change: charged hadrons yield goes down by 1-3%, which is matched by an increase in pf photon candidate yields
- this shows up downstream in various METs in neutral or charged fractions sumET (MET shape itself seems OK)
single-ele 1 TeV sample
- the problem with HLT electron reco is gone
- about 10-20% decrease in pf charged hadrons around 1 TeV (migrating to become electrons), this affects the pfChMET the most (net effect is good).
- general track algos change quite a bit (below), less entries with iter0, partly replaced with entries with iter7)

dijet (wf 38) jet neutral hadron fraction:
$wf38_nhfrac_ak4$

wf 16.0 single-electron 1 TeV

slava77 · 2014-06-12T17:17:41Z

@arizzi
Hi Andrea,
I see that the selections are about the same to make a seed.
This increases CPU already by 3% of total reco in a 2012-like pileup TTbar and by 20% in the dijet (flat to 3 TeV) sample.
Can the the 100 GeV (calo) threshold be increased to give a smaller impact on average?
As I recall, for you the region of interest starts closer to 350 GeV or even higher.

slava77 · 2014-06-12T20:13:29Z

+1

for #4162 4c7a4e3
based on the review above
Since improvements to this iteration are coming soon, I think the CPU use tuning may better be done together with that.

Compared to #3859 fdb4eb4 (tested in pre8) and this PR, the jet core became slower, almost all of the slow down is in jetCoreRegionalStepTrackCandidates (it now takes x3 or so more time than it did in pre8).
Any idea why? (I checked this in 202.0 [ttbar PU] and in 38.0 [QCD dijet])

cmsbuild · 2014-06-12T20:16:31Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_2_X IBs unless changes (tests are also fine).

arizzi · 2014-06-12T20:20:21Z

It's the bugfix, it was not taking the proper maxcand
Il 12/giu/2014 22:13 "Slava Krutelyov" notifications@github.com ha
scritto:

+1

for #4162 #4162 4c7a4e3
4c7a4e3
based on the review above
Since improvements to this iteration are coming soon, I think the CPU use
tuning may better be done together with that.

Compared to #3859 #3859 fdb4eb4
fdb4eb4 (tested in pre8) and
this PR, the jet core became slower, almost all of the slow down is in
jetCoreRegionalStepTrackCandidates (it now takes x3 or so more time than it
did in pre8).
Any idea why? (I checked this in 202.0 [ttbar PU] and in 38.0 [QCD dijet])

—
Reply to this email directly or view it on GitHub
#4162 (comment).

slava77 · 2014-06-12T20:41:00Z

do you mean this one cerati@f7e6c67
.. or your original arizzi@e25ac0c
?

arizzi · 2014-06-13T07:09:53Z

@slava77 let me be more clear, in the old version the increased maxCand that is needed was not taken because a bug in the configuration due to migration of tracking config not propagated to the jetcore step. With the bug fixed (those commits above) now maxCand is larger and it takes more time (i.e. the original PR was buggy, this one has been fixed).
About plans: in a couple of weeks we could probably get a first version of the cluster splitter in (pending discussion in the POG first!) and then we can work on time optimization.

Jet core step from710pre8 fixed

arizzi and others added 30 commits April 14, 2014 12:08

Jet core dedicated tracking

b27f9a6

Conflicts: RecoTracker/CkfPattern/src/TrajectorySegmentBuilder.cc RecoTracker/FinalTrackSelectors/python/MergeTrackCollections_cff.py RecoTracker/IterativeTracking/python/iterativeTk_cff.py

Jet core dedicated tracking

acecdda

change a couple of parameters to be closer to the other tested config

1f600fc

remove PXB2

dd21464

Conflicts: RecoTracker/FinalTrackSelectors/python/MergeTrackCollections_cff.py RecoTracker/IterativeTracking/python/iterativeTk_cff.py

remove hard coded stuff

f244584

fix compilation errors

f608c70

switch to iter7

d7165a2

seeding layers ed producer

0868b5b

fix pvSrc and add jet filter to sequence

5d43445

using track jets and pixel vertices (and fixing MTE for the region) i…

8539af8

…t runs

add validation of jet core step

f16f901

cleanups

3ec0418

remove clusters from JetCoreRegionalStep

a6a6ffd

Revert "remove clusters from JetCoreRegionalStep"

1d686f6

This reverts commit a6a6ffd.

put jetcore step at the end

7f77092

fix conflict

03974aa

new schema for traj filters

3f6d57a

fix measurementtracker usage

6f58740

Merged JetCoreStepFrom710pre7 from repository arizzi

73e8898

fix cone size and switch to calo jets

c799aeb

make track independent caloTowers for caloJets to be used as JetCoreT…

2b8de53

…racking input

default at 0.2 instead of 0.1

e5b6dcd

add hbheprereco to RECO

be9a89f

consider tracks from iter7 as good as the early steps. Change trackme…

2801276

…rger to prefer tracks with more pixel hits

add priority mechanism in track merger

4fafdc6

add priority mechanism in track merger

fdb4eb4

fix proiority logic

55f1b90

Merge branch 'JetCoreStepFrom710pre7' into JetCoreStepFrom710pre8

3668416

Add a boolean flag, with false as default if missing from the config,…

46e5710

… to tell SeedFromConsecutiveHitsCreator to recompute the initial seed kinematic taking into account the direction of the possible eta-phi region used to limit the seeding area

Revert "fix proiority logic"

0923ef2

This reverts commit 55f1b90.

cerati added the tests-rejected label Jun 11, 2014

cerati added comparison-pending labels Jun 12, 2014

cerati added comparison-available labels Jun 12, 2014

ktf added a commit that referenced this pull request Jun 13, 2014

Merge pull request #4162 from cerati/JetCoreStepFrom710pre8_fixed

c3b2c12

Jet core step from710pre8 fixed

ktf merged commit c3b2c12 into cms-sw:CMSSW_7_2_X Jun 13, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jet core step from710pre8 fixed #4162

Jet core step from710pre8 fixed #4162

cerati commented Jun 9, 2014

arizzi commented Jun 11, 2014

ktf commented Jun 11, 2014

cmsbuild commented Jun 12, 2014

cmsbuild commented Jun 12, 2014

deguio commented Jun 12, 2014

slava77 commented Jun 12, 2014

slava77 commented Jun 12, 2014

slava77 commented Jun 12, 2014

cmsbuild commented Jun 12, 2014

arizzi commented Jun 12, 2014

slava77 commented Jun 12, 2014

arizzi commented Jun 13, 2014

Jet core step from710pre8 fixed #4162

Jet core step from710pre8 fixed #4162

Conversation

cerati commented Jun 9, 2014

arizzi commented Jun 11, 2014

ktf commented Jun 11, 2014

cmsbuild commented Jun 12, 2014

cmsbuild commented Jun 12, 2014

deguio commented Jun 12, 2014

slava77 commented Jun 12, 2014

slava77 commented Jun 12, 2014

slava77 commented Jun 12, 2014

cmsbuild commented Jun 12, 2014

arizzi commented Jun 12, 2014

slava77 commented Jun 12, 2014

arizzi commented Jun 13, 2014