Read DT uncertainties map from DB #2993

namapane · 2014-03-24T12:44:48Z

Reading of a new object for DT uncertainties is activated. Depends on PR #2992, which includes in the GT an object which corresponds to the same values used previously; no change of results is thus expected.

A card "useUncertDB" has been included; reverts to the old behaviour if set to False. HLT experts agreed to update ConfDB accordingly.

cmsbuild · 2014-03-24T12:48:26Z

A new Pull Request was created by @namapane (Nicola Amapane) for CMSSW_7_1_X.

Read DT uncertainties map from DB

It involves the following packages:

RecoLocalMuon/DTRecHit

@nclopezo, @cmsbuild, @anton-a, @thspeer, @slava77, @Degano can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
@nclopezo, @ktf you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

cmsbuild · 2014-03-24T12:59:56Z

-1
When I ran the RelVals I found an error in the following worklfows:
4.53 step2

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT.log
----- Begin Fatal Exception 24-Mar-2014 13:49:37 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

401.0 step1

runTheMatrix-results/401.0_TTbarNewMix+TTbarFSPU2+HARVESTFS/step1_TTbarNewMix+TTbarFSPU2+HARVESTFS.log
----- Begin Fatal Exception 24-Mar-2014 13:50:14 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
----- Begin Fatal Exception 24-Mar-2014 13:50:14 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log
----- Begin Fatal Exception 24-Mar-2014 13:51:05 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing run: 165121 lumi: 62 event: 23609118
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module DTRecHitProducer/'dt1DRecHits'
Exception Message:
No "DTRecoUncertaintiesRcd" record found in the EventSetup.
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

8.0 step2

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step2_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log
----- Begin Fatal Exception 24-Mar-2014 13:51:21 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD.log
----- Begin Fatal Exception 24-Mar-2014 13:51:41 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing run: 165121 lumi: 62 event: 23609118
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module DTRecHitProducer/'dt1DRecHits'
Exception Message:
No "DTRecoUncertaintiesRcd" record found in the EventSetup.
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

1306.0 step2

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step2_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 24-Mar-2014 13:51:58 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log
----- Begin Fatal Exception 24-Mar-2014 13:52:02 CET-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing run: 191226 lumi: 398 event: 556818698
   [1] Running path 'reconstruction_step'
   [2] Calling event method for module DTRecHitProducer/'dt1DRecHits'
Exception Message:
No "DTRecoUncertaintiesRcd" record found in the EventSetup.
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

50101.0 step2

runTheMatrix-results/50101.0_SingleMuPt10+SingleMuPt10FSIdINPUT+SingleMuPt10FS_ID/step2_SingleMuPt10+SingleMuPt10FSIdINPUT+SingleMuPt10FS_ID.log
----- Begin Fatal Exception 24-Mar-2014 13:52:15 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

25.0 step2

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step2_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log
----- Begin Fatal Exception 24-Mar-2014 13:59:09 CET-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:
MissingParameter: Parameter 'useUncertDB' not found.
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2993/627/summary.html

namapane · 2014-03-24T13:08:44Z

Note: This is because the new cfg parameter has to be added in ConfDB
and because the PR depends on #2992, as mentioned in the initial comments.

Nicola

On 24-Mar-14 13:59, cmsbuild wrote:

-1
When I ran the RelVals I found an error in the following worklfows:
4.53 step2

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT.log

----- Begin Fatal Exception 24-Mar-2014 13:49:37 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

401.0 step1

runTheMatrix-results/401.0_TTbarNewMix+TTbarFSPU2+HARVESTFS/step1_TTbarNewMix+TTbarFSPU2+HARVESTFS.log

----- Begin Fatal Exception 24-Mar-2014 13:50:14 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

----- Begin Fatal Exception 24-Mar-2014 13:50:14 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

----- Begin Fatal Exception 24-Mar-2014 13:51:05 CET-----------------------

An exception of category 'NoRecord' occurred while
[0] Processing run: 165121 lumi: 62 event: 23609118

[1] Running path 'reconstruction_step'

[2] Calling event method for module DTRecHitProducer/'dt1DRecHits'
Exception Message:

No "DTRecoUncertaintiesRcd" record found in the EventSetup.

Please add an ESSource or ESProducer that delivers such a record.

----- End Fatal Exception -------------------------------------------------

8.0 step2

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step2_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

----- Begin Fatal Exception 24-Mar-2014 13:51:21 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD.log

----- Begin Fatal Exception 24-Mar-2014 13:51:41 CET-----------------------

An exception of category 'NoRecord' occurred while
[0] Processing run: 165121 lumi: 62 event: 23609118

[1] Running path 'reconstruction_step'

[2] Calling event method for module DTRecHitProducer/'dt1DRecHits'
Exception Message:

No "DTRecoUncertaintiesRcd" record found in the EventSetup.

Please add an ESSource or ESProducer that delivers such a record.

----- End Fatal Exception -------------------------------------------------

1306.0 step2

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step2_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

----- Begin Fatal Exception 24-Mar-2014 13:51:58 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log

----- Begin Fatal Exception 24-Mar-2014 13:52:02 CET-----------------------

An exception of category 'NoRecord' occurred while
[0] Processing run: 191226 lumi: 398 event: 556818698

[1] Running path 'reconstruction_step'

[2] Calling event method for module DTRecHitProducer/'dt1DRecHits'
Exception Message:

No "DTRecoUncertaintiesRcd" record found in the EventSetup.

Please add an ESSource or ESProducer that delivers such a record.

----- End Fatal Exception -------------------------------------------------

50101.0 step2

runTheMatrix-results/50101.0_SingleMuPt10+SingleMuPt10FSIdINPUT+SingleMuPt10FS_ID/step2_SingleMuPt10+SingleMuPt10FSIdINPUT+SingleMuPt10FS_ID.log

----- Begin Fatal Exception 24-Mar-2014 13:52:15 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

25.0 step2

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step2_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log

----- Begin Fatal Exception 24-Mar-2014 13:59:09 CET-----------------------

An exception of category 'Configuration' occurred while
[0] Constructing the EventProcessor

[1] Constructing module: class=DTRecHitProducer label='hltDt1DRecHits'
Exception Message:

MissingParameter: Parameter 'useUncertDB' not found.

----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2993/627/summary.html

—
Reply to this email directly or view it on GitHub
#2993 (comment).

rcastello · 2014-03-25T13:10:41Z

This request is stuck and the PR #2992 which fix the error above is there.
Can we bring the test on and try to include in the next available pre-release?
Thanks.
R.

cmsbuild · 2014-03-25T13:13:27Z

Pull request #2993 was updated. @nclopezo, @cmsbuild, @anton-a, @thspeer, @slava77, @Degano can you please check and sign again.

nclopezo · 2014-03-25T13:13:43Z

Hi Roberto,
I queued again the tests for this pull request.

cmsbuild · 2014-03-25T13:46:49Z

+1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2993/645/summary.html

slava77 · 2014-03-28T12:23:35Z

checking

slava77 · 2014-03-29T14:42:10Z

Hi Nicola, I see the following for values of the useUncertDB in different modules in RECO (after expanding the configuration with edmConfigDump):

process.dt1DCosmicRecHits has False
process.dt2DCosmicSegments has True
process.dt4DCosmicSegments has False
process.dt1DRecHits has True
process.dt2DSegments has True
process.dt4DSegments has True

This doesn't look very consistent for cosmics.

I observe some small changes compared to the previous version of the code.
There is no specific pattern in DQM plots (nothing even shows up in the DT DQM, only starts to appear in the parameters of muon seeds).

I looked at parameters of dt4DSegments event by event about 10% of segments are different,
localX changes by under 1 micron in most cases, with similarly small changes in xx error. So, overall, this looks like some round-off issue.

namapane · 2014-03-29T15:35:26Z

Dear Slava,

On 29-Mar-14 15:42, slava77 wrote:

Hi Nicola, I see the following for values of the useUncertDB in
different modules in RECO (after expanding the configuration with
edmConfigDump):

process.dt1DCosmicRecHits has False

process.dt2DCosmicSegments has True

process.dt4DCosmicSegments has False

process.dt1DRecHits has True

process.dt2DSegments has True

process.dt4DSegments has True

This doesn't look very consistent for cosmics.

On my side I just have one cfg for cosmics and one for collisions:
https://github.com/cms-sw/cmssw/blob/CMSSW_7_1_X/RecoLocalMuon/DTRecHit/python/DTLinearDriftFromDBAlgo_CosmicData_cfi.py
https://github.com/cms-sw/cmssw/blob/CMSSW_7_1_X/RecoLocalMuon/DTRecHit/python/DTLinearDriftFromDBAlgo_cfi.py

which correspond to first and fourth in your list.

The others come from segment reco, thus I would suspect that
dt2DCosmicSegments is simply not including the right cfi (independently
of this change). In any case, I wonder why do we still have 2D segments
included in the standard config? They should not be used anymore in
production, AFAIK.

I observe some small changes compared to the previous version of the code.
There is no specific pattern in DQM plots (nothing even shows up in the
DT DQM, only starts to appear in the parameters of muon seeds).

I looked at parameters of dt4DSegments event by event about 10% of
segments are different,
localX changes by under 1 micron in most cases, with similarly small
changes in xx error. So, overall, this looks like some round-off issue.

The change just picks hit errors from a different DB table, but numbers
should be the same and I would not expect rounding issues. I aslo
checked distributions for segments and see no difference, altough I did
not check rounding at the order of the micron, which is negligible anyhow.

Cheers
Nicola

—
Reply to this email directly or view it on GitHub
#2993 (comment).

namapane · 2014-03-29T15:56:35Z

indeed the standard reco includes the sequence muonlocalreco:

https://github.com/cms-sw/cmssw/blob/CMSSW_7_1_X/Configuration/StandardSequences/python/Reconstruction_cff.py#L39

and not muonlocalreco_with_2DSegments:

https://github.com/cms-sw/cmssw/blob/CMSSW_7_1_X/RecoLocalMuon/Configuration/python/RecoLocalMuon_cff.py#L38

that is defined for historical reasons, but not used anymore.

Same for ReconstructionCosmics_cff.py and RecoLocalMuonCosmics_cff.py:

https://github.com/cms-sw/cmssw/blob/CMSSW_7_1_X/RecoLocalMuon/Configuration/python/RecoLocalMuonCosmics_cff.py

dt2DCosmicSegments are in fact obsolete. We could either fix their
include chain or even comment that path out, but I'd consider that a
separate issue, to be decoupled from this PR.

Thanks
Nicola

On 29-Mar-14 15:42, slava77 wrote:

Hi Nicola, I see the following for values of the useUncertDB in
different modules in RECO (after expanding the configuration with
edmConfigDump):

process.dt1DCosmicRecHits has False

process.dt2DCosmicSegments has True

process.dt4DCosmicSegments has False

process.dt1DRecHits has True

process.dt2DSegments has True

process.dt4DSegments has True

This doesn't look very consistent for cosmics.

I observe some small changes compared to the previous version of the code.
There is no specific pattern in DQM plots (nothing even shows up in the
DT DQM, only starts to appear in the parameters of muon seeds).

I looked at parameters of dt4DSegments event by event about 10% of
segments are different,
localX changes by under 1 micron in most cases, with similarly small
changes in xx error. So, overall, this looks like some round-off issue.

—
Reply to this email directly or view it on GitHub
#2993 (comment).

slava77 · 2014-03-29T16:11:13Z

Hi Nicola,

I didn't check if the modules actually make it to any executed path.
I confirm now that indeed the _with_2DSegments sequence is loaded but is not run.

OK

slava77 · 2014-03-29T18:45:05Z

+1

for #2993 09b50e3
tested in CMSSW_7_1_X_2014-03-25-1400 (my test area sign329)

small numerical differences are present (see notes above), no particular pattern

cmsbuild · 2014-03-29T18:48:29Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_1_X IBs unless changes (tests are also fine). @nclopezo, @ktf can you please take care of it?

Reco -- Read DT uncertainties map from DB

Add card to control reading the uncertainty map from DB

27f766d

namapane added pending-signatures labels Mar 24, 2014

handle missing useUncertDB card in old configurations

09b50e3

namapane added pending-signatures labels Mar 25, 2014

nclopezo modified the milestones: CMSSW_7_1_0_pre6, CMSSW_7_1_0_pre5 Mar 26, 2014

namapane added comparison-pending labels Mar 28, 2014

namapane added comparison-pending labels Mar 29, 2014

ktf added a commit that referenced this pull request Mar 29, 2014

Merge pull request #2993 from namapane/DTUncertaintiesDB_71X_d

2868fc7

Reco -- Read DT uncertainties map from DB

ktf merged commit 2868fc7 into cms-sw:CMSSW_7_1_X Mar 29, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Read DT uncertainties map from DB #2993

Read DT uncertainties map from DB #2993

namapane commented Mar 24, 2014

cmsbuild commented Mar 24, 2014

cmsbuild commented Mar 24, 2014

namapane commented Mar 24, 2014

rcastello commented Mar 25, 2014

cmsbuild commented Mar 25, 2014

nclopezo commented Mar 25, 2014

cmsbuild commented Mar 25, 2014

slava77 commented Mar 28, 2014

slava77 commented Mar 29, 2014

namapane commented Mar 29, 2014

namapane commented Mar 29, 2014

slava77 commented Mar 29, 2014

slava77 commented Mar 29, 2014

cmsbuild commented Mar 29, 2014

Read DT uncertainties map from DB #2993

Read DT uncertainties map from DB #2993

Conversation

namapane commented Mar 24, 2014

cmsbuild commented Mar 24, 2014

cmsbuild commented Mar 24, 2014

namapane commented Mar 24, 2014

rcastello commented Mar 25, 2014

cmsbuild commented Mar 25, 2014

nclopezo commented Mar 25, 2014

cmsbuild commented Mar 25, 2014

slava77 commented Mar 28, 2014

slava77 commented Mar 29, 2014

namapane commented Mar 29, 2014

namapane commented Mar 29, 2014

slava77 commented Mar 29, 2014

slava77 commented Mar 29, 2014

cmsbuild commented Mar 29, 2014