JetMET Validation update #1627

schoef · 2013-11-29T11:38:01Z

Reorganization of JetMET Validation code.
*) Major cleanup of all histos/binnings etc.
*) Reorganization of the class structure
*) Included several new collections: type0/1/01 MET, AK5 CHS jets etc.
*) getByLabel ->getByToken

…mmon quantities written consistently, 3) fixed Sam's 'ugly hack', 4) adjusted binning and ranges everywhere

- New API uses iterator pairs rather than container references, adapt everywhere to it. - Get rid of the obsolete API, which is thread unsafe.

…umi, streamId and moduleId info

…ooking

…k package to make it compile

…ia IBooker

cmsbuild · 2013-12-02T12:44:28Z

-1
When I ran the RelVals I found an error in the following worklfows:
401.0 step1

runTheMatrix-results/401.0_TTbarNewMix+TTbarFSPU2+HARVESTFS/step1_TTbarNewMix+TTbarFSPU2+HARVESTFS.log

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1627/1548/summary.html

cmsbuild · 2013-12-02T13:18:06Z

Pull request #1627 was updated. @apfeiffer1, @thspeer, @giamman, @demattia, @eliasron, @civanch, @mdhildreth, @cmsbuild, @nclopezo, @rcastello, @deguio, @slava77, @ggovi, @vadler, @rovere, @danduggan can you please check and sign again.

cmsbuild · 2013-12-02T17:26:11Z

-1
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

401.0 step1

runTheMatrix-results/401.0_TTbarNewMix+TTbarFSPU2+HARVESTFS/step1_TTbarNewMix+TTbarFSPU2+HARVESTFS.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 02-Dec-2013 18:11:21 CET-----------------------
An exception of category 'MultipleExceptions' occurred while
   [0] Calling endJob
Exception Message:
Multiple exceptions were thrown while executing endJob. An exception message follows for each.
----- Exception 1 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'metAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 2 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 3 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'tcMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 4 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'genMetTrueAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 5 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfType0CorrectedMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 6 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfType1CorrectedMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 7 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfType01CorrectedMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- End Fatal Exception -------------------------------------------------

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log
----- Begin Fatal Exception 02-Dec-2013 18:18:51 CET-----------------------
An exception of category 'MultipleExceptions' occurred while
   [0] Calling endJob
Exception Message:
Multiple exceptions were thrown while executing endJob. An exception message follows for each.
----- Exception 1 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'metAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 2 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 3 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'tcMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 4 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'genMetTrueAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 5 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfType0CorrectedMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 6 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfType1CorrectedMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- Exception 7 -----
An exception of category 'MonitorElement' occurred while
   [0] Calling endJob for module METTester/'pfType01CorrectedMetAnalyzer'
Exception Message:
Method 'double MonitorElement::getMean(int) const' cannot be invoked on monitor element '(null)' because it is not a root object
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1627/1555/summary.html

…atrix -l'5.1' was successful.

schoef · 2013-12-02T19:32:39Z

David,
are these replies automatic or does it involve work for you?
-r

cmsbuild · 2013-12-02T19:33:04Z

Pull request #1627 was updated. @apfeiffer1, @thspeer, @giamman, @demattia, @eliasron, @civanch, @mdhildreth, @cmsbuild, @nclopezo, @rcastello, @deguio, @slava77, @ggovi, @vadler, @rovere, @danduggan can you please check and sign again.

nclopezo · 2013-12-03T08:53:59Z

Hi Robert,

I start the tests myself, and when they finish, the message from cmsbuild is automatic.

cmsbuild · 2013-12-03T09:23:39Z

+1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1627/1559/summary.html

apfeiffer1 · 2013-12-03T10:55:44Z

+1

thspeer · 2013-12-03T12:40:23Z

+1
Tested 131ff2e in CMSSW_7_0_X_2013-12-03-0200-1627
No difference in reco, based on RelMon and reco script

civanch · 2013-12-04T11:36:11Z

+1

vadler · 2013-12-04T11:56:09Z

+1

deguio · 2013-12-05T18:43:21Z

+1

rcastello · 2013-12-06T12:25:21Z

+1

cmsbuild · 2013-12-06T12:28:23Z

This pull request is fully signed and it will be integrated in one of the next IBs unless changes or unless it breaks tests. @ktf can you please take care of it?

JetMET Validation update

perrotta · 2013-12-09T09:13:06Z

Validation/RecoJets/plugins/SealModule.cc

@@ -1,18 +1,5 @@
 #include "FWCore/PluginManager/interface/ModuleDef.h"
 #include "FWCore/Framework/interface/MakerMacros.h"

-#include "PFJetTester.h"
-#include "PFJetTesterUnCorr.h"
-#include "CaloJetTester.h"


The removal of this plugin originates the errors in the HI simulation workflows (140.0, 141.0, 143.0, 300.0, 301.0, 302.0) as in
https://cmssdt.cern.ch/SDT/cgi-bin//showMatrixTestLogs.py/slc6_amd64_gcc481/www/sat/7.0-sat-02/CMSSW_7_0_X_2013-12-07-0200/pyRelValMatrixLogs/run/
Please fix either by putting back the plugin, or modifying also the HI validation.

schoef · 2013-12-09T10:08:49Z

Hi, could you tell me how to reproduce HI workflows? Could be, we need to update the sequence.
Should
runTheMatrix.py -l'140.0.'
fail?
Thanks,
Robert

nclopezo · 2013-12-09T10:15:41Z

HI @schoef

yes, more specifically

runTheMatrix.py -l 140.0,141.0,143.0,300.0,301.0,302.0

should fail

schoef · 2013-12-09T12:29:28Z

Hi again,

is there maybe a logfile I can look at? I run the tests since morning but it seems to be very slow.

Thanks,
Robert

perrotta · 2013-12-09T13:09:11Z

Hi Robert.

The original logfiles are accessible through the page of the IB RelVal tests
https://cmssdt.cern.ch/SDT/cgi-bin//showMatrixTestLogs.py/slc6_amd64_gcc481/www/sat/7.0-sat-02/CMSSW_7_0_X_2013-12-07-0200/pyRelValMatrixLogs/run/

Access the link of the last log (the one which has failed) in the
(red) line corresponding to the test you want to examinate.

Andrea

schoef notifications@github.com ha scritto:

Hi again,

is there maybe a logfile I can look at? I run the tests since
morning but it seems to be very slow.

Thanks,
Robert

Reply to this email directly or view it on GitHub:
#1627 (comment)

This message was sent using IMP, the Internet Messaging Program.

schoef · 2013-12-09T13:34:10Z

I'm sorry Andrea,
I can't correlate what I see there with what I'm looking for.

Is it CMSSW_7_0_X_2013-12-07-020? In the 'RelVal' column I see two links
'Details'. I don't see red lines, just red bars.
I followed some of the links provided but I couldn't find any log file.

At some point, I'd expect a msg like 'no module xyz' to appear or a
breaking of the HI validation sequence.

Sorry again,
Best,
Robert

On Mon, Dec 9, 2013 at 2:09 PM, perrotta notifications@github.com wrote:

Hi Robert.

The original logfiles are accessible through the page of the IB RelVal
tests

https://cmssdt.cern.ch/SDT/cgi-bin//showMatrixTestLogs.py/slc6_amd64_gcc481/www/sat/7.0-sat-02/CMSSW_7_0_X_2013-12-07-0200/pyRelValMatrixLogs/run/

Access the link of the last log (the one which has failed) in the
(red) line corresponding to the test you want to examinate.

Andrea

schoef notifications@github.com ha scritto:

Hi again,

is there maybe a logfile I can look at? I run the tests since
morning but it seems to be very slow.

Thanks,
Robert

Reply to this email directly or view it on GitHub:
#1627 (comment)

This message was sent using IMP, the Internet Messaging Program.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/1627#issuecomment-30130415
.

perrotta · 2013-12-09T13:55:42Z

You have something like that:

workflow step1 step2 step3 step4 step5 Comparison

[...]
4 140.0 HydjetQ_MinBias_2760GeV log cmd log cmd log cmd
[...]

(Sorry, I can't make a screenshot now)
If you click the "log" link under step3 you end up in
https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/slc6_amd64_gcc481/CMSSW_7_0_X_2013-12-07-0200/pyRelValMatrixLogs/run/140.0_HydjetQ_MinBias_2760GeV+HydjetQ_MinBias_2760GeVINPUT+DIGIHI+RECOHI+HARVESTHI/step3_HydjetQ_MinBias_2760GeV+HydjetQ_MinBias_2760GeVINPUT+DIGIHI+RECOHI+HARVESTHI.log

schoef · 2013-12-09T16:46:07Z

Hi,
ok, so this needs to be replaced:
http://cmslxr.fnal.gov/lxr/source/Validation/RecoHI/python/JetValidationHeavyIons_cff.py#015

I could do that and possibly provide a fix, but that package doesn't belong
to us.

Should I make a commit with the fix and update the pull request?

Thanks for leading me through the log pages.
-Robert

On Mon, Dec 9, 2013 at 2:55 PM, perrotta notifications@github.com wrote:

You have something like that:
workflow step1 step2 step3 step4 step5 Comparison

[...]
4 140.0 HydjetQ_MinBias_2760GeV log cmd log cmd log cmd
[...]

(Sorry, I can't make a screenshot now)
If you click the "log" link under step3 you end up in

https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/slc6_amd64_gcc481/CMSSW_7_0_X_2013-12-07-0200/pyRelValMatrixLogs/run/140.0_HydjetQ_MinBias_2760GeV+HydjetQ_MinBias_2760GeVINPUT+DIGIHI+RECOHI+HARVESTHI/step3_HydjetQ_MinBias_2760GeV+HydjetQ_MinBias_2760GeVINPUT+DIGIHI+RECOHI+HARVESTHI.log

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/1627#issuecomment-30133294
.

ktf · 2013-12-09T16:55:30Z

http://cmslxr.fnal.gov/lxr/source/Validation/RecoHI/python/JetValidationHeavyIons_cff.py#015

I could do that and possibly provide a fix, but that package doesn't
belong
to us.

Should I make a commit with the fix and update the pull request?

Yes, please, go ahead.

Ciao,
Giulio

schoef · 2013-12-09T19:08:17Z

OK, I updated the tag that went into the pull request.

JetMETdqmval@ce07b69

It doesn't seem to show up here yet.
Any more actions recquired?

The provided fix "should" cure the problem; I can't test it since the runTheMatrix.py test seems to run forever.

Thanks,
Robert

ktf · 2013-12-09T19:32:41Z

Is it a tag or a branch?

schoef · 2013-12-09T19:39:29Z

I pushed to the branch (sorry for the cvs kind of mistake above) I had done the pullrequest for:

https://github.com/JetMETdqmval/cmssw/tree/develop_V10/Validation/RecoJets

schoef and others added 30 commits October 2, 2013 18:46

LetMeTry

f43a89c

added sequence_validation_cfg.py

0844502

Included forgotten JetTester.*

ef8a382

reorganization and rewrite of RecoMET Validation package

b1c394e

modify code to run without JetFileSaver

c83e290

test commit with change of test file

3328aee

MetTester cleanup

087dba7

Small fix

779279d

a few more changes

30e731e

Major changes:1) METType steerable (prev. option too dangerous) 2) co…

08343de

…mmon quantities written consistently, 3) fixed Sam's 'ugly hack', 4) adjusted binning and ranges everywhere

Migrate to the new SiStripFedCabling API for feds.

c549611

- New API uses iterator pairs rather than container references, adapt everywhere to it. - Get rid of the obsolete API, which is thread unsafe.

Jet type now steerable.

91848f5

Added AK5 PF CHS jets -> JEC need fix!

5b3ce56

Included JES for AK5 CHS PF jets.

0658605

Added type0, type1, type01 pfMET

21c4188

Consumes migration

ecfc873

Consumes migration

cba8221

added fragments fro neutron background study in muon systems

b755aae

Fixes suggested by Matthias and Viola (binnings etc.)

a727f90

Fixes suggested by Matthias and Viola (binnings etc.)

160f6e6

Add run, lumi, streamId and moduleID to the MonitorElement class

78a7b20

Add run, streamId and moduleId to the DQMStore

162818f

Adapt DQMStore::findObject to the expanded MonitorElement with run, l…

202c98f

…umi, streamId and moduleId info

Stylistic improvements

900d132

Add first skeleton of DQMStore::IBooker class to manage thread-safe b…

e524d1b

…ooking

Add bookTransaction template function to serialize the booking of MEs

0538272

Fix includes and fwd declarations mess into RecoParticleFlow/Benchmar…

494880c

…k package to make it compile

Add RunSummaryCache and LuminositySummaryCache

862b998

First skeleton of DQMEDAnalyzer

8759b1c

Properly initialize the ibooker_ member and expose setCurrentFolder v…

4fc1084

…ia IBooker

Fixed file name that is used in the global sequence.

a0060e5

Re-introduced Sam's hack. Fixes METTester segfault in endJob. runTheM…

131ff2e

…atrix -l'5.1' was successful.

nclopezo added a commit that referenced this pull request Dec 6, 2013

Merge pull request #1627 from JetMETdqmval/develop_V10

1417416

JetMET Validation update

nclopezo merged commit 1417416 into cms-sw:CMSSW_7_0_X Dec 6, 2013

perrotta reviewed Dec 9, 2013
View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JetMET Validation update #1627

JetMET Validation update #1627

schoef commented Nov 29, 2013

cmsbuild commented Dec 2, 2013

cmsbuild commented Dec 2, 2013

cmsbuild commented Dec 2, 2013

schoef commented Dec 2, 2013

cmsbuild commented Dec 2, 2013

nclopezo commented Dec 3, 2013

cmsbuild commented Dec 3, 2013

apfeiffer1 commented Dec 3, 2013

thspeer commented Dec 3, 2013

civanch commented Dec 4, 2013

vadler commented Dec 4, 2013

deguio commented Dec 5, 2013

rcastello commented Dec 6, 2013

cmsbuild commented Dec 6, 2013

perrotta Dec 9, 2013

schoef commented Dec 9, 2013

nclopezo commented Dec 9, 2013

schoef commented Dec 9, 2013

perrotta commented Dec 9, 2013

schoef commented Dec 9, 2013

perrotta commented Dec 9, 2013

schoef commented Dec 9, 2013

ktf commented Dec 9, 2013

schoef commented Dec 9, 2013

ktf commented Dec 9, 2013

schoef commented Dec 9, 2013

JetMET Validation update #1627

JetMET Validation update #1627

Conversation

schoef commented Nov 29, 2013

cmsbuild commented Dec 2, 2013

cmsbuild commented Dec 2, 2013

cmsbuild commented Dec 2, 2013

schoef commented Dec 2, 2013

cmsbuild commented Dec 2, 2013

nclopezo commented Dec 3, 2013

cmsbuild commented Dec 3, 2013

apfeiffer1 commented Dec 3, 2013

thspeer commented Dec 3, 2013

civanch commented Dec 4, 2013

vadler commented Dec 4, 2013

deguio commented Dec 5, 2013

rcastello commented Dec 6, 2013

cmsbuild commented Dec 6, 2013

perrotta Dec 9, 2013

Choose a reason for hiding this comment

schoef commented Dec 9, 2013

nclopezo commented Dec 9, 2013

schoef commented Dec 9, 2013

perrotta commented Dec 9, 2013

schoef commented Dec 9, 2013

perrotta commented Dec 9, 2013

workflow step1 step2 step3 step4 step5 Comparison

schoef commented Dec 9, 2013

ktf commented Dec 9, 2013

schoef commented Dec 9, 2013

ktf commented Dec 9, 2013

schoef commented Dec 9, 2013