Fix b tag bug in top hlt dqm offline and validation codes #3803

ebouvier · 2014-05-10T21:41:50Z

In PR #3457, several EDProducers and EDFilters have been removed, breaking the Top PAG HLT DQM Offline code. The plots produced for validation in CMSSW_7_1_0_pre7 are therefore empty. This PR fixes this problem and need to be merged ASAP to make the validation anew possible.
The occasion is also taken:

to change the names of the GUI directories to be finally able to use RelMon
to start the migration from ak5 to ak4 (b tagging being available only for ak5, everything can't be migrated)

…use of btagging

cmsbuild · 2014-05-10T21:42:47Z

A new Pull Request was created by @ebouvier for CMSSW_7_1_X.

Fix b tag bug in top hlt dqm offline and validation codes

It involves the following packages:

DQMOffline/Trigger
HLTriggerOffline/Top

@ojeda, @danduggan, @rovere, @cmsbuild, @nclopezo, @deguio, @Degano can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
@nclopezo, @ktf you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

deguio · 2014-05-11T09:37:53Z

+1
as soon as confirmed by jenkins tests

cmsbuild · 2014-05-11T09:42:45Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_1_X IBs unless changes or unless it breaks tests. @nclopezo, @ktf can you please take care of it?

nclopezo · 2014-05-12T08:02:50Z

Hi @ebouvier,

This pull request is unmergeable, could you please rebase it? You can see instructions here:
http://cms-sw.github.io/cmssw/tutorial-forward-port.html

…use of btagging

cmsbuild · 2014-05-12T09:03:54Z

Pull request #3803 was updated. @ojeda, @danduggan, @rovere, @cmsbuild, @nclopezo, @deguio, @Degano can you please check and sign again.

ebouvier · 2014-05-12T09:07:58Z

Hi @nclopezo,
As usual, I've had some troubles to rebase, but I hope it's mergeable now. Does @deguio need to sign again ?
Cheers.

cmsbuild · 2014-05-12T09:08:14Z

Pull request #3803 was updated. @ojeda, @danduggan, @rovere, @cmsbuild, @nclopezo, @deguio, @Degano can you please check and sign again.

deguio · 2014-05-12T09:25:52Z

+1
again, let's wait for the tests to confirm that everything runs fine.
thanks,
F.

cmsbuild · 2014-05-12T09:28:03Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_1_X IBs unless changes or unless it breaks tests. @nclopezo, @ktf can you please take care of it?

cmsbuild · 2014-05-12T16:37:47Z

Pull request #3803 was updated. @ojeda, @danduggan, @rovere, @cmsbuild, @nclopezo, @deguio, @Degano can you please check and sign again.

ktf · 2014-05-12T18:37:27Z

@ebouvier: This is probably fall out of the ak5 -> ak4 migration.

cmsbuild · 2014-05-12T21:13:36Z

-1
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
----- Begin Fatal Exception 12-May-2014 22:56:39 CEST-----------------------
An exception of category 'NoProxyException' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'validation_step'
   [2] Calling event method for module TopSingleLeptonHLTOfflineDQM/'topSingleMuonHLTOfflineDQM'
Exception Message:
No data of type "JetCorrector" with label "ak4PFL1L2L3" in record "JetCorrectionsRecord"
 Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log
----- Begin Fatal Exception 12-May-2014 23:07:17 CEST-----------------------
An exception of category 'EventRequired' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'dqmoffline_step'
   [2] Calling event method for module TopSingleLeptonHLTOfflineDQM/'topSingleMuonHLTOfflineDQM'
Exception Message:
Wrong interface correction(reco::Jet), event required!
----- End Fatal Exception -------------------------------------------------

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log
----- Begin Fatal Exception 12-May-2014 23:07:33 CEST-----------------------
An exception of category 'EventRequired' occurred while
   [0] Processing run: 1 lumi: 1 event: 9
   [1] Running path 'dqmoffline_step'
   [2] Calling event method for module TopSingleLeptonHLTOfflineDQM/'topSingleMuonHLTOfflineDQM'
Exception Message:
Wrong interface correction(reco::Jet), event required!
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3803/1476/summary.html

ebouvier · 2014-05-13T05:30:30Z

Hi @cmsbuild @nclopezo @deguio,
I remove jetCorrector as long as b-tagging is performed on ak5PFJets. This way, there can't be any problem with jenkins tests (or at least I hope so).
Cheers.

cmsbuild · 2014-05-13T05:33:03Z

Pull request #3803 was updated. @ojeda, @danduggan, @rovere, @cmsbuild, @nclopezo, @deguio, @Degano can you please check and sign again.

deguio · 2014-05-13T09:07:32Z

@ebouvier
you can test locally the changes before pushing them. you can run the very same workflow with:
runTheMatrix.py -l 25.0 (for example)

hope it helps,
F.

ebouvier · 2014-05-13T09:14:54Z

@deguio
I've been testing since the beginning of this PR through:
cmsDriver.py validate -s RAW2DIGI,RECO,VALIDATION,DQM -n 100 --filein /store/relval/CMSSW_7_1_0_pre7/RelValTTbar/GEN-SIM-DIGI-RAW-HLTDEBUG/PRE_STA71_V3-v1/00000/04F8F8A0-3AD1-E311-9EF4-003048679188.root --eventcontent FEVTDEBUGHLT --conditions auto:mc --mc
cmsDriver.py harvest -s HARVESTING:validationHarvesting+dqmHarvesting --harvesting AtRunEnd --conditions auto:mc --filein file:validate_RAW2DIGI_RECO_VALIDATION_DQM.root --mc
but as I'm in CMSSW_7_1_0_pre7, I don't have all the latest changes, especially from JetCorrections.
Anyway, I've removed jetCorrections, so I can't see any other reason for it to fail jenkins tests. Has @cmsbuild launched them ?

deguio · 2014-05-13T10:58:49Z

it is always a good idea to test on top of a working IB or try to reproduce exactly what the jenkins tests do. you can find the daily and nightly builds listed here with the outcome of some basic tests to assess their validity:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/summary-merged-prs/merged_prs.html

thanks for keeping an eye on the code. I am sure that the tests will come soon. @nclopezo
F.

cmsbuild · 2014-05-13T12:16:32Z

-1
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
----- Begin Fatal Exception 13-May-2014 14:00:10 CEST-----------------------
An exception of category 'InvalidReference' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'validation_step'
   [2] Calling event method for module TopSingleLeptonHLTOfflineDQM/'topSingleMuonHLTOfflineDQM'
Exception Message:
AssociationVector: trying to use [] operator passing a reference
 with the wrong product id (i.e.: pointing to the wrong collection)
 keyRef.id = 1:1450, ref.id = 1:1448
----- End Fatal Exception -------------------------------------------------

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log
----- Begin Fatal Exception 13-May-2014 14:10:42 CEST-----------------------
An exception of category 'InvalidReference' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'dqmoffline_step'
   [2] Calling event method for module TopSingleLeptonHLTOfflineDQM/'topSingleMuonHLTOfflineDQM'
Exception Message:
AssociationVector: trying to use [] operator passing a reference
 with the wrong product id (i.e.: pointing to the wrong collection)
 keyRef.id = 3:1178, ref.id = 3:1176
----- End Fatal Exception -------------------------------------------------

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log
----- Begin Fatal Exception 13-May-2014 14:10:54 CEST-----------------------
An exception of category 'InvalidReference' occurred while
   [0] Processing run: 1 lumi: 1 event: 9
   [1] Running path 'dqmoffline_step'
   [2] Calling event method for module TopSingleLeptonHLTOfflineDQM/'topSingleMuonHLTOfflineDQM'
Exception Message:
AssociationVector: trying to use [] operator passing a reference
 with the wrong product id (i.e.: pointing to the wrong collection)
 keyRef.id = 3:1178, ref.id = 3:1176
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3803/1489/summary.html

ebouvier · 2014-05-13T12:22:40Z

@cmsbuild @deguio @nclopezo
Ok, I close this PR, get the latest working IB, and start again.

Elvire Bouvier added 3 commits May 9, 2014 12:12

fix gui organisation and move to ak4 in HLTriggerOffline/Top

1b6ad44

fix gui organisation in DQMOffline/Trigger

5c34e36

fix btagging in DQMOffline/Trigger, only partially moving to ak4 beca…

754cc1b

…use of btagging

ebouvier added comparison-pending labels May 10, 2014

ebouvier added comparison-pending labels May 11, 2014

Elvire Bouvier added 4 commits May 12, 2014 10:36

fix gui organisation and move to ak4 in HLTriggerOffline/Top

67fca9c

fix gui organisation in DQMOffline/Trigger

f31137c

fix btagging in DQMOffline/Trigger, only partially moving to ak4 beca…

8a6f2e7

…use of btagging

merging conflicts

672880d

ebouvier added comparison-pending labels May 12, 2014

now it should be ok for merging

9709b66

ebouvier added the comparison-pending label May 12, 2014

ebouvier added comparison-pending labels May 12, 2014

removing jetCorrector as long as btagging is performed on ak5

4f0f303

ebouvier added comparison-pending labels May 13, 2014

ebouvier closed this May 13, 2014

ebouvier mentioned this pull request May 13, 2014

Ak4 migration and b tag in Top HLT DQM Offline and Validation codes #3844

Merged

ebouvier deleted the FixBTagBugForTopHLT branch September 10, 2014 13:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix b tag bug in top hlt dqm offline and validation codes #3803

Fix b tag bug in top hlt dqm offline and validation codes #3803

ebouvier commented May 10, 2014

cmsbuild commented May 10, 2014

deguio commented May 11, 2014

cmsbuild commented May 11, 2014

nclopezo commented May 12, 2014

cmsbuild commented May 12, 2014

ebouvier commented May 12, 2014

cmsbuild commented May 12, 2014

deguio commented May 12, 2014

cmsbuild commented May 12, 2014

cmsbuild commented May 12, 2014

ktf commented May 12, 2014

cmsbuild commented May 12, 2014

ebouvier commented May 13, 2014

cmsbuild commented May 13, 2014

deguio commented May 13, 2014

ebouvier commented May 13, 2014

deguio commented May 13, 2014

cmsbuild commented May 13, 2014

ebouvier commented May 13, 2014

Fix b tag bug in top hlt dqm offline and validation codes #3803

Fix b tag bug in top hlt dqm offline and validation codes #3803

Conversation

ebouvier commented May 10, 2014

cmsbuild commented May 10, 2014

deguio commented May 11, 2014

cmsbuild commented May 11, 2014

nclopezo commented May 12, 2014

cmsbuild commented May 12, 2014

ebouvier commented May 12, 2014

cmsbuild commented May 12, 2014

deguio commented May 12, 2014

cmsbuild commented May 12, 2014

cmsbuild commented May 12, 2014

ktf commented May 12, 2014

cmsbuild commented May 12, 2014

ebouvier commented May 13, 2014

cmsbuild commented May 13, 2014

deguio commented May 13, 2014

ebouvier commented May 13, 2014

deguio commented May 13, 2014

cmsbuild commented May 13, 2014

ebouvier commented May 13, 2014