fix tracking HLT customization #13207

mtosi · 2016-02-06T06:38:23Z

this PR fixes previous issues w/ customization function needed for new template at HLT

cmsbuild · 2016-02-06T06:38:46Z

A new Pull Request was created by @mtosi (mia tosi) for CMSSW_8_0_X.

It involves the following packages:

HLTrigger/Configuration

@Martin-Grunewald, @perrotta, @cmsbuild, @davidlange6, @fwyzard can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @geoff-smith, @jalimena this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

fwyzard · 2016-02-06T06:48:19Z

HLTrigger/Configuration/python/customizeHLTfor2016trackingTemplate.py


    return process

 def customiseFor2016trackingTemplate(process):
    process = bug_fix(process)
    process = new_selector(process)
+    new_selector(process)


this should be

process = new_selector(process)

fwyzard · 2016-02-06T06:49:53Z

for the record, I still strongly believe that we should not use these customisations, and have the actual configuration for 2016 already in the ConfDB-generated dumps.

Martin-Grunewald · 2016-02-06T07:17:55Z

@fwyzard
Of course. This is just intermediate.

Martin-Grunewald · 2016-02-06T07:18:15Z

please test

cmsbuild · 2016-02-06T07:18:29Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11037/console

cmsbuild · 2016-02-06T08:54:03Z

-1
Tested at: 4169b7d
When I ran the RelVals I found an error in the following worklfows:
135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log
----- Begin Fatal Exception 06-Feb-2016 08:34:47 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'HLT_PFJet40_v4'
   [2] Calling event method for module TrackClusterRemover/'hltIter1ClustersRefRemoval'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edmNew::DetSetVector
Looking for module label: hltSiStripRawToClustersFacility
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13207/11037/summary.html

Martin-Grunewald · 2016-02-08T14:59:28Z

@mtosi
You also must fix the issue mentioned by Andrea B. above!

cmsbuild · 2016-02-08T15:13:22Z

Pull request #13207 was updated. @Martin-Grunewald, @perrotta, @cmsbuild, @davidlange6, @fwyzard can you please check and sign again.

Martin-Grunewald · 2016-02-08T15:17:24Z

please test

cmsbuild · 2016-02-08T15:17:42Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11069/console

cmsbuild · 2016-02-08T15:33:26Z

Pull request #13207 was updated. @Martin-Grunewald, @perrotta, @cmsbuild, @davidlange6, @fwyzard can you please check and sign again.

Martin-Grunewald · 2016-02-08T16:23:16Z

@mtosi
Remove all print statements everywhere!

Martin-Grunewald · 2016-02-08T16:24:45Z

HLTrigger/Configuration/python/customizeHLTforCMSSW.py

+        from Configuration.StandardSequences.Eras import eras
+        if not eras.fastSim.isChosen():
+            process = customiseFor2016trackingTemplate(process)
+            pass


This solution is incorrect: once we move the 2016 tracking into ConfDB, you must handle fastsim based on that tracking as it will have been applied in ConfDB's config!

cmsbuild · 2016-02-08T17:10:38Z

-1
Tested at: 3529958
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

8.0 step2

runTheMatrix-results/8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step2_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log

9.0 step2

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step2_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step2

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step2_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log
----- Begin Fatal Exception 08-Feb-2016 16:39:56 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'HLT_PFJet40_v4'
   [2] Calling event method for module TrackClusterRemover/'hltIter1ClustersRefRemoval'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edmNew::DetSetVector
Looking for module label: hltSiStripRawToClustersFacility
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

50202.0 step2

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step2_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13207/11069/summary.html

Martin-Grunewald · 2016-02-09T07:57:42Z

please test
(for "final patch")

cmsbuild · 2016-02-09T07:58:02Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11084/console
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11094/console

Martin-Grunewald · 2016-02-09T15:47:13Z

@smuzaffar
jenkins tests of this PR seem to be stuck, can you please have a look? Thanks!

smuzaffar · 2016-02-09T15:56:08Z

@Martin-Grunewald , yes looks like it was stuck. i have re-started it and now it looks good (already runign short matrix now).

cmsbuild · 2016-02-09T17:23:52Z

-1
Tested at: ba9eca5
When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log
----- Begin Fatal Exception 09-Feb-2016 17:06:34 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 1 lumi: 1 event: 1
   [1] Running path 'HLT_PFJet40_v4'
   [2] Calling event method for module ChargedRefCandidateProducer/'hltTrackIter0RefsForJets4Iter1'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for a container with elements of type: reco::Track
Looking for module label: hltIter0PFlowTrackSelectionHighPurity
Looking for productInstanceName: 
   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.
----- End Fatal Exception -------------------------------------------------

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log
----- Begin Fatal Exception 09-Feb-2016 17:46:23 CET-----------------------
An exception of category 'FallbackFileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
   [2] Calling RootFileSequenceBase::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling XrdFile::open()
Exception Message:
Failed to open the file 'root://xrootd-cms.infn.it//store/relval/CMSSW_7_6_0_pre7/RelValMinBias_13/GEN-SIM/76X_mcRun2_asymptotic_v9_realBS-v1/00000/1E73E3F5-407C-E511-81C2-0025905B85B2.root'
   Additional Info:
      [a] XrdCl::File::Open(name='root://eoscms.cern.ch//eos/cms/store/relval/CMSSW_7_6_0_pre7/RelValMinBias_13/GEN-SIM/76X_mcRun2_asymptotic_v9_realBS-v1/00000/1E73E3F5-407C-E511-81C2-0025905B85B2.root', flags=0x10, permissions=0660) => error '[FATAL] Connection error' (errno=0, code=108)
      [b] Remote server already encountered a fatal error; no redirections were performed.
      [c] Input file root://eoscms.cern.ch//eos/cms/store/relval/CMSSW_7_6_0_pre7/RelValMinBias_13/GEN-SIM/76X_mcRun2_asymptotic_v9_realBS-v1/00000/1E73E3F5-407C-E511-81C2-0025905B85B2.root could not be opened.
Fallback Input file root://xrootd-cms.infn.it//store/relval/CMSSW_7_6_0_pre7/RelValMinBias_13/GEN-SIM/76X_mcRun2_asymptotic_v9_realBS-v1/00000/1E73E3F5-407C-E511-81C2-0025905B85B2.root also could not be opened.
Original exception info is above; fallback exception info is below.
      [d] XrdCl::File::Open(name='root://xrootd-cms.infn.it//store/relval/CMSSW_7_6_0_pre7/RelValMinBias_13/GEN-SIM/76X_mcRun2_asymptotic_v9_realBS-v1/00000/1E73E3F5-407C-E511-81C2-0025905B85B2.root', flags=0x10, permissions=0660) => error '[ERROR] Operation expired' (errno=0, code=206). No additional data servers were found.
      [e] Last URL tried: root://xrootd.unl.edu:1094//store/relval/CMSSW_7_6_0_pre7/RelValMinBias_13/GEN-SIM/76X_mcRun2_asymptotic_v9_realBS-v1/00000/1E73E3F5-407C-E511-81C2-0025905B85B2.root?tried=+1213llrxrd-redir.in2p3.fr,
      [f] Problematic data server: xrootd.unl.edu:1094
      [g] Disabled source: xrootd.unl.edu:1094
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13207/11094/summary.html

Martin-Grunewald · 2016-02-15T17:02:56Z

@mtosi
Please close this PR as it is no longer needed and also superseeded.

mtosi · 2016-02-15T17:49:49Z

the needed changes have been integrated -succefully- in confDB and they should be available in PR #13270 thanks andrea and martin)
=> this PR can be closed

fix tracking HLT customization

4169b7d

cmsbuild added this to the Next CMSSW_8_0_X milestone Feb 6, 2016

cmsbuild added hlt-pending pending-signatures tests-pending orp-pending comparison-pending labels Feb 6, 2016

fwyzard reviewed Feb 6, 2016
View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Feb 6, 2016

cmsbuild added tests-rejected and removed tests-started labels Feb 6, 2016

delete not needed line

6c9200e

fisrt patch

3529958

cmsbuild added tests-pending and removed tests-rejected labels Feb 8, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 8, 2016

final patch (?)

ba9eca5

cmsbuild removed the tests-started label Feb 8, 2016

cmsbuild added the tests-pending label Feb 8, 2016

Martin-Grunewald reviewed Feb 8, 2016
View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Feb 9, 2016

cmsbuild added tests-rejected and removed tests-started labels Feb 9, 2016

mtosi closed this Feb 15, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix tracking HLT customization #13207

fix tracking HLT customization #13207

mtosi commented Feb 6, 2016

cmsbuild commented Feb 6, 2016

fwyzard Feb 6, 2016

fwyzard commented Feb 6, 2016

Martin-Grunewald commented Feb 6, 2016

Martin-Grunewald commented Feb 6, 2016

cmsbuild commented Feb 6, 2016

cmsbuild commented Feb 6, 2016

Martin-Grunewald commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

Martin-Grunewald commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

Martin-Grunewald commented Feb 8, 2016

Martin-Grunewald Feb 8, 2016

cmsbuild commented Feb 8, 2016

Martin-Grunewald commented Feb 9, 2016

cmsbuild commented Feb 9, 2016

Martin-Grunewald commented Feb 9, 2016

smuzaffar commented Feb 9, 2016

cmsbuild commented Feb 9, 2016

Martin-Grunewald commented Feb 15, 2016

mtosi commented Feb 15, 2016

fix tracking HLT customization #13207

fix tracking HLT customization #13207

Conversation

mtosi commented Feb 6, 2016

cmsbuild commented Feb 6, 2016

fwyzard Feb 6, 2016

Choose a reason for hiding this comment

fwyzard commented Feb 6, 2016

Martin-Grunewald commented Feb 6, 2016

Martin-Grunewald commented Feb 6, 2016

cmsbuild commented Feb 6, 2016

cmsbuild commented Feb 6, 2016

Martin-Grunewald commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

Martin-Grunewald commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

cmsbuild commented Feb 8, 2016

Martin-Grunewald commented Feb 8, 2016

Martin-Grunewald Feb 8, 2016

Choose a reason for hiding this comment

cmsbuild commented Feb 8, 2016

Martin-Grunewald commented Feb 9, 2016

cmsbuild commented Feb 9, 2016

Martin-Grunewald commented Feb 9, 2016

smuzaffar commented Feb 9, 2016

cmsbuild commented Feb 9, 2016

Martin-Grunewald commented Feb 15, 2016

mtosi commented Feb 15, 2016