Update to L1T Global for 2016 Menu (81X) #14282

mulhearn · 2016-04-27T19:59:02Z

This brings the updated uGT software with bitwise correct emulation of correlation conditions.

It also calculates external conditions using the menu to determine the bit locations.

It uses new conditions, but in a way that allows the HLT configuration to either remain unchanged or update their configs to the new (temporary, pre-O2O) interface as they see fit.

I'll cleanup the history, leaving final product unchanged, while tests are running.

This replaces #14215.

cmsbuild · 2016-04-27T19:59:29Z

A new Pull Request was created by @mulhearn for CMSSW_8_1_X.

It involves the following packages:

DQM/L1TMonitor
DataFormats/L1TGlobal
EventFilter/L1TRawToDigi
HLTrigger/HLTfilters
L1Trigger/L1TCommon
L1Trigger/L1TGlobal

@perrotta, @cmsbuild, @dmitrijus, @Martin-Grunewald, @rekovic, @deguio, @fwyzard, @vanbesien, @mulhearn, @davidlange6 can you please review it and eventually sign? Thanks.
@Martin-Grunewald this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

mulhearn · 2016-04-27T20:02:09Z

please test

cmsbuild · 2016-04-27T20:02:50Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12681/console

cmsbuild · 2016-04-27T21:34:39Z

-1

Tested at: c991b58

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
c991b58
54efe54
3df37a6
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12681/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12681/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12681/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

RelVals:

When I ran the RelVals I found an error in the following worklfows:
134.911 step2

runTheMatrix-results/134.911_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_2016+RECODR2_2016reHLT+HARVESTDR2/step2_RunSinglePh2015D+RunSinglePh2015D+HLTDR2_2016+RECODR2_2016reHLT+HARVESTDR2.log

1306.0 step2

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step2_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step2

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step2_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

25202.0 step2

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step2_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

AddOn:

I found errors in the following addon tests:

cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:29:31 2016-date Wed Apr 27 23:18:48 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:29:31 2016-date Wed Apr 27 23:18:48 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:29:12 2016-date Wed Apr 27 23:18:49 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --data --scenario=HeavyIons -n 10 --conditions auto:run2_data_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_DATA.root --fileout file:RelVal_Raw_HIon_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:29:12 2016-date Wed Apr 27 23:18:49 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:34:09 2016-date Wed Apr 27 23:18:52 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_MC.root --fileout file:RelVal_Raw_GRun_MC_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:34:09 2016-date Wed Apr 27 23:18:52 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PRef.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:33:03 2016-date Wed Apr 27 23:18:56 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_MC.root --fileout file:RelVal_Raw_PRef_MC_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:33:03 2016-date Wed Apr 27 23:18:56 2016 s - exit: 16640
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,HLT:@relval2016,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 --magField 38T_PostLS1 : FAILED - time: date Wed Apr 27 23:25:05 2016-date Wed Apr 27 23:19:01 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:29:17 2016-date Wed Apr 27 23:19:06 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:29:17 2016-date Wed Apr 27 23:19:06 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_HIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:32:55 2016-date Wed Apr 27 23:19:13 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:HIon,RAW2DIGI,L1Reco,RECO --mc --scenario=HeavyIons -n 10 --conditions auto:run2_mc_HIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016,Run2_HI --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_HIon_MC.root --fileout file:RelVal_Raw_HIon_MC_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:32:55 2016-date Wed Apr 27 23:19:13 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_GRun.py realData=True globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:30:31 2016-date Wed Apr 27 23:19:25 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_25ns --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:30:31 2016-date Wed Apr 27 23:19:25 2016 s - exit: 16640
cmsRun /cvmfs/cms-ib.cern.ch/week0/slc6_amd64_gcc530/cms/cmssw/CMSSW_8_1_X_2016-04-26-2300/src/HLTrigger/Configuration/test/OnLine_HLT_PIon.py realData=False globalTag=@ inputFiles=@ : FAILED - time: date Wed Apr 27 23:33:00 2016-date Wed Apr 27 23:19:34 2016 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --mc --scenario=pp -n 10 --conditions auto:run2_mc_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_MC.root --fileout file:RelVal_Raw_PIon_MC_HLT_RECO.root : FAILED - time: date Wed Apr 27 23:33:00 2016-date Wed Apr 27 23:19:34 2016 s - exit: 16640

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
c991b58
54efe54
3df37a6
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12681/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12681/git-merge-result

…ExtBlk

Conflicts: L1Trigger/L1TGlobal/interface/CorrelationTemplate.h L1Trigger/L1TGlobal/interface/L1TGlobalUtil.h L1Trigger/L1TGlobal/plugins/GtProducer.cc L1Trigger/L1TGlobal/plugins/GtProducer.h L1Trigger/L1TGlobal/plugins/GtRecordDump.cc L1Trigger/L1TGlobal/plugins/TriggerMenuXmlParser.cc L1Trigger/L1TGlobal/src/CorrCondition.cc L1Trigger/L1TGlobal/src/CorrelationTemplate.cc L1Trigger/L1TGlobal/src/GtBoard.cc L1Trigger/L1TGlobal/src/L1TGlobalUtil.cc L1Trigger/L1TGlobal/src/MuCondition.cc L1Trigger/L1TGlobal/test/l1tGlobalEmulation_FromDigi.py L1Trigger/L1TGlobal/test/l1tGlobalEmulation_FromRAW.py L1Trigger/L1TGlobal/test/runGlobalFakeInputProducer.py

…umper

…wable window for uGT

mulhearn · 2016-04-28T01:31:25Z

HLTrigger/Configuration/python/customizeHLTforCMSSW.py

@@ -24,6 +24,13 @@ def esproducers_by_type(process, *types):
 #                     pset.minGoodStripCharge = cms.PSet(refToPSet_ = cms.string('HLTSiStripClusterChargeCutNone'))
 #     return process

+def customiseFor14282(process):


Because O2O is based on a new CondFormat, and L1T O2O is not yet ready, it seems some change to HLT configs is unavoidable. The only thing missing is this EmptyESSource.

For 81X this is fine, but: This will not work online as online does not use this file!
We need to plan a proper migration for 80X.
Would the added ES source spoil things if already used without this PR?
Seems fine to add it, so I'll put it into the menu in ConfDB itself and update the HLT PR.

No problem: now that #14201 is in 80X, you can immediately start producing the Empty source as here and everyone will ignore it until this PR is also back-ported to 80x.

cmsbuild · 2016-04-28T03:28:04Z

+1
Tested at: bff9816
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12682/summary.html

cmsbuild · 2016-04-28T04:19:13Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14282/12682/summary.html

Martin-Grunewald · 2016-04-28T06:50:59Z

+1

mulhearn · 2016-04-28T07:47:58Z

+1

cmsbuild · 2016-04-28T08:04:49Z

Pull request #14282 was updated. @perrotta, @cmsbuild, @dmitrijus, @Martin-Grunewald, @rekovic, @deguio, @fwyzard, @vanbesien, @mulhearn, @davidlange6 can you please check and sign again.

mulhearn · 2016-04-28T08:04:51Z

Minor update to remove debug flag from Makefile.

davidlange6 · 2016-04-28T08:15:22Z

DataFormats/L1TGlobal/src/GlobalObject.cc

+    l1t::GlobalObject value = (GlobalObject) - 1;
+
+    bool found = false;
+    for (int i = 0; l1TGtObjectStringToEnumMap[i].label && (!found); ++i)


much cleaner to replace the found statements by a continue instead the if statement

cmsbuild added this to the Next CMSSW_8_1_X milestone Apr 27, 2016

cmsbuild added dqm-pending l1-pending hlt-pending pending-signatures tests-pending orp-pending comparison-pending labels Apr 27, 2016

mulhearn mentioned this pull request Apr 27, 2016

Update to L1T Global for 2016 Menu (81X) - No Conditions Changes #14215

Closed

cmsbuild added tests-started and removed tests-pending labels Apr 27, 2016

cmsbuild added tests-rejected and removed tests-started labels Apr 27, 2016

puigh and others added 13 commits April 28, 2016 00:21

Convert external conditions from legacy GT data to stage 2 uGT Global…

ff0f8bc

…ExtBlk

New uGT ESProducer and bugfixes for CondFormats

baf1392

move access of menu to GtProducer

f33d863

Include testing of L1TUtmTriggerMenuESProducer and L1TUtmTriggerMenuD…

bf2db23

…umper

Now working with L1TUtm Menu, but with a recast

1349f68

Include protection against object data coming from bx outside of allo…

a31b0b7

…wable window for uGT

Some code cleanup

8088b90

Clean out old header files that are no longer used

c0034ae

fix and cleanup standard L1TGlobal python configuration

8ac8564

extend object map to include object types and fill them in emulator

3ea6912

Fix up standard test to align with new python configs

940e031

Changes for new L1TGtObject

b56a9b5

cmsbuild added tests-started and removed tests-pending labels Apr 28, 2016

mulhearn reviewed Apr 28, 2016
View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Apr 28, 2016

cmsbuild added comparison-available and removed comparison-pending labels Apr 28, 2016

cmsbuild added hlt-approved and removed hlt-pending labels Apr 28, 2016

cmsbuild added l1-approved and removed l1-pending labels Apr 28, 2016

remove debugging lines from L1TGlobal buildfile

04afbfa

cmsbuild added l1-pending hlt-pending tests-pending comparison-pending and removed comparison-available hlt-approved l1-approved tests-approved labels Apr 28, 2016

davidlange6 reviewed Apr 28, 2016
View reviewed changes

davidlange6 merged commit 5e996cd into cms-sw:CMSSW_8_1_X Apr 28, 2016

This was referenced Apr 28, 2016

Merge Latest uGT Emulation (included bitwise correlation conditions) into CMSSW cms-l1t-offline/cmssw#261

Closed

Update to L1T Global for 2016 Menu (80X) #14290

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to L1T Global for 2016 Menu (81X) #14282

Update to L1T Global for 2016 Menu (81X) #14282

mulhearn commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mulhearn commented Apr 27, 2016

cmsbuild commented Apr 27, 2016 •

edited

cmsbuild commented Apr 27, 2016

mulhearn Apr 28, 2016

Martin-Grunewald Apr 28, 2016 •

edited

mulhearn Apr 28, 2016

cmsbuild commented Apr 28, 2016

cmsbuild commented Apr 28, 2016

Martin-Grunewald commented Apr 28, 2016

mulhearn commented Apr 28, 2016

cmsbuild commented Apr 28, 2016

mulhearn commented Apr 28, 2016

davidlange6 Apr 28, 2016

Update to L1T Global for 2016 Menu (81X) #14282

Update to L1T Global for 2016 Menu (81X) #14282

Conversation

mulhearn commented Apr 27, 2016

cmsbuild commented Apr 27, 2016

mulhearn commented Apr 27, 2016

cmsbuild commented Apr 27, 2016 • edited

cmsbuild commented Apr 27, 2016

mulhearn Apr 28, 2016

Choose a reason for hiding this comment

Martin-Grunewald Apr 28, 2016 • edited

Choose a reason for hiding this comment

mulhearn Apr 28, 2016

Choose a reason for hiding this comment

cmsbuild commented Apr 28, 2016

cmsbuild commented Apr 28, 2016

Martin-Grunewald commented Apr 28, 2016

mulhearn commented Apr 28, 2016

cmsbuild commented Apr 28, 2016

mulhearn commented Apr 28, 2016

davidlange6 Apr 28, 2016

Choose a reason for hiding this comment

cmsbuild commented Apr 27, 2016 •

edited

Martin-Grunewald Apr 28, 2016 •

edited