Update Standard Sequences for L1T in 2016 #13271

mulhearn · 2016-02-12T21:16:09Z

Updates the standard sequences for RawToDigi, DigiToRaw, and SimL1Emulation using Eras to correctly configure these sequences based on the Legacy, Stage-1 (2015), or Stage-2 (2016+) L1 trigger.

It adds a new version of L1REPACK (Full) which re-Emulates the entire L1T including HF Trigger Primitives. This is currently only supported for the Stage-2 Era.

The intention is that the new features (supporting Stage-2 configuration via Eras) will not disrupt the legacy and 2015 RelVals, and recent successful tests indicate this is indeed working.

Since it is not yet included in any rel vals, a simple test of the new L1REPACK feature is to run:
cmsDriver.py RepackTest -s L1REPACK:Full,RAW2DIGI --era=Run2_2016 --conditions=auto:run2_data -n 10 --data --filein=/store/data/Run2015D/ZeroBias/RAW/v1/000/260/627/00000/00A76FFA-0C82-E511-B441-02163E01450F.root --no_output --customise=L1Trigger/Configuration/customiseUtils.L1TStage2DigisSummary
which L1REPACK's followed by vanilla RAW2DIGI and a screen dump of the Stage-2 output.

mulhearn · 2016-02-12T21:16:18Z

please test

cmsbuild · 2016-02-12T21:16:31Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11183/console

cmsbuild · 2016-02-12T21:16:32Z

A new Pull Request was created by @mulhearn for CMSSW_8_0_X.

It involves the following packages:

Configuration/StandardSequences
EventFilter/L1TRawToDigi
L1Trigger/Configuration
L1Trigger/L1TCommon
L1Trigger/L1TGlobal

@cmsbuild, @franzoni, @mulhearn, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @GiacomoSguazzoni, @rovere, @VinInn, @Martin-Grunewald, @cerati, @dgulhan this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

cmsbuild · 2016-02-12T23:12:24Z

-1
Tested at: 7784de0
When I ran the RelVals I found an error in the following worklfows:
1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15+HARVESTUP15.log

50202.0 step3

runTheMatrix-results/50202.0_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50/step3_TTbar_13+TTbar_13+DIGIUP15_PU50+RECOUP15_PU50+HARVESTUP15_PU50.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13271/11183/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
e9016ef
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13271/11183/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13271/11183/git-merge-result

mulhearn · 2016-02-12T23:37:42Z

please test

mulhearn · 2016-02-14T19:53:51Z

@Martin-Grunewald this version satisfies the requirement that every parameter has a default value. Parameters in HLT/L1T interface are still singled out (in comments).

cmsbuild · 2016-02-14T22:36:12Z

+1
Tested at: 39b71e8
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13271/11201/summary.html

cmsbuild · 2016-02-14T23:39:55Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13271/11201/summary.html

Martin-Grunewald · 2016-02-15T05:32:49Z

@mulhearn
Great, thanks!

Update Standard Sequences for L1T in 2016

smuzaffar · 2016-02-16T09:11:40Z

@davidlange6 , could this Pr be the cause of few of RelVals failures in 80X IB

  File "/afs/cern.ch/cms/sw/ReleaseCandidates/vol0/slc6_amd64_gcc493/cms/cmssw/CMSSW_8_0_X_2016-02-15-1300/python/Configuration/StandardSequences/SimL1EmulatorDM_cff.py", line 12, in <module>
    simRpcTechTrigDigis.RPCDigiLabel = 'mixData'
NameError: name 'simRpcTechTrigDigis' is not defined

davidlange6 · 2016-02-16T09:30:17Z

Yes, it is.. @mulhearn is fixing

Martin-Grunewald · 2016-02-16T13:26:56Z

Hi

In todays IB CMSSW_8_0_X_2016-02-15-2300 I find the following
error (while CMSSW_8_0_X_2016-02-14-2300 runs fine):

 ################################################################### 
 # WARNING: this module is deprecated.                             # 
 # Please use CondCore.CondDB.CondDB_cfi.py                        # 
 ################################################################### 
L1TRawToDigi Sequence configured for Stage-1 (2015) trigger. 
L1TReco Sequence configured for Stage-1 (2015) trigger. 
WARNING:  L1Trigger/L1TCommon/python/caloStage1LegacyFormatDigis_cfi.py has been deprecated...
WARNING:  please use L1Trigger/L1TCalorimeter/python/caloStage1LegacyFormatDigis_cfi.py
# Conditions read from  CMS_CONDITIONS  via FrontierProd 
16-Feb-2016 07:43:06 CET  Initiating request to open file file:RelVal_HLT_PIon_DATA.root
16-Feb-2016 07:43:06 CET  Successfully opened file file:RelVal_HLT_PIon_DATA.root
disableME1a = 0
%MSG-e TkDetLayers:   ConversionTrackCandidateProducer:conversionTrackCandidates@streamBeginRun  16-Feb-2016 07:43:26 CET Run: 256677
 ForwardDiskSectorBuilderFromDet: Trying to build Petal Wedge from Dets at different z positions !! Delta_z = 0.800217
%MSG
Begin processing the 1st record. Run 256677, Event 397024298, LumiSection 380 at 16-Feb-2016 07:43:30.469 CET
----- Begin Fatal Exception 16-Feb-2016 07:43:40 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing run: 256677 lumi: 380 event: 397024298
   [1] Running path 'dqmoffline_step'
   [2] Calling event method for module L1TCaloRCTToUpgradeConverter/'valRctUpgradeFormatDigis'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<L1CaloEmCand>
Looking for module label: gctDigis
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------
%MSG-w BeamFitter:  AlcaBeamMonitor:AlcaBeamMonitor@endLumi  16-Feb-2016 07:43:40 CET Run: 256677 Lumi: 380
No event read! No Fitting!
%MSG
Another exception was caught while trying to clean up files after the primary fatal exception.

I suspect it to be related to this PR.

Dr15Jones · 2016-02-16T13:45:56Z

L1Trigger/Configuration/python/SimL1Emulator_cff.py

-import L1Trigger.RPCTechnicalTrigger.rpcTechnicalTrigger_cfi
-simRpcTechTrigDigis = L1Trigger.RPCTechnicalTrigger.rpcTechnicalTrigger_cfi.rpcTechnicalTrigger.clone()
+#import L1Trigger.RPCTechnicalTrigger.rpcTechnicalTrigger_cfi
+#simRpcTechTrigDigis = L1Trigger.RPCTechnicalTrigger.rpcTechnicalTrigger_cfi.rpcTechnicalTrigger.clone()


This broke a number of IB RelVal workflows. E.g. see

https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/slc6_amd64_gcc493/CMSSW_8_0_X_2016-02-15-1300/pyRelValMatrixLogs/run/500200.1_ProdZEE_13_pmx50ns+ProdZEE_13+DIGIPRMXUP15_PROD_PU50+RECOPRMXUP15PROD_PU50/step2_ProdZEE_13_pmx50ns+ProdZEE_13+DIGIPRMXUP15_PROD_PU50+RECOPRMXUP15PROD_PU50.log

Martin-Grunewald · 2016-02-16T14:35:07Z

Hmm, in my error case, my tests also works fine in CMSSW_8_0_X_2016-02-15-1100 which
includes this PR - so it can not be this PR - apologies - rather, I guess it must be #13206 instead....

Dr15Jones · 2016-02-16T14:38:40Z

#13206 didn't get into the IB until CMSSW_8_0_X_2016-02-15-2300 but the failures started in the IB before that (CMSSW_8_0_X_2016-02-15-1300) which is where this pull request came in.

@Martin-Grunewald notice that there was an additional IB run (1300 after 1100) after your test.

Martin-Grunewald · 2016-02-16T14:55:43Z

@Dr15Jones
Yes, you are correct: -1100 is fine, -1300 shows also "my" error, so it is not #13206...

mulhearn · 2016-02-16T15:56:31Z

@Martin-Grunewald can you tell me what work flow failed in IB so that I can investigate?

Dr15Jones · 2016-02-16T16:01:24Z

All the tests that returned 256 failed because of the python configuration problem

https://cms-sw.github.io/relvalLogDetail.html#slc6_amd64_gcc493;CMSSW_8_0_X_2016-02-15-1300

Martin-Grunewald · 2016-02-16T16:03:26Z

@mulhearn - it is one in our TSG tests (have not checked relval IB tests),
but you can see it also in the "HLT validation" tests (run with each IB) of the -1300 IB build:

https://cmssdt.cern.ch/SDT/jenkins-artifacts/HLT-Validation/CMSSW_8_0_X_2016-02-15-1300/slc6_amd64_gcc493/

look at the RelVal_RECO_GRun_DATA.log file:

https://cmssdt.cern.ch/SDT/jenkins-artifacts/HLT-Validation/CMSSW_8_0_X_2016-02-15-1300/slc6_amd64_gcc493/RelVal_RECO_GRun_DATA.log

Martin-Grunewald · 2016-02-16T16:05:10Z

BTW, "my" problem is not fixed by #13298 ... and has exit status 65, not 256...

Martin-Grunewald · 2016-02-16T16:44:34Z

@mulhearn
Recipe to run the HLT-Validation subset showing the error:

...developer area...
cd src
cmsenv
git cms-addpkg HLTrigger/Configuration
scram b -j 4
cd HLTrigger/Configuration/test
cp /afs/cern.ch/user/g/gruen/public/cmsDriver.csh .
cp /afs/cern.ch/user/g/gruen/public/runOne.csh .
cp /afs/cern.ch/user/g/gruen/public/runAll.csh .
./runAll.csh # go for a coffee

the above script generates several cfg files and then runs some of them leading to the observed error.

mulhearn · 2016-02-16T22:17:21Z

I can reproduce it. It is a problem with "DQMOffline.Configuration.DQMOffline_cff"...

mulhearn · 2016-02-16T22:28:04Z

Fine... OK... I'll just unpack everything always...

mulhearn · 2016-02-16T23:02:02Z

OK, fix is here: #13310

Martin-Grunewald · 2016-02-17T07:54:37Z

@mulhearn
Thanks, this fixes the problem (tested #13310 and #13311 together).

mulhearn added 11 commits February 12, 2016 21:36

update L1T standard sequences

64f44e9

cleanup summary module and related customs

1dbaf20

add ReEmul customization for cmsDriver workflows and more utilities

f2e9c75

disable GMT regional muon packing until fixed

30054b1

bugfix for standard sequences for pre6

33fb04a

remove unsupported examples

8f91a02

fix l1t text summary in example code

7c6a6cc

add test of HLT unpack sequence

ea2c235

Merged l1t-hlt-80x from repository cms-l1t-offline

e9016ef

fix typo in Legacy RawToDigi

b34073f

force checkout of L1TGlobal... missing dependency not understood

7784de0

cmsbuild added this to the Next CMSSW_8_0_X milestone Feb 12, 2016

cmsbuild added l1-pending operations-pending pending-signatures tests-pending orp-pending comparison-pending labels Feb 12, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 12, 2016

workaround crashes from legacy vs new workflows

d213eb0

cmsbuild added tests-rejected tests-pending and removed tests-started tests-rejected labels Feb 12, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 14, 2016

cmsbuild added tests-approved and removed tests-started labels Feb 14, 2016

cmsbuild added comparison-available and removed comparison-pending labels Feb 14, 2016

davidlange6 added a commit that referenced this pull request Feb 15, 2016

Merge pull request #13271 from cms-l1t-offline/l1t-hlt-80x

c3f353c

Update Standard Sequences for L1T in 2016

davidlange6 merged commit c3f353c into cms-sw:CMSSW_8_0_X Feb 15, 2016

diguida mentioned this pull request Feb 15, 2016

Further cleanup of Condition access in AlCaDB, online DQM, Geometry, HLT, Simulation, and standard configuration; Further fixes for online DQM #13170

Closed

Dr15Jones reviewed Feb 16, 2016
View reviewed changes

mulhearn mentioned this pull request Feb 16, 2016

Debugging Cycle For 80x cms-l1t-offline/cmssw#145

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Standard Sequences for L1T in 2016 #13271

Update Standard Sequences for L1T in 2016 #13271

mulhearn commented Feb 12, 2016

mulhearn commented Feb 12, 2016

cmsbuild commented Feb 12, 2016

cmsbuild commented Feb 12, 2016

cmsbuild commented Feb 12, 2016

mulhearn commented Feb 12, 2016

mulhearn commented Feb 14, 2016

cmsbuild commented Feb 14, 2016

cmsbuild commented Feb 14, 2016

Martin-Grunewald commented Feb 15, 2016

smuzaffar commented Feb 16, 2016

davidlange6 commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Dr15Jones Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Dr15Jones commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

mulhearn commented Feb 16, 2016

Dr15Jones commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

mulhearn commented Feb 16, 2016

mulhearn commented Feb 16, 2016

mulhearn commented Feb 16, 2016

Martin-Grunewald commented Feb 17, 2016

Update Standard Sequences for L1T in 2016 #13271

Update Standard Sequences for L1T in 2016 #13271

Conversation

mulhearn commented Feb 12, 2016

mulhearn commented Feb 12, 2016

cmsbuild commented Feb 12, 2016

cmsbuild commented Feb 12, 2016

cmsbuild commented Feb 12, 2016

mulhearn commented Feb 12, 2016

mulhearn commented Feb 14, 2016

cmsbuild commented Feb 14, 2016

cmsbuild commented Feb 14, 2016

Martin-Grunewald commented Feb 15, 2016

smuzaffar commented Feb 16, 2016

davidlange6 commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Dr15Jones Feb 16, 2016

Choose a reason for hiding this comment

Martin-Grunewald commented Feb 16, 2016

Dr15Jones commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

mulhearn commented Feb 16, 2016

Dr15Jones commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

Martin-Grunewald commented Feb 16, 2016

mulhearn commented Feb 16, 2016

mulhearn commented Feb 16, 2016

mulhearn commented Feb 16, 2016

Martin-Grunewald commented Feb 17, 2016