ALCARECO HLT-selection streamlining AND shorten AfterAbortGap=>AGG

[franzoni]

• names longer than Tier0 can handle (see: https://hypernews.cern.ch/HyperNews/CMS/get/tier0-Ops/1439/2/1/1/1/1/2/1/1/1/1/1/2/1/1.html ) of two Alcarecos have been shortened:
  . SiStripCalMinBiasAfterAbortGap=>SiStripCalMinBiasAAG
  . and SiStripCalMinBiasAfterAbortGapHI=>SiStripCalMinBiasAAGHI

• HTL-results filtering
  HLTrigger.HLTfilters.triggerResultsFilter_cfi
  included in the sequence of all alcareco, streamlining the name of the corresponding alcareco trigger bits key, and setting to "*" the bits wherever there was no selection. This will ease operation by allowing update of inputs to AlCARECO w/o needing a patch release

• AlCaRecoTriggerBitsRcd of all MC GT's has been updated to AlCaRecoHLTpaths_2017_MC_v1 ( was: AlCaRecoHLTpaths5e32_v19_mc ), to to modify the existing keys (AAG) and including keys wherever HLT filtering introduced anew

• AlCaRecoTriggerBitsRcd of all data GT's has been updated to AlCaRecoHLTpaths_2017_v4 (was: AlCaRecoHLTpaths8e29_1e31_v24_offline) to to modify the existing keys (AAG) and including keys wherever HLT filtering introduced anew . Only the prompt-like GT was originally set to a different tag ( AlCaRecoHLTpaths8e29_5e33_v2_prompt i.e. the tag consumed in prompt), which will be updated with a new IOV before we'll deploy this PR at T0

Work done by @trtomei @ghellwig @tocheng, and @mmusich

Local tests completed successfully:

runTheMatrix.py -l 1001.0 --command='-n 1' >& 1001.a.log  &
runTheMatrix.py -l 1000.0 --command='-n 1' >& 1000.a.log  &
runTheMatrix.py   -l 25,10024,10042,10824 -j 2 --command='-n 1' >& thiagos-updates.2.log &
runTheMatrix.py  --what standard -l 134.813,136.733,136.769,136.77,136.772 -j 2 --command='-n 1' >& tonngguang-updates.1.log &

[franzoni]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/19852/console Started: 2017/05/15 22:16

[cmsbuild]

A new Pull Request was created by @franzoni (Giovanni Franzoni) for master.

It involves the following packages:

Alignment/CommonAlignmentProducer
CalibMuon/DTCalibration
CalibTracker/SiPixelLorentzAngle
CalibTracker/SiStripChannelGain
Calibration/EcalAlCaRecoProducers
Calibration/HcalAlCaRecoProducers
Calibration/TkAlCaRecoProducers
Configuration/AlCa
Configuration/EventContent
Configuration/PyReleaseValidation
Configuration/StandardSequences
DQMOffline/CalibTracker
DQMOffline/Configuration

@ghellwig, @dmitrijus, @kkousour, @arunhep, @cerminar, @fabozzi, @cmsbuild, @franzoni, @kpedro88, @GurpreetSinghChahal, @mmusich, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @pakhotin, @makortel, @felicepantaleo, @GiacomoSguazzoni, @gbenelli, @tocheng, @VinInn, @Martin-Grunewald, @jlagram, @dkotlins, @ebrondol, @rovere, @tlampen, @mschrode, @OlivierBondu, @mmusich, @argiro, @threus, @dgulhan, @rociovilar this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

-1

Tested at: d21369b

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
2bdd8b2
af858f4
5fc01ba
058409b
5b874cd
cbdf389
9711d54
3c1f529
e29b6c1
0c2604c
6188af6
2f4c392
d67ce63
7a463e8
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18753/19852/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18753/19852/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18753/19852/summary.html

I found follow errors while testing this PR

Failed tests: RelVals AddOn

• RelVals:

When I ran the RelVals I found an error in the following worklfows:
5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log

135.4 step1

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

• AddOn:

I found errors in the following addon tests:

cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Mon May 15 23:49:20 2017-date Mon May 15 23:48:30 2017 s - exit: 21248
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns --magField 38T_PostLS1 : FAILED - time: date Mon May 15 23:49:40 2017-date Mon May 15 23:48:47 2017 s - exit: 21248
cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 --magField 38T_PostLS1 : FAILED - time: date Mon May 15 23:50:59 2017-date Mon May 15 23:49:13 2017 s - exit: 21248

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
2bdd8b2
af858f4
5fc01ba
058409b
5b874cd
cbdf389
9711d54
3c1f529
e29b6c1
0c2604c
6188af6
2f4c392
d67ce63
7a463e8
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18753/19852/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18753/19852/git-merge-result

[cmsbuild]

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

[franzoni]

no unexpected changes in relval tests:

• express: 1001.0_RunMinBias2011A missing relmon
• prompt: 1000.0_RunMinBias2011A
• 13A6.731_RunSinglePh2016B
• 10042.0_ZMM_13 10024.0_TTbar_13

[franzoni]

+1

[franzoni]

hello @kpedro88
would you have a moment to take a look at this ?
( for my education, why is upgraded signature needed here ? )

Thanks !

[kpedro88]

@franzoni upgrade category is automatically assigned to any PR touching Configuration/PyReleaseValidation (among others) due to frequency of upgrade changes in that package

[kpedro88]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

[franzoni]

Thanks @kpedro88

[davidlange6]

+1

[davidlange6]

hi @franzoni - looks like this is causing a number of problems in the IB for cosmics workflows. Can you fix in the next hour(s)?

cosmic: along momentum...
----- Begin Fatal Exception 17-May-2017 11:11:34 CEST-----------------------
An exception of category 'Configuration' occurred while
[0] Processing Event run: 1 lumi: 1 event: 188 stream: 1
[1] Running path 'pathALCARECOMuAlGlobalCosmics'
[2] Calling method for module HLTHighLevel/'ALCARECOMuAlGlobalCosmicsHLT'
Exception Message:
HLTHighLevel [instance: ALCARECOMuAlGlobalCosmicsHLT - path: pathALCARECOMuAlGlobalCosmics]: No triggerList with key MuAlGlobalCosmics in AlCaRecoTriggerBitsRcd

[franzoni]

thanks @davidlange6
addressing

[franzoni]

we see 3 WF's runTheMatrix.py --what standard -l 7.2,7.3,7.4
failing because of a mismatch between the alcarecotrigger bits and the key used to retrieve it mismatch;

( eventSetupPathsKey
:

cmssw/Alignment/CommonAlignmentProducer/python/ALCARECOMuAlGlobalCosmics_cff.py

Line 7 in 09c3fce

eventSetupPathsKey = 'MuAlGlobalCosmics',

)

The problem affects only MC 2017, and can be fixed with a 1 line change.

@trtomei and @tocheng working to a change + tests.
a corretting PR can be available by ~1-2h.

[trtomei]

The PR #18812 makes a one-line change to the MuAlGlobalCosmics AlCaReco, to allow it to also read the "MuAlGlobalCosmicsInCollisions" key. This fixes the 3 crashing workflows - I have manually ran them to completion.