Adjustment of GTs for internal TSG tests (93X) #19467

Martin-Grunewald · 2017-06-29T06:51:29Z

Adjustment of GTs for internal TSG tests (93X)
Based on CMSSW_9_3_X_2017-06-26-2300

cmsbuild · 2017-06-29T06:51:50Z

A new Pull Request was created by @Martin-Grunewald (Martin Grunewald) for master.

It involves the following packages:

Configuration/HLT
HLTrigger/Configuration

@Martin-Grunewald, @silviodonato, @cmsbuild, @fwyzard, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @makortel, @geoff-smith, @jalimena this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

Martin-Grunewald · 2017-06-29T06:52:27Z

please test

cmsbuild · 2017-06-29T06:52:42Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/20974/console Started: 2017/06/29 08:53

cmsbuild · 2017-06-29T08:04:03Z

-1

Tested at: 1ba3f7d

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
0f37c81
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/20974/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/20974/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/20974/summary.html

I found follow errors while testing this PR

Failed tests: AddOn

AddOn:

I found errors in the following addon tests:

cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_PRef --relval 9000,50 --datatier "RAW" --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2017 --magField 38T_PostLS1 --eventcontent RAW --fileout file:RelVal_Raw_PRef_DATA.root --filein /store/data/Run2016B/JetHT/RAW/v1/000/272/762/00000/C666CDE2-E013-E611-B15A-02163E011DBE.root : FAILED - time: date Thu Jun 29 09:46:18 2017-date Thu Jun 29 09:39:11 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 29 09:46:18 2017-date Thu Jun 29 09:39:11 2017 s - exit: 16640
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_PIon --relval 9000,50 --datatier "RAW" --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2017 --magField 38T_PostLS1 --eventcontent RAW --fileout file:RelVal_Raw_PIon_DATA.root --filein /store/data/Run2016B/JetHT/RAW/v1/000/272/762/00000/C666CDE2-E013-E611-B15A-02163E011DBE.root : FAILED - time: date Thu Jun 29 09:49:05 2017-date Thu Jun 29 09:44:07 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:PIon,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_PIon --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_PIon_DATA.root --fileout file:RelVal_Raw_PIon_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 29 09:49:05 2017-date Thu Jun 29 09:44:07 2017 s - exit: 16640
cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_GRun --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2017 --magField 38T_PostLS1 --fileout file:RelVal_Raw_GRun_DATA.root --filein /store/data/Run2016B/JetHT/RAW/v1/000/272/762/00000/C666CDE2-E013-E611-B15A-02163E011DBE.root : FAILED - time: date Thu Jun 29 09:56:29 2017-date Thu Jun 29 09:51:36 2017 s - exit: 16640
cmsDriver.py RelVal -s HLT:GRun,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_GRun --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2017 --magField 38T_PostLS1 --processName=HLTRECO --filein file:RelVal_Raw_GRun_DATA.root --fileout file:RelVal_Raw_GRun_DATA_HLT_RECO.root : FAILED - time: date Thu Jun 29 09:56:29 2017-date Thu Jun 29 09:51:36 2017 s - exit: 16640

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
0f37c81
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/20974/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/20974/git-merge-result

cmsbuild · 2017-06-29T08:04:05Z

Comparison job queued.

Martin-Grunewald · 2017-06-29T08:15:44Z

@arunhep @franzoni
Is this related to a more recent GT update?

----- Begin Fatal Exception 29-Jun-2017 09:46:11 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  stream begin Run run: 272762 stream: 3
   [1] Calling method for module ConvertedPhotonProducer/'conversions'
   [2] Using EventSetup component TransientTrackBuilderESProducer/'' to make data TransientTrackBuilder/'TransientTrackBuilder' in record TransientTrackRecord
   [3] Using EventSetup component GlobalTrackingGeometryESProducer/'' to make data GlobalTrackingGeometry/'' in record GlobalTrackingGeometryRecord
   [4] Using EventSetup component GEMGeometryESModule/'' to make data GEMGeometry/'' in record MuonGeometryRecord
   [5] While getting dependent Record from Record MuonGeometryRecord
Exception Message:
No "GEMRecoGeometryRcd" record found in the EventSetup for synchronization value
Run: 272762 LuminosityBlock: 0 Event: 0 Time: 6281711230306763776
 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

cmsbuild · 2017-06-29T08:20:12Z

Pull request #19467 was updated. @Martin-Grunewald, @silviodonato, @cmsbuild, @fwyzard, @davidlange6 can you please check and sign again.

cmsbuild · 2017-06-29T10:09:41Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/20974/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/build/cmsbld/jenkins-workarea/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-19467/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17
/build/cmsbld/jenkins-workarea/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-19467/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D19_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D19+RecoFullGlobal_2023D19+HARVESTFullGlobal_2023D19

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1756063
DQMHistoTests: Total failures: 15027
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1740870
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

davidlange6 · 2017-06-29T10:42:48Z

please test

cmsbuild · 2017-06-29T10:43:06Z

The tests are being triggered in jenkins.

cmsbuild · 2017-07-05T08:25:27Z

+1
Tested at: 12084b4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/21162/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
63024c6
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/21162/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/21162/git-merge-result

cmsbuild · 2017-07-05T08:25:30Z

Comparison job queued.

cmsbuild · 2017-07-05T09:01:17Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19467/21162/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 22
DQMHistoTests: Total histograms compared: 1785410
DQMHistoTests: Total failures: 102
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 1785142
DQMHistoTests: Total skipped: 166
DQMHistoTests: Total Missing objects: 0
Checked 90 log files, 14 edm output root files, 22 DQM output files

Martin-Grunewald · 2017-07-05T09:40:05Z

+1

Martin-Grunewald · 2017-07-05T09:40:47Z

@Dr15Jones
please sign this PR so that @davidlange6 can integrate
Thanks

Dr15Jones · 2017-07-05T11:31:22Z

+1
Seems odd that a core signature is needed.

cmsbuild · 2017-07-05T11:31:42Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

davidlange6 · 2017-07-05T11:34:32Z

@Dr15Jones - the core sig is from Utilities/ReleaseScripts (which makes sense I think)

davidlange6 · 2017-07-05T11:36:14Z

Configuration/HLT/python/autoCondHLT.py

    'GRun2016'     : ( ','.join( [ 'L1Menu_Collisions2016_v9_m2_xml'                         ,l1tMenuRecord,connectionString,l1tMenuLabel, "2016-10-06 19:36:53.000"] ), ),
    'HIon'         : ( ','.join( [ 'L1Menu_CollisionsHeavyIons2015_v5_uGT_xml'               ,l1tMenuRecord,connectionString,l1tMenuLabel, "2016-03-04 15:00:00.000"] ), ),
    'PIon'         : ( ','.join( [ 'L1Menu_HeavyIons2016_v3_m2_xml'                          ,l1tMenuRecord,connectionString,l1tMenuLabel, "2016-11-22 11:11:00.000"] ), ),
    'PRef'         : ( ','.join( [ 'L1Menu_Collisions2015_5TeV_pp_reference_v5_uGT_v2_mc_xml',l1tMenuRecord,connectionString,l1tMenuLabel, "2016-03-04 15:00:00.000"] ), ),
 }

+pfRecord = 'GBRDWrapperRcd'


hi @Martin-Grunewald - so what are these lines doing in the end?

These lines set the L1T menus for the TSG GTs used in our TSG tests (which change more frequently). Eventually when we update the relval2017 menu the L1T menu needs to be in the actual GT key of AlCa's autoCond.py. But this is like this since many years!

not the L1, the GBRWrapper lines..

These are payloads consumed by running HLT but not yet included in the run2_data_relval or run2_data_promptlike GT keys from AlCa, this is still on the to-do list as referred to earlier to
still have RECO+HLT doable.

@Martin-Grunewald look at this #19467 (comment) I mentioned that all the new code compliant PF payloads have been included in Prompt like key now.

But I'd like to get this PR integrated now and then make a new PR later when you have added it and made new GTs.
@davidlange6

@arunhep - can a gt be available today?

@davidlange6 not possible today but we keep note and we will revive the default keys soon.

"soon" can be when? Middle of next week?

@davidlange6 when CTPPS PR #19470 will be merged we can provide the GT. we still prefer to deploy these tags in prompt like GT.

davidlange6 · 2017-07-06T11:51:43Z

+1

cmsbuild added this to the CMSSW_9_3_X milestone Jun 29, 2017

cmsbuild added comparison-pending hlt-pending orp-pending pending-signatures tests-pending labels Jun 29, 2017

cmsbuild added tests-started and removed tests-pending labels Jun 29, 2017

cmsbuild added tests-rejected and removed tests-started labels Jun 29, 2017

Martin-Grunewald force-pushed the autoAdjustment93X branch from 1ba3f7d to f85816a Compare June 29, 2017 08:19

cmsbuild added tests-pending and removed tests-rejected labels Jun 29, 2017

Martin-Grunewald closed this Jun 29, 2017

Martin-Grunewald deleted the autoAdjustment93X branch June 29, 2017 08:21

Martin-Grunewald restored the autoAdjustment93X branch June 29, 2017 08:55

Martin-Grunewald reopened this Jun 29, 2017

cmsbuild added tests-started and removed tests-pending labels Jun 29, 2017

Adjustment of GTs for internal TSG tests and Fix of addOnTests

50427c3

cmsbuild added comparison-pending tests-pending tests-started and removed tests-pending labels Jul 5, 2017

cmsbuild added tests-approved and removed tests-started labels Jul 5, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jul 5, 2017

cmsbuild added hlt-approved and removed hlt-pending labels Jul 5, 2017

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Jul 5, 2017

davidlange6 reviewed Jul 5, 2017

View reviewed changes

cmsbuild added orp-approved and removed orp-pending labels Jul 6, 2017

cmsbuild merged commit 98f552c into cms-sw:master Jul 6, 2017

perrotta mentioned this pull request Jul 7, 2017

Ctpps pixel clusterizer db 93x #19470

Merged

slava77 mentioned this pull request Jul 17, 2017

had PF + em PF calibration and AlCa update for V1 MC (backport) #19723

Merged

Martin-Grunewald deleted the autoAdjustment93X branch September 7, 2017 13:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adjustment of GTs for internal TSG tests (93X) #19467

Adjustment of GTs for internal TSG tests (93X) #19467

Martin-Grunewald commented Jun 29, 2017 •

edited

cmsbuild commented Jun 29, 2017

Martin-Grunewald commented Jun 29, 2017

cmsbuild commented Jun 29, 2017 •

edited

cmsbuild commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

Martin-Grunewald commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

davidlange6 commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

cmsbuild commented Jul 5, 2017

cmsbuild commented Jul 5, 2017

cmsbuild commented Jul 5, 2017

Martin-Grunewald commented Jul 5, 2017

Martin-Grunewald commented Jul 5, 2017

Dr15Jones commented Jul 5, 2017

cmsbuild commented Jul 5, 2017

davidlange6 commented Jul 5, 2017

davidlange6 Jul 5, 2017

Martin-Grunewald Jul 5, 2017

davidlange6 Jul 5, 2017

Martin-Grunewald Jul 5, 2017 •

edited

arunhep Jul 5, 2017

Martin-Grunewald Jul 5, 2017

davidlange6 Jul 6, 2017

arunhep Jul 6, 2017

davidlange6 Jul 6, 2017

arunhep Jul 6, 2017

davidlange6 commented Jul 6, 2017

Adjustment of GTs for internal TSG tests (93X) #19467

Adjustment of GTs for internal TSG tests (93X) #19467

Conversation

Martin-Grunewald commented Jun 29, 2017 • edited

cmsbuild commented Jun 29, 2017

Martin-Grunewald commented Jun 29, 2017

cmsbuild commented Jun 29, 2017 • edited

cmsbuild commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

Martin-Grunewald commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

davidlange6 commented Jun 29, 2017

cmsbuild commented Jun 29, 2017

cmsbuild commented Jul 5, 2017

cmsbuild commented Jul 5, 2017

cmsbuild commented Jul 5, 2017

Martin-Grunewald commented Jul 5, 2017

Martin-Grunewald commented Jul 5, 2017

Dr15Jones commented Jul 5, 2017

cmsbuild commented Jul 5, 2017

davidlange6 commented Jul 5, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martin-Grunewald Jul 5, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidlange6 commented Jul 6, 2017

Martin-Grunewald commented Jun 29, 2017 •

edited

cmsbuild commented Jun 29, 2017 •

edited

Martin-Grunewald Jul 5, 2017 •

edited