Introduce Run3 (post-LS2) workflow #19889

kpedro88 · 2017-07-24T19:16:17Z

This PR introduces a Run3 workflow for the post-LS2 detector (including GE1/1 and HB upgrades, along with Castor removal). The Run2_2018 GT is used for now. Because HCAL conditions don't exist yet for the post-LS2 geometry, the hardcode conditions are used (via the new modifier setup introduced in #19712).

A few modifications are made to the GEM geometry based on phase2 developments (discussed with experts). In addition, I had to fix some logic in a muon DQM analyzer to prevent a crash; with GE1/1 included, we apparently get some muons that fail all the conditions isGlobalMuon(), isTrackerMuon(), isRPCMuon(), isStandAloneMuon(). For now, those muons are just skipped. Muon experts may want to improve this handling in the future.

attn: @calabria, @bsunanda, @ianna

cmsbuild · 2017-07-24T19:16:38Z

A new Pull Request was created by @kpedro88 (Kevin Pedro) for master.

It involves the following packages:

Configuration/Eras
Configuration/Geometry
Configuration/PyReleaseValidation
Configuration/StandardSequences
DQMOffline/Muon
Geometry/CMSCommonData
Geometry/MuonSimData
SimGeneral/MixingModule

@civanch, @Dr15Jones, @vazzolini, @dmitrijus, @kmaeshima, @ianna, @kpedro88, @fabozzi, @cmsbuild, @kkousour, @franzoni, @mdhildreth, @GurpreetSinghChahal, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @barvic, @bellan, @makortel, @felicepantaleo, @abbiendi, @GiacomoSguazzoni, @jhgoh, @VinInn, @Martin-Grunewald, @calderona, @HuguesBrun, @rovere, @ptcox, @battibass, @ebrondol, @trocino, @dgulhan, @rociovilar this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

kpedro88 · 2017-07-24T19:16:50Z

please test

cmsbuild · 2017-07-24T19:16:55Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21719/console Started: 2017/07/24 21:17

cmsbuild · 2017-07-24T21:01:32Z

-1

Tested at: 03417eb

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
a1f0a3b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19889/21719/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19889/21719/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19889/21719/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following worklfows:
11624.0 step3

runTheMatrix-results/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+ALCAFull_2019+HARVESTFull_2019/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+ALCAFull_2019+HARVESTFull_2019.log

Material budget:

There was error running material budget tests.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
a1f0a3b
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19889/21719/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19889/21719/git-merge-result

cmsbuild · 2017-07-24T21:01:34Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

kpedro88 · 2017-07-24T21:17:39Z

relval failure was expected, seems to be due to pixel Phase 1 DQM changes (ongoing investigation, discussion in #19811)
no idea about the material budget failure, the log doesn't indicate anything.

kpedro88 · 2017-07-24T21:18:14Z

(NB: this ran fine in CMSSW_9_3_0_pre2)

kpedro88 · 2017-08-01T03:49:11Z

+1

ianna · 2017-08-01T06:07:26Z

+1

civanch · 2017-08-01T07:47:47Z

+1

dmitrijus · 2017-08-01T14:44:31Z

+1

kpedro88 · 2017-08-02T16:18:59Z

@fabozzi please sign

kpedro88 · 2017-08-04T15:17:33Z

@fabozzi ping

fabozzi · 2017-08-05T14:13:54Z

+1

davidlange6 · 2017-08-08T09:53:24Z

Configuration/StandardSequences/python/Reconstruction_cff.py

-from Configuration.Eras.Modifier_phase2_common_cff import phase2_common
-phase2_common.toReplaceWith(localreco, _phase2_localreco)
-phase2_common.toReplaceWith(localreco_HcalNZS, _phase2_localreco_HcalNZS)
+_run3_localreco = localreco.copyAndExclude([castorreco])


Hi @kpedro88 - was this choice based on some discussion?

The post-LS2 beampipe requires CASTOR to be removed.

@hvanhaev
please confirm that there will be no running with castor after LS2.
Thank you.

davidlange6 · 2017-08-08T09:54:36Z

Geometry/MuonSimData/data/2019/muonProdCuts.xml

@@ -0,0 +1,70 @@
+<?xml version="1.0"?>


what is the idea behind copying this file? Does it reflect a detector change?

L37 is different to reflect the addition of GEMS:

<PartSelector path="//GE.*"/>

davidlange6 · 2017-08-09T07:06:02Z

merge

Dr15Jones · 2017-08-09T14:56:09Z

It appears this pull request added workflow 11626 to the nightly IB and that workflow is failing:
https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/slc7_aarch64_gcc700/CMSSW_9_3_X_2017-08-09-1100/pyRelValMatrixLogs/run/11626.0_QCD_Pt_600_800_13+QCD_Pt_600_800_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+ALCAFull_2019+HARVESTFull_2019/step2_QCD_Pt_600_800_13+QCD_Pt_600_800_13TeV_TuneCUETP8M1_2019_GenSimFull+DigiFull_2019+RecoFull_2019+ALCAFull_2019+HARVESTFull_2019.log

the exception being thrown is some variation of a request out of bounds

----- Begin Fatal Exception 09-Aug-2017 13:34:49 CEST-----------------------
An exception of category 'Out of Range' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'digitisation_step'
   [2] Calling method for module HcalTrigPrimDigiProducer/'simHcalTriggerPrimitiveDigis'
Exception Message:
LUT has 1024 entries for (HcalTrigTower v0: -3,50) but 1596 was requested.
----- End Fatal Exception -------------------------------------------------

kpedro88 · 2017-08-09T14:57:21Z

@Dr15Jones ah, the short matrix only runs 11624... I'll investigate

kpedro88 · 2017-08-09T16:29:42Z

@Dr15Jones see #20104

hvanhaev · 2017-08-21T12:11:57Z

Dear Slava, Indeed, at the moment there are no data taking plans with CASTOR after LS2. Cheers, Hans On 8 Aug 2017, at 17:07, Slava Krutelyov <notifications@github.com<mailto:notifications@github.com>> wrote: @slava77 commented on this pull request.

________________________________ In Configuration/StandardSequences/python/Reconstruction_cff.py<#19889 (comment)>:

@@ -50,16 +50,16 @@

localreco = cms.Sequence(bunchSpacingProducer+trackerlocalreco+muonlocalreco+calolocalreco+castorreco) localreco_HcalNZS = cms.Sequence(bunchSpacingProducer+trackerlocalreco+muonlocalreco+calolocalrecoNZS+castorreco)

…

-_phase2_localreco = localreco.copyAndExclude([castorreco]) -_phase2_localreco_HcalNZS = localreco_HcalNZS.copyAndExclude([castorreco]) -from Configuration.Eras.Modifier_phase2_common_cff import phase2_common -phase2_common.toReplaceWith(localreco, _phase2_localreco) -phase2_common.toReplaceWith(localreco_HcalNZS, _phase2_localreco_HcalNZS) +_run3_localreco = localreco.copyAndExclude([castorreco]) @hvanhaev<https://github.com/hvanhaev> please confirm that there will be no running with castor after LS2. Thank you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#19889 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEgkFyQOqXC6yr8G07z0Pu0wSuNpB6IKks5sWHnJgaJpZM4OhnqF>.

cmsbuild added this to the CMSSW_9_3_X milestone Jul 24, 2017

cmsbuild added comparison-pending dqm-pending geometry-pending operations-pending orp-pending pdmv-pending pending-signatures simulation-pending tests-pending upgrade-pending labels Jul 24, 2017

cmsbuild added tests-started and removed tests-pending labels Jul 24, 2017

kpedro88 mentioned this pull request Jul 24, 2017

Added check to ensure simhit collection in pixel rechit validation exists #19811

Merged

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jul 24, 2017

kpedro88 added 5 commits July 31, 2017 14:58

adjustments to muon 2019 geometry

7104066

create postLS2/Run3 workflow

e3803b7

put castor removal in run3_common

0b1663f

fix logic

365d165

add 2019 to matrix tests

3337ca4

cmsbuild added the comparison-available label Aug 1, 2017

cmsbuild added upgrade-approved and removed upgrade-pending labels Aug 1, 2017

cmsbuild added geometry-approved and removed geometry-pending labels Aug 1, 2017

cmsbuild added simulation-approved and removed simulation-pending labels Aug 1, 2017

davidcarbonis mentioned this pull request Aug 1, 2017

PixelPhase Validation bin reduction and protection against memory leak. #19992

Merged

cmsbuild added dqm-approved and removed dqm-pending labels Aug 1, 2017

cmsbuild added pdmv-approved and removed pdmv-pending labels Aug 5, 2017

kpedro88 mentioned this pull request Aug 7, 2017

Run2 sim16 Add scenarios for T1/T2/BCML2 for post LS2 scenario #19883

Merged

davidlange6 reviewed Aug 8, 2017

View reviewed changes

cmsbuild merged commit 0dee4e3 into cms-sw:master Aug 9, 2017

kpedro88 mentioned this pull request Aug 9, 2017

update HCAL trigger mode for Run3 #20104

Merged

kpedro88 mentioned this pull request Aug 22, 2017

Creation of provisional Run3 (postLS2) scenario #16680

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Run3 (post-LS2) workflow #19889

Introduce Run3 (post-LS2) workflow #19889

kpedro88 commented Jul 24, 2017

cmsbuild commented Jul 24, 2017

kpedro88 commented Jul 24, 2017

cmsbuild commented Jul 24, 2017 •

edited

cmsbuild commented Jul 24, 2017

cmsbuild commented Jul 24, 2017

kpedro88 commented Jul 24, 2017

kpedro88 commented Jul 24, 2017

kpedro88 commented Aug 1, 2017

ianna commented Aug 1, 2017

civanch commented Aug 1, 2017

dmitrijus commented Aug 1, 2017

kpedro88 commented Aug 2, 2017

kpedro88 commented Aug 4, 2017

fabozzi commented Aug 5, 2017

davidlange6 Aug 8, 2017

kpedro88 Aug 8, 2017

slava77 Aug 8, 2017

davidlange6 Aug 8, 2017

kpedro88 Aug 8, 2017

davidlange6 commented Aug 9, 2017

Dr15Jones commented Aug 9, 2017

kpedro88 commented Aug 9, 2017

kpedro88 commented Aug 9, 2017

hvanhaev commented Aug 21, 2017 via email

Introduce Run3 (post-LS2) workflow #19889

Introduce Run3 (post-LS2) workflow #19889

Conversation

kpedro88 commented Jul 24, 2017

cmsbuild commented Jul 24, 2017

kpedro88 commented Jul 24, 2017

cmsbuild commented Jul 24, 2017 • edited

cmsbuild commented Jul 24, 2017

cmsbuild commented Jul 24, 2017

kpedro88 commented Jul 24, 2017

kpedro88 commented Jul 24, 2017

kpedro88 commented Aug 1, 2017

ianna commented Aug 1, 2017

civanch commented Aug 1, 2017

dmitrijus commented Aug 1, 2017

kpedro88 commented Aug 2, 2017

kpedro88 commented Aug 4, 2017

fabozzi commented Aug 5, 2017

davidlange6 Aug 8, 2017

Choose a reason for hiding this comment

kpedro88 Aug 8, 2017

Choose a reason for hiding this comment

slava77 Aug 8, 2017

Choose a reason for hiding this comment

davidlange6 Aug 8, 2017

Choose a reason for hiding this comment

kpedro88 Aug 8, 2017

Choose a reason for hiding this comment

davidlange6 commented Aug 9, 2017

Dr15Jones commented Aug 9, 2017

kpedro88 commented Aug 9, 2017

kpedro88 commented Aug 9, 2017

hvanhaev commented Aug 21, 2017 via email

cmsbuild commented Jul 24, 2017 •

edited