era for HI pp reference run and relval wf #21122

ttrk · 2017-11-01T20:11:46Z

adds "Run2_2017_ppRef" era to run customized reco for HI pp reference run
adds a corresponding relval wf (149)
runs and stores photonIsolationHIProducer objects using the era

… a corresponding relval wf

…f" era

cmsbuild · 2017-11-01T20:12:07Z

The code-checks are being triggered in jenkins.

cmsbuild · 2017-11-01T20:14:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-21122/1751

cmsbuild · 2017-11-01T20:15:07Z

A new Pull Request was created by @ttrk (Kaya Tatar) for master.

It involves the following packages:

Configuration/DataProcessing
Configuration/Eras
Configuration/PyReleaseValidation
Configuration/StandardSequences
RecoEcal/Configuration
RecoEgamma/Configuration

@perrotta, @prebello, @kpedro88, @fabozzi, @cmsbuild, @franzoni, @slava77, @GurpreetSinghChahal, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @varuns23, @Sam-Harper, @argiro, @makortel, @felicepantaleo, @jainshilpi, @GiacomoSguazzoni, @rovere, @lgray, @Martin-Grunewald, @ebrondol, @VinInn, @dgulhan this is something you requested to watch as well.
@davidlange6, @slava77 you are the release manager for this.

cms-bot commands are listed here

slava77 · 2017-11-01T20:22:05Z

Configuration/DataProcessing/python/Impl/ppEra_Run2_2017_ppRef.py

+        self.eras=Run2_2017_ppRef
+        self.promptCustoms += [ 'Configuration/DataProcessing/RecoTLR.customisePostEra_Run2_2017_ppRef' ]
+        self.expressCustoms += [ 'Configuration/DataProcessing/RecoTLR.customisePostEra_Run2_2017_ppRef' ]
+        self.visCustoms += [ 'Configuration/DataProcessing/RecoTLR.customisePostEra_Run2_2017_ppRef' ]


let's reuse the available customisePostEra_Run2_2017

nm, I actually opted to make a new method for Run2_2017_pp_on_XeXe.
so, no change

slava77 · 2017-11-01T20:25:14Z

RecoEcal/Configuration/python/RecoEcal_cff.py

+
+_ecalClustersHI_ppRef = ecalClusters.copy()
+_ecalClustersHI_ppRef += islandBasicClusters
+ppRef_2017.toReplaceWith(ecalClusters, _ecalClustersHI_ppRef)


can this be simply the same as what's used above for pA_2016 etc?

This new era does not need the full islandClusteringSequence, but only islandBasicClusters.
Your suggestion would also work, in that case the other modules in islandClusteringSequence would not be run, since their output will not be used.
I chose to make the customization only for those modules that are going to be run. That is why I made it separate.
What do you suggest ? If you want, I can move the modification and make it part of what is done before.

slava77 · 2017-11-01T20:27:22Z

RecoEgamma/Configuration/python/RecoEgamma_EventContent_cff.py

+from Configuration.Eras.Modifier_ppRef_2017_cff import ppRef_2017
+for ec in [RecoEgammaAOD.outputCommands, RecoEgammaRECO.outputCommands, RecoEgammaFEVT.outputCommands]:
+    ppRef_2017.toModify( ec, func=lambda outputCommands: outputCommands.extend(['keep recoHIPhotonIsolationedmValueMap_photonIsolationHIProducerppGED_*_*',
+                                                                                'keep recoHIPhotonIsolationedmValueMap_photonIsolationHIProducerpp_*_*'


is there an obvious need to be surgical here and not simply follow the same mods as for pA_2016 above?

same as the previous comment. islandPhotons are not added by this. So I made a separate block. If you agree, then I can move it to the block of pA_2016.

slava77 · 2017-11-01T20:27:40Z

@cmsbuild please test

cmsbuild · 2017-11-01T20:29:22Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/24109/console Started: 2017/11/01 21:30

cmsbuild · 2017-11-01T22:42:30Z

+1
Tested at: a8f77ac
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24109/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
c0a86c6
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24109/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24109/git-merge-result

cmsbuild · 2017-11-01T22:42:33Z

Comparison job queued.

cmsbuild · 2017-11-01T23:27:45Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24109/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 1 differences found in the comparisons
DQMHistoTests: Total files compared: 26
DQMHistoTests: Total histograms compared: 2838442
DQMHistoTests: Total failures: 112
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2838159
DQMHistoTests: Total skipped: 171
DQMHistoTests: Total Missing objects: 0
Checked 107 log files, 10 edm output root files, 26 DQM output files

cmsbuild · 2017-11-06T21:22:14Z

The code-checks are being triggered in jenkins.

cmsbuild · 2017-11-06T21:25:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-21122/1846

cmsbuild · 2017-11-06T21:25:49Z

Pull request #21122 was updated. @perrotta, @prebello, @kpedro88, @fabozzi, @cmsbuild, @franzoni, @slava77, @GurpreetSinghChahal, @davidlange6 can you please check and sign again.

slava77 · 2017-11-06T21:52:08Z

@cmsbuild please test workflow 149.0

cmsbuild · 2017-11-06T21:54:14Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/24218/console Started: 2017/11/06 22:55

cmsbuild · 2017-11-07T00:30:52Z

+1
Tested at: 243c2e4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24218/summary.html

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
fcaeeab
You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24218/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24218/git-merge-result

cmsbuild · 2017-11-07T00:30:55Z

Comparison job queued.

cmsbuild · 2017-11-07T02:22:12Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21122/24218/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 26
DQMHistoTests: Total histograms compared: 2900266
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2900094
DQMHistoTests: Total skipped: 171
DQMHistoTests: Total Missing objects: 0
Checked 107 log files, 10 edm output root files, 26 DQM output files

slava77 · 2017-11-07T09:02:06Z

+1

for #21122 243c2e4

ppRef_2017 modifier, Run2_2017_ppRef era, and ppEra_Run2_2017_ppRef T0 processing scenarios are defined to keep additional producers for analysis.
- apparently for T0 all other setups should use the regular 2017
jenkins tests pass, including wf 149.0
local test of 149.0 shows that the extra producers and their products show up and do not introduce a large increase in cost.

davidlange6 · 2017-11-07T09:16:39Z

merge

perrotta · 2017-11-07T09:31:03Z

Configuration/PyReleaseValidation/python/relval_steps.py

@@ -1569,6 +1575,8 @@ def gen2018HiMix(fragment,howMuch):
 steps['RECOHI2015']=merge([hiDefaults2015,{'-s':'RAW2DIGI,L1Reco,RECO,VALIDATION,DQM'},step3Up2015Defaults])
 steps['RECOHI2011']=merge([hiDefaults2011,{'-s':'RAW2DIGI,L1Reco,RECO,VALIDATION,DQM'},step3Defaults])

+steps['RECOPPREF2017']=merge([ppRefDefaults2017,step3Up2015Defaults])


I notice that in the 94X backport PR (#21069) this line reads

steps['RECOPPREF2017']=merge([ppRefDefaults2017,{'-s':'RAW2DIGI,L1Reco,RECO,EI,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@miniAODDQM'},step3Up2015Defaults])

i.e. the RECOSIM got removed: I think the same should be also done here

@perrotta
I don't understand your comment.
Please elaborate.

While reviewing #21069 I noticed that this line differs from
https://github.com/cms-sw/cmssw/pull/21069/files#diff-819af2ec677272fddf1fef50e788a88bR1578

I consider this PR the correct implementation #21122 (comment)

Ok, thank you Slava. I supposed it was done on purpose, instead.
Then it has to be fixed in the backport: I will add a note o that PR

fabozzi · 2017-11-07T11:24:01Z

+1

Kaya Tatar added 2 commits November 1, 2017 20:40

add "Run2_2017_ppRef" era to run customized reco for pp reference and…

6be5e63

… a corresponding relval wf

run and store photonIsolationHIProducer objects using "Run2_2017_ppRe…

a8f77ac

…f" era

cmsbuild added this to the CMSSW_10_0_X milestone Nov 1, 2017

cmsbuild added code-checks-pending comparison-pending operations-pending orp-pending pdmv-pending pending-signatures reconstruction-pending tests-pending upgrade-pending labels Nov 1, 2017

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 1, 2017

slava77 reviewed Nov 1, 2017

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Nov 1, 2017

cmsbuild added tests-approved and removed tests-started labels Nov 1, 2017

cmsbuild removed the comparison-pending label Nov 1, 2017

cmsbuild added the tests-pending label Nov 6, 2017

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 6, 2017

cmsbuild added tests-started and removed tests-pending labels Nov 6, 2017

cmsbuild added tests-approved and removed tests-started labels Nov 7, 2017

cmsbuild added comparison-available and removed comparison-pending labels Nov 7, 2017

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Nov 7, 2017

cmsbuild added orp-approved and removed orp-pending labels Nov 7, 2017

cmsbuild merged commit 2a61fa0 into cms-sw:master Nov 7, 2017

perrotta reviewed Nov 7, 2017

View reviewed changes

thomreis mentioned this pull request Nov 7, 2017

L1T offine DQM changes for pp reference run - 94x #21197

Closed

cmsbuild added pdmv-approved and removed pdmv-pending labels Nov 7, 2017

ttrk mentioned this pull request Nov 15, 2017

keep islandPhoton related collections via HI eras #21310

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

era for HI pp reference run and relval wf #21122

era for HI pp reference run and relval wf #21122

ttrk commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

slava77 Nov 1, 2017

slava77 Nov 1, 2017

slava77 Nov 1, 2017

ttrk Nov 1, 2017 •

edited

slava77 Nov 1, 2017

ttrk Nov 1, 2017 •

edited

slava77 commented Nov 1, 2017

cmsbuild commented Nov 1, 2017 •

edited

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 6, 2017

cmsbuild commented Nov 6, 2017

cmsbuild commented Nov 6, 2017

slava77 commented Nov 6, 2017

cmsbuild commented Nov 6, 2017 •

edited

cmsbuild commented Nov 7, 2017

cmsbuild commented Nov 7, 2017

cmsbuild commented Nov 7, 2017

slava77 commented Nov 7, 2017

davidlange6 commented Nov 7, 2017

perrotta Nov 7, 2017

slava77 Nov 7, 2017

perrotta Nov 7, 2017

slava77 Nov 7, 2017

perrotta Nov 7, 2017

fabozzi commented Nov 7, 2017

era for HI pp reference run and relval wf #21122

era for HI pp reference run and relval wf #21122

Conversation

ttrk commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ttrk Nov 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ttrk Nov 1, 2017 • edited

Choose a reason for hiding this comment

slava77 commented Nov 1, 2017

cmsbuild commented Nov 1, 2017 • edited

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 1, 2017

cmsbuild commented Nov 6, 2017

cmsbuild commented Nov 6, 2017

cmsbuild commented Nov 6, 2017

slava77 commented Nov 6, 2017

cmsbuild commented Nov 6, 2017 • edited

cmsbuild commented Nov 7, 2017

cmsbuild commented Nov 7, 2017

cmsbuild commented Nov 7, 2017

slava77 commented Nov 7, 2017

davidlange6 commented Nov 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabozzi commented Nov 7, 2017

ttrk Nov 1, 2017 •

edited

ttrk Nov 1, 2017 •

edited

cmsbuild commented Nov 1, 2017 •

edited

cmsbuild commented Nov 6, 2017 •

edited