Switch to heavy-ion jets in HI miniAOD #30898

mandrenguyen · 2020-07-24T14:28:10Z

PR description:

Replaces pp jets in HI miniAOD with HI jets, using the Cs UE subtraction algorithm (as updated in #30057 ).

PR validation:

Tested with workflows 158.01 and 140.5611

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2020-07-24T14:28:34Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-07-24T14:36:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30898/17291

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py modified in PR(s): Addition of packedCandidates track chi2 information to pp_on_AA miniAOD #30566, [RFC] Store TrackExtra, TrackingRecHits, and SiPixel/StripClusters for muons in AOD+MINIAOD #30544, Add the HF Filters in Heavy Ions miniAOD #30810
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): Add new MVA tau ID for phase II #30341

cmsbuild · 2020-07-24T14:36:39Z

A new Pull Request was created by @mandrenguyen (Matthew Nguyen) for master.

It involves the following packages:

PhysicsTools/PatAlgos
RecoHI/HiTracking

@perrotta, @jpata, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @schoef, @emilbols, @echapon, @peruzzim, @seemasharmafnal, @mmarionncern, @ahinzmann, @smoortga, @dgulhan, @jdolen, @ferencek, @yetkinyilmaz, @jdamgov, @nhanvtran, @gkasieczka, @hatakeyamak, @mariadalfonso, @clelange, @riga, @jazzitup, @JyothsnaKomaragiri, @mverzett, @yenjie, @kurtejung, @gpetruc, @andrzejnovak this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

slava77 · 2020-07-24T14:41:32Z

@cmsbuild please test

cmsbuild · 2020-07-24T14:41:56Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-07-23-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/8273/console Started: 2020/07/24 17:26

cmsbuild · 2020-07-24T16:46:17Z

+1
Tested at: 9aaff11
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc9d6c/8273/summary.html
CMSSW: CMSSW_11_2_X_2020-07-23-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-07-24T16:46:20Z

Comparison job queued.

cmsbuild · 2020-07-24T18:13:08Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc9d6c/8273/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 131 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2526188
DQMHistoTests: Total failures: 797
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2525344
DQMHistoTests: Total skipped: 47
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 144 log files, 17 edm output root files, 34 DQM output files

cmsbuild · 2020-10-01T22:52:15Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc9d6c/9694/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 515 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2542225
DQMHistoTests: Total failures: 970
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2541233
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

slava77

I was looking at the results in wf 140.5611. Most of the changes are as expected, however there are still some top level oddities or points of concern

slimmedJets_tagInfos take up 25% of the file size ( increases the total file size by 18%) ; are they really worth it? Note that the cost in less busy events of wf 158.01 is only 4% of the disk size. Some preselection may be useful, likely for a follow up PR.
puppi is still running ; see the comment inline for patPhotons addPuppiIsolation
isolatedTracks module is a bit heavy (9% CPU, second slowest module, after akCs4PFJets); with only 1% disk use though. Note that in 158.01 it's pretty low on the list (0.4% of CPU). So, it's probably more in the land of general CPU optimization with a bit more interest/motivation for HI needs.

Additionally, in wf 158.0 the pfJetsEI are now filled (these run but are not written in AOD). The reason is in the ak4PFJets (see a comment for that).

I think that most of these if not more can be followed up in #31647

slava77 · 2020-10-01T23:27:49Z

PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py

@@ -120,20 +124,25 @@ def miniAOD_customizeCommon(process):
    from Configuration.Eras.Modifier_phase2_muon_cff import phase2_muon


around line 93 I suggest to add

_hiGeneral.toModify(process.patPhotons, addPuppiIsolation = False)

it looks like the only reason why "puppi" module is still running

slava77 · 2020-10-02T00:47:49Z

RecoJets/JetProducers/python/ak4PFJets_cfi.py

-pp_on_AA_2018.toModify(ak4PFJets,src = "pfNoPileUpJMEHI", inputEtMin = 9999)
-pp_on_AA_2018.toModify(ak4PFJetsCHS,src = "pfNoPileUpJMEHI", inputEtMin = 9999)
+from Configuration.Eras.Modifier_pp_on_PbPb_run3_cff import pp_on_PbPb_run3
+(pp_on_AA_2018 | pp_on_PbPb_run3).toModify(ak4PFJets, src = "pfEmptyCollection")


Suggested change

(pp_on_AA_2018 | pp_on_PbPb_run3).toModify(ak4PFJets, src = "pfEmptyCollection")

(pp_on_AA_2018 | pp_on_PbPb_run3).toModify(ak4PFJets, src = "pfEmptyCollection", inputEtMin = 9999)

I think that the threshold should still be applied; better yet, a more thorough cleanup is needed in CommonTools/ParticleFlow/python/EITopPAG_cff.py

slava77 · 2020-10-02T01:14:05Z

+1

for #30898 0fcdc06

default jets for HI are switched to akCs4PFJets, PAT is adapted to use these jets in the default patJets; miniAOD (miniAOD_tools.py) is updated/refactored to not configure standard pp parts that are not needed in HI (this approach can be extended to other possible special miniAOD workflows)
jenkins tests pass and comparisons with the baseline show differences only in the HI workflows using the modified pp reco/miniAOD.

A few items that somewhat extend beyond the scope of just having the jets configured for miniAOD can be followed up in #31647

mandrenguyen · 2020-10-03T04:55:00Z

@silviodonato Can you merge this one, please? There are other PRs I need to make on top of it that are needed for the HI-miniAOD in 11_2_X.

silviodonato · 2020-10-03T08:26:48Z

merge

santocch · 2020-10-04T15:07:48Z

+1

cmsbuild · 2020-10-04T15:08:11Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

makortel · 2020-10-05T13:47:03Z

Apparently this PR broke workflows 140.0, 145.0, and 150.0 with

Traceback (most recent call last):
  File "/cvmfs/cms-ib.cern.ch/nweek-02649/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_2_X_2020-10-04-0000/bin/slc7_amd64_gcc820/cmsDriver.py", line 56, in <module>
    run()
  File "/cvmfs/cms-ib.cern.ch/nweek-02649/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_2_X_2020-10-04-0000/bin/slc7_amd64_gcc820/cmsDriver.py", line 28, in run
    configBuilder.prepare()
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/Configuration/Applications/ConfigBuilder.py", line 2135, in prepare
    self.addStandardSequences()
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/Configuration/Applications/ConfigBuilder.py", line 757, in addStandardSequences
    getattr(self,"prepare_"+stepName)(sequence = getattr(self,stepName+"DefaultSeq"))
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/Configuration/Applications/ConfigBuilder.py", line 1773, in prepare_VALIDATION
    self.loadDefaultOrSpecifiedCFF(sequence,self.VALIDATIONDefaultCFF)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/Configuration/Applications/ConfigBuilder.py", line 1198, in loadDefaultOrSpecifiedCFF
    l=self.loadAndRemember(defaultCFF)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/Configuration/Applications/ConfigBuilder.py", line 325, in loadAndRemember
    self.process.load(includeFile)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/FWCore/ParameterSet/Config.py", line 684, in load
    self.extend(sys.modules[moduleName])
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/FWCore/ParameterSet/Config.py", line 731, in extend
    self.__setattr__(name, task)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/FWCore/ParameterSet/Config.py", line 507, in __setattr__
    newValue._place(name,self)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/FWCore/ParameterSet/SequenceTypes.py", line 1441, in _place
    proc._placeTask(name,self)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/FWCore/ParameterSet/Config.py", line 650, in _placeTask
    self._validateTask(task, name)
  File "/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_2_X_2020-10-04-2300/python/FWCore/ParameterSet/Config.py", line 884, in _validateTask
    raise RuntimeError("An entry in task " + label + ' has not been attached to the process')
RuntimeError: An entry in task recoJetsHIpostAODTask has not been attached to the process

see e.g. https://cmssdt.cern.ch/SDT/cgi-bin/logreader/slc7_amd64_gcc820/CMSSW_11_2_X_2020-10-04-2300/pyRelValMatrixLogs/run/140.0_HydjetQ_B12_5020GeV_2011+HydjetQ_B12_5020GeV_2011+DIGIHI2011+RECOHI2011+HARVESTHI2011/step3_HydjetQ_B12_5020GeV_2011+HydjetQ_B12_5020GeV_2011+DIGIHI2011+RECOHI2011+HARVESTHI2011.log#/

mandrenguyen · 2020-10-05T15:30:18Z

@makortel I tracked down the issue, which is a naming conflict with a simple fix.
Should I make a new PR, since this one is already merged?

makortel · 2020-10-05T15:32:25Z

Should I make a new PR, since this one is already merged?

Yes, please make a new PR.

makortel · 2020-10-05T16:01:21Z

This PR appears to also cause many HI workflows to crash (see #31670). I hope the fix will fix those as well.

mandrenguyen · 2020-10-05T16:23:44Z

@makortel Ah, there's another issue affecting 158.1, but I can see the fix for that as well.
I will make a combined PR to fix both of them. ETA is late tonight CERN time though.

cmsbuild added this to the CMSSW_11_2_X milestone Jul 24, 2020

cmsbuild added analysis-pending code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 24, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 24, 2020

cmsbuild added tests-started and removed tests-pending labels Jul 24, 2020

cmsbuild added tests-approved and removed tests-started labels Jul 24, 2020

cmsbuild added comparison-available code-checks-pending comparison-pending tests-pending and removed comparison-pending code-checks-approved comparison-available tests-approved labels Jul 24, 2020

cmsbuild added the tests-approved label Oct 1, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 1, 2020

slava77 mentioned this pull request Oct 2, 2020

cleanup/optimize pp on HI in miniAOD/PAT, EI (and maybe RECO) steps to align better with heavy ions needs #31647

Closed

5 tasks

slava77 reviewed Oct 2, 2020

View reviewed changes

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 2, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 3, 2020

cmsbuild merged commit 9691783 into cms-sw:master Oct 3, 2020

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Oct 4, 2020

makortel mentioned this pull request Oct 5, 2020

Segfault in BDHadronTrackMonitoringAnalyzer #31670

Closed

mandrenguyen mentioned this pull request Oct 5, 2020

Use akCs4PFJets for candidate-based tagInfos for HI workflows #31674

Merged

jfernan2 mentioned this pull request Oct 6, 2020

Unit test failures in DQMOffline/Configuration #31669

Closed

mandrenguyen mentioned this pull request Oct 10, 2020

Change input jet collection for b2g validation in HI wfs #31742

Merged

mandrenguyen deleted the hiJets4PAT branch May 26, 2021 06:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to heavy-ion jets in HI miniAOD #30898

Switch to heavy-ion jets in HI miniAOD #30898

mandrenguyen commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

slava77 commented Jul 24, 2020

cmsbuild commented Jul 24, 2020 •

edited

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Oct 1, 2020

slava77 left a comment

slava77 Oct 1, 2020

slava77 Oct 2, 2020

slava77 commented Oct 2, 2020

mandrenguyen commented Oct 3, 2020

silviodonato commented Oct 3, 2020

santocch commented Oct 4, 2020

cmsbuild commented Oct 4, 2020

makortel commented Oct 5, 2020

mandrenguyen commented Oct 5, 2020

makortel commented Oct 5, 2020

makortel commented Oct 5, 2020

mandrenguyen commented Oct 5, 2020

		@@ -120,20 +124,25 @@ def miniAOD_customizeCommon(process):
		from Configuration.Eras.Modifier_phase2_muon_cff import phase2_muon

	(pp_on_AA_2018 \| pp_on_PbPb_run3).toModify(ak4PFJets, src = "pfEmptyCollection")
	(pp_on_AA_2018 \| pp_on_PbPb_run3).toModify(ak4PFJets, src = "pfEmptyCollection", inputEtMin = 9999)

Switch to heavy-ion jets in HI miniAOD #30898

Switch to heavy-ion jets in HI miniAOD #30898

Conversation

mandrenguyen commented Jul 24, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

slava77 commented Jul 24, 2020

cmsbuild commented Jul 24, 2020 • edited

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Jul 24, 2020

cmsbuild commented Oct 1, 2020

slava77 left a comment

Choose a reason for hiding this comment

slava77 Oct 1, 2020

Choose a reason for hiding this comment

slava77 Oct 2, 2020

Choose a reason for hiding this comment

slava77 commented Oct 2, 2020

mandrenguyen commented Oct 3, 2020

silviodonato commented Oct 3, 2020

santocch commented Oct 4, 2020

cmsbuild commented Oct 4, 2020

makortel commented Oct 5, 2020

mandrenguyen commented Oct 5, 2020

makortel commented Oct 5, 2020

makortel commented Oct 5, 2020

mandrenguyen commented Oct 5, 2020

cmsbuild commented Jul 24, 2020 •

edited