Add ParticleNetAK4 to JME custom NanoAOD #31714

hqucms · 2020-10-08T15:41:41Z

PR description:

This PR adds the ParticleNetAK4 tagger scores to the JME custom NanoAOD. The tagger is added to both AK4 CHS and Puppi jets.

PR validation:

Tested NANO step on a UL17 ttbar sample w/ --customise PhysicsTools/NanoAOD/custom_jme_cff.PrepJMECustomNanoAOD_MC, and the tagger scores are correctly added.

The event size increases from 11.78kb to 12.07kb (+2.5%) on ttbar MC.

FYI @camclean @alefisico

cmsbuild · 2020-10-08T15:42:07Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-08T15:46:30Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31714/18899

This PR adds an extra 20KB to repository

cmsbuild · 2020-10-08T15:46:55Z

A new Pull Request was created by @hqucms (Huilin Qu) for master.

It involves the following packages:

PhysicsTools/NanoAOD

@gouskos, @cmsbuild, @fgolf, @mariadalfonso, @santocch, @peruzzim can you please review it and eventually sign? Thanks.
@gpetruc this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2020-10-08T17:55:57Z

please test workflow 25202.15,10224.15,11024.15

cmsbuild · 2020-10-08T17:56:18Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 11024.15,10224.15,25202.15
CMSSW_11_2_X_2020-10-08-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9818/console Started: 2020/10/08 19:58

mariadalfonso · 2020-10-08T18:31:36Z

PhysicsTools/NanoAOD/python/custom_jme_cff.py

@@ -25,7 +25,8 @@
  'pfDeepFlavourJetTags:probb','pfDeepFlavourJetTags:probbb','pfDeepFlavourJetTags:problepb',


pfCombinedMVAV2BJetTags and pfCombinedInclusiveSecondaryVertexV2BJetTags are already in the central nano, why are they added also in the JMEnano with a different name ?

btagCMVA = Var("bDiscriminator('pfCombinedMVAV2BJetTags')",float,doc="CMVA V2 btag discriminator",precision=10), btagCSVV2 = Var("bDiscriminator('pfCombinedInclusiveSecondaryVertexV2BJetTags')",float,doc=" pfCombinedInclusiveSecondaryVertexV2 b-tag discriminator (aka CSVV2)",precision=10)

@camclean @alefisico any comment ?

These variables are included in the central nano for the Jet collection (AK4 CHS) but in our JMEnano we included other AK4 collections (PUPPI/PF) where we want to have the same content for comparison. So we need to include these variables in the new jet collections.

cmsbuild · 2020-10-08T19:20:43Z

+1
Tested at: 590576d
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-015821/9818/summary.html
CMSSW: CMSSW_11_2_X_2020-10-08-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-08T19:20:45Z

Comparison job queued.

cmsbuild · 2020-10-08T20:49:31Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-015821/9818/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-015821/10224.15_TTbar_13+2017PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-015821/11024.15_TTbar_13+2018PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-015821/25202.15_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15MC_PU25_JME

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2542225
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2542196
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

mariadalfonso · 2020-10-12T08:14:17Z

+xpog

central nano are not touched, only JMEnano

silviodonato · 2020-11-09T17:50:36Z

Hi @gouskos sorry for the late reply, but I was also very surprise by the nJet multiplicity. We performed some checks and the difference is real. We see similar nJet multiplicity for genJets and JetPuppi (as you can see in the plot below, thanks @kirschen) AK4 Puppi jets performs really good at low pt, removing most of the pileup jets that CHS finds.
I think this is something important to keep in mind. We will need CHS and PUPPI AK4 jets in the transition period, but after that the size of our JMEnano samples will be significantly smaller.

@kirschen are you asking @gouskos @mariadalfonso to keep the PR as it is?

kirschen · 2020-11-09T18:46:02Z

Hi @gouskos sorry for the late reply, but I was also very surprise by the nJet multiplicity. We performed some checks and the difference is real. We see similar nJet multiplicity for genJets and JetPuppi (as you can see in the plot below, thanks @kirschen) AK4 Puppi jets performs really good at low pt, removing most of the pileup jets that CHS finds.
I think this is something important to keep in mind. We will need CHS and PUPPI AK4 jets in the transition period, but after that the size of our JMEnano samples will be significantly smaller.

@kirschen are you asking @gouskos @mariadalfonso to keep the PR as it is?

The PUPPI jet multiplicity and size consolidation as suggested in cms-nanoAOD#549 are unrelated to this PR, so I would say this can go ahead. We will address what jets exactly to keep at low pt and what level of information to keep down to which pt in a separate PR.

santocch · 2020-11-10T10:51:41Z

+1

mariadalfonso · 2020-11-10T13:18:28Z

please test workflow 25202.15,10224.15,11024.15

cmsbuild · 2020-11-10T13:18:51Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 11024.15,10224.15,25202.15
CMSSW_11_2_X_2020-11-10-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10616/console Started: 2020/11/10 14:21

cmsbuild · 2020-11-10T14:51:57Z

+1
Tested at: f9fbb26
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-015821/10616/summary.html
CMSSW: CMSSW_11_2_X_2020-11-10-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-11-10T14:52:00Z

Comparison job queued.

cmsbuild · 2020-11-10T16:13:42Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-015821/10616/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-015821/10224.15_TTbar_13+2017PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-015821/11024.15_TTbar_13+2018PU_JMENano+TTbar_13TeV_TuneCUETP8M1_GenSim+DigiPU+RecoFakeHLTPU+HARVESTFakeHLTPU+Nano
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-015821/25202.15_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15MC_PU25_JME

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2529296
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2529273
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 148 log files, 22 edm output root files, 35 DQM output files

mariadalfonso · 2020-11-10T19:19:02Z

+xpog

changes only to the jmenano;
the size keep increasing and refinement will be done in a separate PR;

cmsbuild · 2020-11-10T19:19:25Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-11-10T20:34:57Z

+1

cmsbuild added this to the CMSSW_11_2_X milestone Oct 8, 2020

cmsbuild added analysis-pending code-checks-pending comparison-pending orp-pending pending-signatures tests-pending xpog-pending labels Oct 8, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 8, 2020

hqucms mentioned this pull request Oct 8, 2020

[10_6_X] Add ParticleNetAK4 to JME custom NanoAOD. #31717

Merged

cmsbuild added tests-started and removed tests-pending labels Oct 8, 2020

mariadalfonso mentioned this pull request Oct 8, 2020

features for V8 nanoAOD cms-nanoAOD/cmssw#533

Closed

55 tasks

mariadalfonso reviewed Oct 8, 2020

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Oct 8, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 8, 2020

cmsbuild added xpog-approved and removed xpog-pending labels Oct 12, 2020

cmsbuild added analysis-approved and removed analysis-pending labels Nov 10, 2020

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Nov 10, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 10, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 10, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 10, 2020

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Nov 10, 2020

cmsbuild added orp-approved and removed orp-pending labels Nov 10, 2020

cmsbuild merged commit 8c057b2 into cms-sw:master Nov 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ParticleNetAK4 to JME custom NanoAOD #31714

Add ParticleNetAK4 to JME custom NanoAOD #31714

hqucms commented Oct 8, 2020 •

edited

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

mariadalfonso commented Oct 8, 2020

cmsbuild commented Oct 8, 2020 •

edited

mariadalfonso Oct 8, 2020

mariadalfonso Oct 10, 2020

alefisico Oct 12, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

mariadalfonso commented Oct 12, 2020

silviodonato commented Nov 9, 2020

kirschen commented Nov 9, 2020

santocch commented Nov 10, 2020

mariadalfonso commented Nov 10, 2020

cmsbuild commented Nov 10, 2020 •

edited

cmsbuild commented Nov 10, 2020

cmsbuild commented Nov 10, 2020

cmsbuild commented Nov 10, 2020

mariadalfonso commented Nov 10, 2020

cmsbuild commented Nov 10, 2020

silviodonato commented Nov 10, 2020

		@@ -25,7 +25,8 @@
		'pfDeepFlavourJetTags:probb','pfDeepFlavourJetTags:probbb','pfDeepFlavourJetTags:problepb',

Add ParticleNetAK4 to JME custom NanoAOD #31714

Add ParticleNetAK4 to JME custom NanoAOD #31714

Conversation

hqucms commented Oct 8, 2020 • edited

PR description:

PR validation:

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

mariadalfonso commented Oct 8, 2020

cmsbuild commented Oct 8, 2020 • edited

mariadalfonso Oct 8, 2020

Choose a reason for hiding this comment

mariadalfonso Oct 10, 2020

Choose a reason for hiding this comment

alefisico Oct 12, 2020

Choose a reason for hiding this comment

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

mariadalfonso commented Oct 12, 2020

silviodonato commented Nov 9, 2020

kirschen commented Nov 9, 2020

santocch commented Nov 10, 2020

mariadalfonso commented Nov 10, 2020

cmsbuild commented Nov 10, 2020 • edited

cmsbuild commented Nov 10, 2020

cmsbuild commented Nov 10, 2020

cmsbuild commented Nov 10, 2020

mariadalfonso commented Nov 10, 2020

cmsbuild commented Nov 10, 2020

silviodonato commented Nov 10, 2020

hqucms commented Oct 8, 2020 •

edited

cmsbuild commented Oct 8, 2020 •

edited

cmsbuild commented Nov 10, 2020 •

edited