PUID UL18 for nanoAOD (backport) #32951

alefisico · 2021-02-19T12:48:07Z

PR description:

This is the backport of #32887

PR validation:

Passes all the runTheMatrix.py -l limited -i all --ibeos tests.

FYI: @camclean @mariadalfonso @gouskos

cmsbuild · 2021-02-19T12:48:32Z

A new Pull Request was created by @alefisico (Alejandro Gomez Espinosa) for CMSSW_10_6_X.

It involves the following packages:

PhysicsTools/NanoAOD

@cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please review it and eventually sign? Thanks.
@gpetruc, @peruzzim, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

silviodonato · 2021-02-23T12:43:08Z

I'd say that we have to use run2_nanoAOD_106Xv1 to avoid changes in NanoAODV8

alefisico · 2021-02-23T13:20:13Z

I'd say that we have to use run2_nanoAOD_106Xv1 to avoid changes in NanoAODV8

sorry, where?

mariadalfonso · 2021-02-23T13:27:10Z

@silviodonato @alefisico
let's wait for the #32960 to be merged
after that will be cleaner to complete this backport

silviodonato · 2021-03-02T12:27:36Z

@alefisico we should have no change for the past UL NanoV8. This means no change if run2_nanoAOD_106Xv1 and ~run2_nanoAOD_devel (see #32960) cc @mariadalfonso

alefisico · 2021-03-04T13:37:33Z

PhysicsTools/NanoAOD/python/jets_cff.py

@@ -256,7 +258,10 @@
    )
 for modifier in run2_miniAOD_80XLegacy, run2_nanoAOD_94X2016:
  modifier.toModify( jetTable.variables, jetId = Var("userInt('tightIdLepVeto')*4+userInt('tightId')*2+userInt('looseId')",int,doc="Jet ID flags bit1 is loose, bit2 is tight, bit3 is tightLepVeto"))
+run2_nanoAOD_102Xv1.toModify( jetTable.variables, puIdDisc = Var("userFloat('puId102XDisc')",float,doc="Pileup ID discriminant with 102X (2018) training",precision=10) )
+run2_nanoAOD_102Xv1.toModify( jetTable.variables, puId = Var("userInt('pileupJetId:fullId')",int,doc="Pileup ID flags for pre-UL trainings") )


after #32960 it means that I should leave pu102XDisc and pileupJetId:fullId as it wasm and include:

run2_nanoAOD_devel.toModify( puIdDisc = Var("userFloat('puId106XUL18Disc')", float,doc="Pileup ID discriminant with 106X (2018) training",precision=10) ) run2_nanoAOD_devel.toModify( puId = Var("userInt('puId106XUL18Id')", int,doc="Pileup ID flags with 106X (2018) training") )

Is this correct @mariadalfonso ?

mariadalfonso · 2021-03-05T10:25:59Z

PhysicsTools/NanoAOD/python/jets_cff.py

+run2_nanoAOD_102Xv1.toModify( jetTable.variables, puIdDisc = Var("userFloat('puId102XDisc')",float,doc="Pileup ID discriminant with 102X (2018) training",precision=10) )
+run2_nanoAOD_102Xv1.toModify( jetTable.variables, puId = Var("userInt('pileupJetId:fullId')",int,doc="Pileup ID flags for pre-UL trainings") )


you can change line 261 and 262 into

( (run2_nanoAOD_106Xv1 and ~run2_nanoAOD_devel) | run2_nanoAOD_102Xv1).toModify( jetTable.variables, puIdDisc = Var("userFloat('puId102XDisc')",float,doc="Pileup ID discriminant with 102X (2018) training",precision=10) )
( (run2_nanoAOD_106Xv1 and ~run2_nanoAOD_devel) | run2_nanoAOD_102Xv1).toModify( jetTable.variables, puId = Var("userInt('pileupJetId:fullId')",int,doc="Pileup ID flags for pre-UL trainings") )

ok thanks, I'll commit a new version

@alefisico
can you replace and with & ?
so that we have
( (run2_nanoAOD_106Xv1 & ~run2_nanoAOD_devel) | run2_nanoAOD_102Xv1 ).toModify(

cmsbuild · 2021-03-05T10:51:01Z

Pull request #32951 was updated. @cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please check and sign again.

mariadalfonso · 2021-03-05T12:36:39Z

please test

cmsbuild · 2021-03-05T12:41:59Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-355152/13298/summary.html
COMMIT: ba172d6
CMSSW: CMSSW_10_6_X_2021-03-05-1100/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32951/13298/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation warning when building: See details on the summary page.

cmsbuild · 2021-03-05T13:48:13Z

Pull request #32951 was updated. @cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please check and sign again.

silviodonato · 2021-03-05T14:29:21Z

please test

cmsbuild · 2021-03-05T17:30:34Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-355152/13302/summary.html
COMMIT: 8506a9f
CMSSW: CMSSW_10_6_X_2021-03-05-1100/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32951/13302/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 3215540
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3215204
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 143 log files, 29 edm output root files, 35 DQM output files

mariadalfonso · 2021-03-08T16:38:50Z

@alefisico

no differences are shown in the runTheMatrix. However I would have expected to see

wf 1325.81 of type 106Xv1 -- no expected changes, thanks to this (run2_nanoAOD_106Xv1 and ~run2_nanoAOD_devel)
wf 136.8523 of type 106Xv2 -- expected changes, this would take the default that is now the UL18.

Can you check locally ?

alefisico · 2021-03-09T18:07:16Z

@alefisico

no differences are shown in the runTheMatrix. However I would have expected to see

wf 1325.81 of type 106Xv1 -- no expected changes, thanks to this (run2_nanoAOD_106Xv1 and ~run2_nanoAOD_devel)
wf 136.8523 of type 106Xv2 -- expected changes, this would take the default that is now the UL18.

Can you check locally ?

actually, the ~run2_nanoAOD_devel part is reverting the puIdDisc and puId to the 102X version and not the default 106XUL18

alefisico · 2021-03-09T18:08:01Z

sorry, I close it by mistake

cmsbuild · 2021-03-10T09:06:05Z

Pull request #32951 was updated. @cmsbuild, @mariadalfonso, @gouskos, @fgolf can you please check and sign again.

mariadalfonso · 2021-03-10T09:24:59Z

please test

cmsbuild · 2021-03-10T12:19:49Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-355152/13405/summary.html
COMMIT: 343c10c
CMSSW: CMSSW_10_6_X_2021-03-09-2300/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32951/13405/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 3215540
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3215198
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 143 log files, 29 edm output root files, 35 DQM output files

mariadalfonso · 2021-03-10T14:39:17Z

+xpog

wf 1325.81 of type 106Xv1 untouched
wf 136.8523 of type 106Xv2 updated

cmsbuild · 2021-03-10T14:39:36Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-03-10T15:25:54Z

+1

alefisico added 2 commits February 17, 2021 11:06

adding PUID UL18 nanoAOD

222f5ab

add PUID UL18 nanoAOD

728b46e

cmsbuild added this to the CMSSW_10_6_X milestone Feb 19, 2021

cmsbuild added orp-pending pending-signatures tests-pending xpog-pending labels Feb 19, 2021

mariadalfonso mentioned this pull request Feb 19, 2021

features for V9 nanoAOD cms-nanoAOD/cmssw#555

Closed

55 tasks

alefisico commented Mar 4, 2021

View reviewed changes

mariadalfonso reviewed Mar 5, 2021

View reviewed changes

fix modifiers for PUID

ba172d6

cmsbuild added tests-started and removed tests-pending labels Mar 5, 2021

cmsbuild added tests-rejected and removed tests-started labels Mar 5, 2021

fix modifiers PUID

8506a9f

cmsbuild added tests-pending and removed tests-rejected labels Mar 5, 2021

cmsbuild added tests-started and removed tests-pending labels Mar 5, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 5, 2021

alefisico closed this Mar 9, 2021

alefisico reopened this Mar 9, 2021

fix modifiers puID

343c10c

cmsbuild added tests-pending and removed tests-approved labels Mar 10, 2021

cmsbuild added tests-started and removed tests-pending labels Mar 10, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 10, 2021

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Mar 10, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 10, 2021

cmsbuild merged commit 2a35bf6 into cms-sw:CMSSW_10_6_X Mar 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PUID UL18 for nanoAOD (backport) #32951

PUID UL18 for nanoAOD (backport) #32951

alefisico commented Feb 19, 2021

cmsbuild commented Feb 19, 2021

silviodonato commented Feb 23, 2021

alefisico commented Feb 23, 2021

mariadalfonso commented Feb 23, 2021

silviodonato commented Mar 2, 2021

alefisico Mar 4, 2021

mariadalfonso Mar 5, 2021

alefisico Mar 5, 2021

mariadalfonso Mar 9, 2021

cmsbuild commented Mar 5, 2021

mariadalfonso commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

silviodonato commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

mariadalfonso commented Mar 8, 2021

alefisico commented Mar 9, 2021 •

edited

alefisico commented Mar 9, 2021

cmsbuild commented Mar 10, 2021

mariadalfonso commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

mariadalfonso commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

qliphy commented Mar 10, 2021

		run2_nanoAOD_102Xv1.toModify( jetTable.variables, puIdDisc = Var("userFloat('puId102XDisc')",float,doc="Pileup ID discriminant with 102X (2018) training",precision=10) )
		run2_nanoAOD_102Xv1.toModify( jetTable.variables, puId = Var("userInt('pileupJetId:fullId')",int,doc="Pileup ID flags for pre-UL trainings") )

PUID UL18 for nanoAOD (backport) #32951

PUID UL18 for nanoAOD (backport) #32951

Conversation

alefisico commented Feb 19, 2021

PR description:

PR validation:

cmsbuild commented Feb 19, 2021

silviodonato commented Feb 23, 2021

alefisico commented Feb 23, 2021

mariadalfonso commented Feb 23, 2021

silviodonato commented Mar 2, 2021

alefisico Mar 4, 2021

Choose a reason for hiding this comment

mariadalfonso Mar 5, 2021

Choose a reason for hiding this comment

alefisico Mar 5, 2021

Choose a reason for hiding this comment

mariadalfonso Mar 9, 2021

Choose a reason for hiding this comment

cmsbuild commented Mar 5, 2021

mariadalfonso commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

Build

cmsbuild commented Mar 5, 2021

silviodonato commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

Comparison Summary

mariadalfonso commented Mar 8, 2021

alefisico commented Mar 9, 2021 • edited

alefisico commented Mar 9, 2021

cmsbuild commented Mar 10, 2021

mariadalfonso commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

Comparison Summary

mariadalfonso commented Mar 10, 2021

cmsbuild commented Mar 10, 2021

qliphy commented Mar 10, 2021

alefisico commented Mar 9, 2021 •

edited