NanoAOD: add flag to keep all PS weights #31276

swertz · 2020-08-28T10:17:17Z

PR description:

Add a flag (default false) to keep all the available PS weights instead of the regular 4 (ISR/FSR x up/down), as asked by JME for their custom ntuple production. If set to true, the whole PS weight array is kept, apart from the first two entries (these correspond to either [XWGTUP * PSweight * match_eff, XWGTUP * PSweight] or [PSweight * match_eff, PSweight], so they'll be equal to 1 most of the time after normalizing the weights by the second entry, see Fix PSWeight being incorrectly scaled cms-nanoAOD/cmssw#506).
The flag is set to true for the nanoGEN format
Add the PS weights to nanoDQM
Fix the PS weight normalization in the case where the weights are parsed according to the lumi header (see Saving LHE weights in nanoAOD when using randomised parameter scans cms-nanoAOD/cmssw#441): the PS weights should be normalized by the Baseline weight (assumed to be the first PS weight seen).
Add a protection for the case of empty weight arrays - fixes Bug for MC genweight vector of size 1 cms-nanoAOD/cmssw#538

Note: was originally cms-nanoAOD#542
FYI @camclean @alefisico

PR validation:

Tested in 106X: produced nanoAOD files for TTToSemiLeptonic (2016 and 2018) and for one of the parameter scan files mentioned in cms-nanoAOD#441 [1].

[1] /store/mc/RunIIAutumn18MiniAOD/SMS-TChiWZ_ZToLL_mZMin-0p1_mC1-325to1000_TuneCP2_13TeV-madgraphMLM-pythia8/MINIAODSIM/GridpackScan_102X_upgrade2018_realistic_v15-v1/70000/FFE78E85-6F7A-9441-8963-980A38403D1F.root

cmsbuild · 2020-08-28T10:17:37Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-28T10:24:15Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31276/17971

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nanoDQM_cfi.py modified in PR(s): Enabling re-application of Type1 Corrections for PuppiMET and adding PUPPI MET Uncertainties #30922
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): GEN-only NanoAOD configs (NanoGEN) and cmsDriver support #30306, Enabling re-application of Type1 Corrections for PuppiMET and adding PUPPI MET Uncertainties #30922

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31276/17971/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31276/17971/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31276/17971/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31276/17971/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2020-08-28T11:37:06Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-28T11:43:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31276/17973

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File PhysicsTools/NanoAOD/python/nanoDQM_cfi.py modified in PR(s): Enabling re-application of Type1 Corrections for PuppiMET and adding PUPPI MET Uncertainties #30922
- File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): GEN-only NanoAOD configs (NanoGEN) and cmsDriver support #30306, Enabling re-application of Type1 Corrections for PuppiMET and adding PUPPI MET Uncertainties #30922

cmsbuild · 2020-08-28T11:44:22Z

A new Pull Request was created by @swertz (Sébastien Wertz) for master.

It involves the following packages:

PhysicsTools/NanoAOD

@gouskos, @cmsbuild, @fgolf, @mariadalfonso, @santocch, @peruzzim can you please review it and eventually sign? Thanks.
@gpetruc this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2020-08-28T12:16:41Z

please test

cmsbuild · 2020-08-28T12:17:03Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-08-27-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/8983/console Started: 2020/08/28 14:25

cmsbuild · 2020-08-28T13:38:46Z

+1
Tested at: 50bf2c3
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7cb620/8983/summary.html
CMSSW: CMSSW_11_2_X_2020-08-27-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-08-28T13:38:49Z

Comparison job queued.

cmsbuild · 2020-08-28T15:05:41Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7cb620/8983/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2609667
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2609644
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.324 KiB( 34 files compared)
DQMHistoSizes: changed ( 1325.7 ): 0.324 KiB Physics/NanoAODDQM
Checked 149 log files, 22 edm output root files, 35 DQM output files

swertz · 2020-09-22T08:30:38Z

Wouldn't the LHEScaleWeight or LHEPdfWeight be empty or not produced in cases where they don't exist?

Indeed, so it would be more consistent to also leave the PSWeight array empty if there are no weights.

agrohsje · 2020-09-22T11:03:30Z

My concern was not so much consistency, but more being stable for users. I would prefer keeping the current implementation.

mariadalfonso · 2020-09-22T17:43:25Z

+xpog
changes are inline with the PR description.

If activated by the flag, the PS weights appear as listed here
#31276 (comment)

agrohsje · 2020-09-22T18:34:20Z

+generators

qliphy · 2020-09-23T00:09:47Z

+1

mariadalfonso · 2020-09-23T08:29:42Z

@swertz
consider back-porting this PR in 10_6

agrohsje · 2020-09-23T09:59:39Z

@mariadalfonso What about 10_2 and 9_4? Can one backport?

mariadalfonso · 2020-09-23T14:49:32Z

@agrohsje
Master or 10_6 can perfectly run on EOY miniAOD make with 10_2 and 94X.
for the moment we are back porting all the nano only to 10_6 for the UL reprocessing.
Is there any particular reason to backport the GEN nano-fixes to previous releases ?

Note: NanoGen is only available in 11_2

agrohsje · 2020-09-23T14:57:46Z

v7 is currently using 10_2:
https://cms-pdmv.cern.ch/mcm/campaigns?prepid=Run*NanoAODv7
Will we move to 10_6 also for these?

mariadalfonso · 2020-09-23T20:07:28Z

v7 is currently using 10_2:
https://cms-pdmv.cern.ch/mcm/campaigns?prepid=Run*NanoAODv7
Will we move to 10_6 also for these?

We will not be making anymore V7 centrally, and we didn't get requests on fixing older releases for private renano of the V7.

agrohsje · 2020-09-24T06:51:09Z

Hi @mariadalfonso . The problem is that we had failing HIG requests due to the bug. So they would need to resubmit nano-aod. I assume for this it might be easiest to update v7?

mariadalfonso · 2020-09-24T08:34:22Z

Hi @mariadalfonso . The problem is that we had failing HIG requests due to the bug. So they would need to resubmit nano-aod. I assume for this it might be easiest to update v7?

ok, we will follow up with the PC group, so that's will be clear what are the requests for the EOY and 10_2 and how to best help the private reprocessing. There are other bug-fixes besides this one among the cms-nanoAOD#533

For the time being let's complete the 10_6

santocch · 2020-09-24T16:10:37Z

+1

cmsbuild · 2020-09-24T16:11:02Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

cmsbuild added this to the CMSSW_11_2_X milestone Aug 28, 2020

cmsbuild added analysis-pending code-checks-pending comparison-pending orp-pending pending-signatures tests-pending xpog-pending labels Aug 28, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Aug 28, 2020

mariadalfonso mentioned this pull request Aug 28, 2020

features for V8 nanoAOD cms-nanoAOD/cmssw#533

Closed

55 tasks

cmsbuild added code-checks-pending and removed code-checks-rejected labels Aug 28, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 28, 2020

cmsbuild added tests-started and removed tests-pending labels Aug 28, 2020

cmsbuild added tests-approved and removed tests-started labels Aug 28, 2020

cmsbuild removed the comparison-pending label Aug 28, 2020

cmsbuild added xpog-approved and removed xpog-pending labels Sep 22, 2020

cmsbuild added generators-approved and removed generators-pending labels Sep 22, 2020

cmsbuild added orp-approved and removed orp-pending labels Sep 23, 2020

qliphy merged commit 5cfc0ce into cms-sw:master Sep 23, 2020

This was referenced Sep 23, 2020

Phase2-gex13 Add a scenario where ME0 is replaced by GE0 #31541

Merged

Initial Changes for Incoporating GEM GE0 into the Reco Pipeline #31309

Merged

swertz mentioned this pull request Sep 24, 2020

[106X] NanoAOD: add flag to keep all PS weights #31564

Merged

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Sep 24, 2020

swertz mentioned this pull request Sep 30, 2020

[102X] NanoAOD: add flag to keep all PS weights #31620

Merged

mariadalfonso mentioned this pull request Oct 23, 2020

Backport of #30306, #31591, and #31973: NanoGen support in 10_6_X #31909

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NanoAOD: add flag to keep all PS weights #31276

NanoAOD: add flag to keep all PS weights #31276

swertz commented Aug 28, 2020 •

edited

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

mariadalfonso commented Aug 28, 2020

cmsbuild commented Aug 28, 2020 •

edited

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

swertz commented Sep 22, 2020

agrohsje commented Sep 22, 2020

mariadalfonso commented Sep 22, 2020

agrohsje commented Sep 22, 2020

qliphy commented Sep 23, 2020

mariadalfonso commented Sep 23, 2020

agrohsje commented Sep 23, 2020

mariadalfonso commented Sep 23, 2020

agrohsje commented Sep 23, 2020

mariadalfonso commented Sep 23, 2020

agrohsje commented Sep 24, 2020

mariadalfonso commented Sep 24, 2020 •

edited

santocch commented Sep 24, 2020

cmsbuild commented Sep 24, 2020

NanoAOD: add flag to keep all PS weights #31276

NanoAOD: add flag to keep all PS weights #31276

Conversation

swertz commented Aug 28, 2020 • edited

PR description:

PR validation:

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

mariadalfonso commented Aug 28, 2020

cmsbuild commented Aug 28, 2020 • edited

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

cmsbuild commented Aug 28, 2020

swertz commented Sep 22, 2020

agrohsje commented Sep 22, 2020

mariadalfonso commented Sep 22, 2020

agrohsje commented Sep 22, 2020

qliphy commented Sep 23, 2020

mariadalfonso commented Sep 23, 2020

agrohsje commented Sep 23, 2020

mariadalfonso commented Sep 23, 2020

agrohsje commented Sep 23, 2020

mariadalfonso commented Sep 23, 2020

agrohsje commented Sep 24, 2020

mariadalfonso commented Sep 24, 2020 • edited

santocch commented Sep 24, 2020

cmsbuild commented Sep 24, 2020

swertz commented Aug 28, 2020 •

edited

cmsbuild commented Aug 28, 2020 •

edited

mariadalfonso commented Sep 24, 2020 •

edited