Fix PSWeights in LumiHeader to work w diff pythia format

[dteague]

PR description:

For UL, a new format for PartonShower weights is used in the LumiHeader (isr:murfac=0.5 instead of isrDefLo), so this PR is primarily aimed at fixing this for creating the Gen weight table. Further code is added to avoid edge cases where weight names are missing in the header (possibly caused by these names being put in by hand).

For more discussion, look at this discussion as well as the comments from this PR on a different branch

PR validation:

Here is a list of files that were tested on. The code was tested in two different modes: keepAllPSWeights=True and keepAllPSWeights=False (ie only store 4 main variations).

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31982/19457

• This PR adds an extra 24KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc modified in PR(s): Workaround for NanoAOD LHE weights in newer MadGraph #31680, Add PDF sets to nano, prioritize by order in LHE file #31946

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-checks:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31982/19457/code-checks.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31982/19457/code-checks.patch | patch -p1
  You can also run scram build code-checks to apply code checks directly

[mariadalfonso]

assign generators

[mariadalfonso]

@dteague
please resolve the code-checks committing the result of scram build code-checks so that we can start the test

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31982/19478

• This PR adds an extra 24KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File PhysicsTools/NanoAOD/plugins/GenWeightsTableProducer.cc modified in PR(s): Workaround for NanoAOD LHE weights in newer MadGraph #31680, Add PDF sets to nano, prioritize by order in LHE file #31946

[cmsbuild]

New categories assigned: generators

@alberto-sanchez,@SiewYan,@GurpreetSinghChahal,@mkirsano,@agrohsje you have been requested to review this Pull request/Issue and eventually sign? Thanks

[cmsbuild]

A new Pull Request was created by @dteague (Dylan Teague) for master.

It involves the following packages:

PhysicsTools/NanoAOD

@SiewYan, @gouskos, @mkirsano, @cmsbuild, @GurpreetSinghChahal, @fgolf, @alberto-sanchez, @agrohsje, @mariadalfonso, @santocch can you please review it and eventually sign? Thanks.
@gpetruc, @peruzzim, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[mariadalfonso]

please test

[cmsbuild]

The tests are being triggered in jenkins.

• CMSSW_11_2_X_2020-10-29-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10402/console Started: 2020/10/30 09:41

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-9cd476/10402/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 35
• DQMHistoTests: Total histograms compared: 2544092
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2544069
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
• Checked 149 log files, 22 edm output root files, 35 DQM output files

[agrohsje]

Hi @dteague From this discussion: cms-nanoAOD#546 (comment)
I understand the PR addresses all cases (spaces in pre UL, as well as differences between P8.230 and P8.240) ?
Did you test?

[dteague]

Hi @dteague From this discussion: cms-nanoAOD#546 (comment)
I understand the PR addresses all cases (spaces in pre UL, as well as differences between P8.230 and P8.240) ?
Did you test?

For my tests, I tested the old and new P8 weight format, files that had no lumiheader (possibly has weights or not), and files with the errors in the lumi header (only with P8.230, though the code should work for the p8.240 as long as there is at least one weight name in the header). For specific files I tested on that are exemplars of the types of files I was testing for:

• Old Format: /DYJetsToLL_M-5to50_TuneCP5_13TeV-madgraphMLM-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v2/MINIAODSIM
• Old Format/Handmade: /TTWJetsToLNu_TuneCP5_PSweights_13TeV-amcatnloFXFX-madspin-pythia8/RunIIFall17MiniAODv2-PU2017_12Apr2018_94X_mc2017_realistic_v14-v1/MINIAODSIM
• New Format: /TTJets_TuneCP5_13TeV-amcatnloFXFX-pythia8/RunIISummer19UL18MiniAOD-106X_upgrade2018_realistic_v11_L1v1-v2/MINIAODSIM
• New Format/Handmade: None found
• No Header: /NMSSM_XToYHTo4b_MX-1200_TuneCP5_13TeV-madgraph-pythia8/RunIIFall17MiniAODv2-PU2017_12Apr2018_RP_94X_mc2017_realistic_v14-v1/MINIAODSIM

I think that should be exhaustive of the types of files one might find, at least from my understanding of the discussion in that issue thread, but if there are other sample that should be tested on, point me to them and I can test them out.

[mariadalfonso]

@agrohsje
the list provided by @dteague is enough ? or you envisage more testing ?

[agrohsje]

+1

[mariadalfonso]

+xpog

testing shows no difference as expected for the MC that are not interested by this fix

[silviodonato]

merge

[santocch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.