Change puppi weight compression

[ahinzmann]

Adresses issue #23366.

The puppi weight is stored with linear instead of log scale compression to 8 bits.

The new compression scheme has improved precision for weights around 0.5, while is was at 10% level before (see https://github.com/cms-sw/cmssw/files/2045014/weight_not_charged_data_MC.pdf)

The iorules allow to read files created with older releases within the precision of the old an new compression schemes.

The size increase was check with 1000 ttbar events with patMiniAOD_standard_cfg.
before this commit
patPackedCandidates_packedPFCandidates__PAT. 136489 20525

after this commit
patPackedCandidates_packedPFCandidates__PAT. 136489 20540.4

In addition, the puppi weight is now set for packed candidates created by the PUPPI algorithm, when cloning the packed candidates from MiniAOD.

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-23414/4954

[cmsbuild]

A new Pull Request was created by @ahinzmann for master.

It involves the following packages:

CommonTools/PileupAlgos
DataFormats/PatCandidates

@perrotta, @monttj, @cmsbuild, @slava77, @gpetruc, @arizzi can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @rovere, @jdolen, @gkasieczka, @cbernet, @gpetruc this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[slava77]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28366/console Started: 2018/06/01 15:09

[cmsbuild]

+1
Tested at: 85e34d9
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23414/28366/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23414/28366/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 278 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 2902471
• DQMHistoTests: Total failures: 484
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901797
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 128 log files, 14 edm output root files, 31 DQM output files

[perrotta]

This is how do puppi weights get modified by this PR (from the jenkins comparisons in the SinglePh2017B 136.788 workflow)

@ahinzmann for my own education. could you please describe in some detail how do the weights move from the old black curve to the new red one?
The hole between 0.45 and 0.55 is still there: I am missing why...
What is the meaning/origin of the few weights larger than 1?

[slava77]

On 6/8/18 1:31 AM, perrotta wrote: This is how do puppi weights get modified by this PR (from the jenkins comparisons in the SinglePh2017B 136.788 workflow) image <https://user-images.githubusercontent.com/4069749/41146994-6d0771f6-6b05-11e8-8835-3e90c1c91f45.png> @ahinzmann <https://github.com/ahinzmann> for my own education. could you please describe in some detail how do the weights move from the old black curve to the new red one? The hole between 0.45 and 0.55 is still there: I am missing why... What is the meaning/origin of the few weights larger than 1?

One thing to keep in mind is that the jenkins plotting for these plots is first loading the old file and then the new file. This means that the dictionaries of the old file will be applied to read the data from the files. I suggest to download the .root files locally and rerun the script inverting the order or "old" and "new" arguments.

[perrotta]

Thank you Slava for the suggestion. Indeed, if I plot it in the opposite order I get something definitely more understandable:

[perrotta]

+1

• Puppi weights stored with linear scale compression, as in the description: it recovers precision at intermediate values of such weights
• Tiny effects are propagated to miniAOD objects which depend on those weights, as expected
• Old weights are readable with the new dictionary
• Event size increase in the miniAOD output was verified in a real data 2017 workflow to be small and similar to the one reported for MC TTbar events in the description of the pull request:

-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
   1949.0 ->      1954.2          5      0.3   0.01     patJets_slimmedJetsPuppi__reRECO.
  24724.6 ->     24740.5         16      0.1   0.03     patPackedCandidates_packedPFCandidates__reRECO.
-------------------------------------------------------------
    54403 ->       54424         21             0.0     ALL BRANCHES

[fabiocos]

+1

[fabiocos]

merge