keeping MET sumPtUnclustered (backport of #28370) #29276

danbarto · 2020-03-23T19:42:59Z

PR description:

The sum of the unclustered energy is one of the inputs to MET significance, but is currently not stored in miniAOD, making recomputation of the significance and the MET covariance matrix difficult. With this PR, the calculated sum of the pTs of the unclustered particles is stored, and can be accessed with metSumPtUnclustered(), similar to metSignificance().

PR validation:

All tests have been run, and were passed.

if this PR is a backport please specify the original PR and why you need to backport that PR:

backport of #28370, needed for nanoAODv7

cmsbuild · 2020-03-23T19:43:28Z

A new Pull Request was created by @danbarto (Daniel) for CMSSW_10_2_X.

It involves the following packages:

DataFormats/PatCandidates
PhysicsTools/PatAlgos
RecoMET/METAlgorithms
RecoMET/METProducers

@perrotta, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @hatakeyamak, @emilbols, @peruzzim, @seemasharmafnal, @mmarionncern, @ahinzmann, @smoortga, @jdolen, @ferencek, @rovere, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @mariadalfonso, @clelange, @riga, @JyothsnaKomaragiri, @mverzett, @cbernet, @gpetruc, @andrzejnovak this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

Backported from Keeping sumPt from MET Significance calculation #28370

slava77 · 2020-03-23T20:17:20Z

@cmsbuild please test

cmsbuild · 2020-03-23T20:21:24Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5334/console Started: 2020/03/23 21:28

peruzzim · 2020-03-23T20:33:38Z

thank you really much @danbarto for providing this backport

cmsbuild · 2020-03-23T21:58:35Z

+1
Tested at: aab6c2e
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6622b1/5334/summary.html
CMSSW: CMSSW_10_2_X_2020-03-23-1100
SCRAM_ARCH: slc6_amd64_gcc700

cmsbuild · 2020-03-23T21:58:38Z

Comparison job queued.

cmsbuild · 2020-03-23T22:42:55Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6622b1/5334/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 31
DQMHistoTests: Total histograms compared: 3007491
DQMHistoTests: Total failures: 4
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3007297
DQMHistoTests: Total skipped: 190
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
Checked 129 log files, 14 edm output root files, 31 DQM output files

perrotta · 2020-03-24T09:58:36Z

DataFormats/PatCandidates/interface/MET.h

@@ -250,6 +254,8 @@ namespace pat {

      // MET significance
      double metSig_;
+      // MET metSumPtUnclustered for MET Significance
+      double metSumPtUnclustered_;


Why to change the name of this private class member wrt the one in the master?

Sorry, my mistake. I'll fix that.

perrotta · 2020-03-24T10:10:42Z

DataFormats/PatCandidates/src/classes_def_objects.xml

-  <class name="pat::MET"  ClassVersion="15">
-    <version ClassVersion="15" checksum="428901429"/>
+  <class name="pat::MET"  ClassVersion="16">
+    <version ClassVersion="16" checksum="2969659150"/>


@Dr15Jones @slava77 (also for my own understanding): is it correct that the same ClassVersion (here "16") can get two different checksum's in different releases (here 10_2 vs 11_1)?

Once a checksum has been assigned a version number it should not be changed. The only exception is when it is deemed that version was never used for any useful jobs.

changes of existing versions in a release are not allowed

same versions in different releases with different checksums are not allowed

Thank you @Dr15Jones and @slava77

I assume then that, once metSumPtUnclustered_ is renamed as sumPtUnclustered_. as in the master, the very same structure in terms of class members and methods will be there for this backport and the master. Therefore the same version "16" and checksum as the master can be also backported.

if for whatever reason we have to increment a version number here, we should make sure to add it to the master as well

if for whatever reason we have to increment a version number here, we should make sure to add it to the master as well

Once class members and methods are sinchronized as suggested above, the same class version can be considered for both releases.

The only difference in the code itself between the master and the 10_2 backport will come from the application of the clang format updates in the master, and not yet in 10_2.

That is not enough to trigger a version number change. (And even if so, then probably both

DataFormats/PatCandidates/interface/MET.h DataFormats/PatCandidates/src/MET.cc

can be directly copied from the version in the master, and at that point they will be absolutely identical in the two CMSSW releases).

After fixing the inconsistencies in MET.h/MET.cc the version is now consistent with the master. I'll update the PR soon.

perrotta · 2020-03-24T10:14:16Z

RecoMET/METAlgorithms/interface/METSignificance.h

@@ -44,7 +44,8 @@ namespace metsig {
 					  JME::JetResolution & resPtObj,
 					  JME::JetResolution & resPhiObj,
 					  JME::JetResolutionScaleFactor & resSFObj,
-					  bool isRealData);
+					  bool isRealData,
+            double& sumPtUnclustered);


fix indentation

perrotta · 2020-03-24T10:14:24Z

RecoMET/METAlgorithms/src/METSignificance.cc

@@ -54,7 +54,8 @@ metsig::METSignificance::getCovariance(const edm::View<reco::Jet>& jets,
 				       JME::JetResolution& resPtObj,
 				       JME::JetResolution& resPhiObj,
 				       JME::JetResolutionScaleFactor& resSFObj,
-				       bool isRealData) {
+				       bool isRealData,
+               double& sumPtUnclustered) {


fix indentation

cmsbuild · 2020-03-24T15:53:48Z

Pull request #29276 was updated. @perrotta, @cmsbuild, @santocch, @slava77 can you please check and sign again.

perrotta · 2020-03-24T15:58:52Z

@cmsbuild please test

cmsbuild · 2020-03-25T16:20:47Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18f2ad/5374/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 214 differences found in the comparisons
DQMHistoTests: Total files compared: 31
DQMHistoTests: Total histograms compared: 3007475
DQMHistoTests: Total failures: 33
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3007252
DQMHistoTests: Total skipped: 190
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 9.013 KiB( 30 files compared)
DQMHistoSizes: changed ( 1325.7 ): 9.013 KiB Physics/NanoAODDQM
Checked 129 log files, 14 edm output root files, 31 DQM output files

cmsbuild · 2020-03-26T13:34:55Z

The tests are being triggered in jenkins.
Tested with other pull request(s) #29294,cms-sw/cmsdist#5697
Test Parameters:

MATRIX_EXTRAS = 1325.7,1325.6,136.8521,1325.8,136.7722,1329.1,136.7952
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5398/console Started: 2020/03/26 14:42

cmsbuild · 2020-03-26T15:48:47Z

+1
Tested at: f08ddc4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18f2ad/5398/summary.html
CMSSW: CMSSW_10_2_X_2020-03-26-1100
SCRAM_ARCH: slc6_amd64_gcc700

cmsbuild · 2020-03-26T15:48:54Z

Comparison job queued.

cmsbuild · 2020-03-26T16:32:07Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-18f2ad/5398/summary.html

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 214 differences found in the comparisons
DQMHistoTests: Total files compared: 31
DQMHistoTests: Total histograms compared: 3007475
DQMHistoTests: Total failures: 34
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3007251
DQMHistoTests: Total skipped: 190
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 9.013 KiB( 30 files compared)
DQMHistoSizes: changed ( 1325.7 ): 9.013 KiB Physics/NanoAODDQM
Checked 129 log files, 14 edm output root files, 31 DQM output files

perrotta · 2020-03-27T14:50:29Z

+1

"double sumPtUnclustered" added to the MET DataFormat
Jenkins tests pass and show no other differences than this additional double added to the dataformat; changes observable in the output of automatic tests originate from NanoAODv7 code updates [102X] #29294, which was run together in those tests
Since only one additional double is added to the pat::MET, while all other class members stay untouched, I think this can fulfil the requirements for a backport. The algorithm to read from it will need to be enabled with an era modifier.

silviodonato · 2020-03-30T08:34:18Z

merge
@santocch

santocch · 2020-03-30T13:54:58Z

+1

cmsbuild · 2020-03-30T13:55:28Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_1_X is complete. This pull request will be automatically merged.

keeping MET sumPtUnclustered (back port)

aab6c2e

cmsbuild added this to the CMSSW_10_2_X milestone Mar 23, 2020

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 23, 2020

cmsbuild added tests-started and removed tests-pending labels Mar 23, 2020

peruzzim mentioned this pull request Mar 23, 2020

NanoAODv7 campaign preparation #29180

Open

3 tasks

cmsbuild added tests-approved and removed tests-started labels Mar 23, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 23, 2020

perrotta reviewed Mar 24, 2020

View reviewed changes

sumPtUnclustered name fix, update version, indentation fix

e7f74fb

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Mar 24, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 25, 2020

cmsbuild added tests-started and removed tests-approved labels Mar 26, 2020

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Mar 26, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 26, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 27, 2020

cmsbuild added orp-approved and removed orp-pending labels Mar 30, 2020

cmsbuild merged commit d19efa1 into cms-sw:CMSSW_10_2_X Mar 30, 2020

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Mar 30, 2020

slava77 mentioned this pull request Apr 8, 2020

Backport of inverse beta variable in pat::Muons for 10_6_X #29324

Merged

perrotta mentioned this pull request Apr 21, 2020

keeping METsumPtUnclustered (backport of #28370) #29521

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

keeping MET sumPtUnclustered (backport of #28370) #29276

keeping MET sumPtUnclustered (backport of #28370) #29276

danbarto commented Mar 23, 2020

cmsbuild commented Mar 23, 2020 •

edited

slava77 commented Mar 23, 2020

cmsbuild commented Mar 23, 2020 •

edited

peruzzim commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

perrotta Mar 24, 2020

danbarto Mar 24, 2020

perrotta Mar 24, 2020

Dr15Jones Mar 24, 2020

slava77 Mar 24, 2020

perrotta Mar 24, 2020

slava77 Mar 24, 2020

perrotta Mar 24, 2020

danbarto Mar 24, 2020

perrotta Mar 24, 2020

perrotta Mar 24, 2020

cmsbuild commented Mar 24, 2020

perrotta commented Mar 24, 2020

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 26, 2020 •

edited

cmsbuild commented Mar 26, 2020

cmsbuild commented Mar 26, 2020

cmsbuild commented Mar 26, 2020

perrotta commented Mar 27, 2020

silviodonato commented Mar 30, 2020

santocch commented Mar 30, 2020

cmsbuild commented Mar 30, 2020

keeping MET sumPtUnclustered (backport of #28370) #29276

keeping MET sumPtUnclustered (backport of #28370) #29276

Conversation

danbarto commented Mar 23, 2020

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Mar 23, 2020 • edited

slava77 commented Mar 23, 2020

cmsbuild commented Mar 23, 2020 • edited

peruzzim commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

cmsbuild commented Mar 23, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Mar 24, 2020

perrotta commented Mar 24, 2020

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 26, 2020 • edited

cmsbuild commented Mar 26, 2020

cmsbuild commented Mar 26, 2020

cmsbuild commented Mar 26, 2020

perrotta commented Mar 27, 2020

silviodonato commented Mar 30, 2020

santocch commented Mar 30, 2020

cmsbuild commented Mar 30, 2020

cmsbuild commented Mar 23, 2020 •

edited

cmsbuild commented Mar 23, 2020 •

edited

cmsbuild commented Mar 26, 2020 •

edited