Extension of electron matching in nanoAOD #32966

sscruz · 2021-02-22T17:08:42Z

PR description:

Electron matching to MC truth has some inefficiency due to FSR. This PR extends the matching to recover such cases, by adding the possibility that electrons can be matched to gendressed particles. This issue has been reported here:
https://indico.cern.ch/event/962278/contributions/4060711/attachments/2136665/3599026/electron_matching_nano_nov4.pdf

and further discussed here:

cms-nanoAOD#552

The electron matching is performed as follows:

Electrons are first matched to a collection of dressed electrons. This collection is identical to the GenDressedLepton collection that is stored in nanoAOD, but with a looser pt cut. If there's a match, the Electron_genPartFlav variable is set to 1 or 15, depending on the mother of the dressed lepton. Electron_genPartIdx is the index of the electron en GenPart that corresponds to the constituent of the matched dressed leptons with highest pt and abs(pdgId)==11.
For the electrons not matched in the previous step, the logic is the same as in the previous implementation: electrons are matched to GenPart and then Electron_genPartIdx and Electron_genPartFlav are assigned according to this match.

cmsbuild · 2021-02-22T17:13:58Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32966/21201

This PR adds an extra 32KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32966/21201/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32966/21201/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32966/21201/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32966/21201/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2021-02-22T17:27:40Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32966/21202

This PR adds an extra 32KB to repository

cmsbuild · 2021-02-22T17:28:03Z

A new Pull Request was created by @sscruz for master.

It involves the following packages:

PhysicsTools/HepMCCandAlgos
PhysicsTools/NanoAOD

@cmsbuild, @santocch, @mariadalfonso, @gouskos, @fgolf can you please review it and eventually sign? Thanks.
@gpetruc, @peruzzim, @swertz this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2021-02-22T18:13:29Z

please test

cmsbuild · 2021-02-22T21:52:14Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6df655/13026/summary.html
COMMIT: bf77de3
CMSSW: CMSSW_11_3_X_2021-02-22-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32966/13026/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2958 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2750983
DQMHistoTests: Total failures: 73889
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2677072
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 156 log files, 37 edm output root files, 37 DQM output files

qliphy · 2021-02-23T01:45:58Z

please test

cmsbuild · 2021-03-23T17:50:55Z

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6df655/13701/summary.html
COMMIT: 00c4431
CMSSW: CMSSW_11_3_X_2021-03-23-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32966/13701/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals

1325.81

----- Begin Fatal Exception 23-Mar-2021 18:00:27 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
   Additional Info:
      [a] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class reco::JetFlavourInfo with content:

#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef EIGEN_DONT_PARALLELIZE
  #define EIGEN_DONT_PARALLELIZE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 90300
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <utility>
#include <vector>

#include "DataFormats/Common/interface/Wrapper.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfoMatching.h"
#include "SimDataFormats/JetMatching/interface/JetFlavour.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourMatching.h"
#include "SimDataFormats/JetMatching/interface/MatchedPartons.h"
#include "SimDataFormats/JetMatching/interface/JetMatchedPartons.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

10024.0

----- Begin Fatal Exception 23-Mar-2021 18:18:23 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
   Additional Info:
      [a] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class reco::JetFlavourInfo with content:

#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef EIGEN_DONT_PARALLELIZE
  #define EIGEN_DONT_PARALLELIZE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 90300
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <utility>
#include <vector>

#include "DataFormats/Common/interface/Wrapper.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfoMatching.h"
#include "SimDataFormats/JetMatching/interface/JetFlavour.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourMatching.h"
#include "SimDataFormats/JetMatching/interface/MatchedPartons.h"
#include "SimDataFormats/JetMatching/interface/JetMatchedPartons.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

25202.0

----- Begin Fatal Exception 23-Mar-2021 18:20:35 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
   Additional Info:
      [a] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class reco::JetFlavourInfo with content:

#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef EIGEN_DONT_PARALLELIZE
  #define EIGEN_DONT_PARALLELIZE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 90300
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <utility>
#include <vector>

#include "DataFormats/Common/interface/Wrapper.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfoMatching.h"
#include "SimDataFormats/JetMatching/interface/JetFlavour.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourMatching.h"
#include "SimDataFormats/JetMatching/interface/MatchedPartons.h"
#include "SimDataFormats/JetMatching/interface/JetMatchedPartons.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

1325.611325.61_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOAODMC2017_106XMiniAODv1/step2_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOAODMC2017_106XMiniAODv1.log
1325.811325.81_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1/step2_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1.log
11634.9111634.91_TTbar_14TeV+2021_trackdnn+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_TTbar_14TeV+2021_trackdnn+TTbar_14TeV_TuneCP5_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log

smuzaffar · 2021-03-23T19:19:21Z

please test
a newer full IB is available now

cmsbuild · 2021-03-23T21:52:34Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6df655/13709/summary.html
COMMIT: 00c4431
CMSSW: CMSSW_11_3_X_2021-03-23-1400/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32966/13709/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 13 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2639935
DQMHistoTests: Total failures: 15
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2639897
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 36 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: found differences in 7 / 36 workflows

mariadalfonso · 2021-03-24T12:45:46Z

hi, validating the new matching with more stats I found an obvious bug in the new implementation, that is fixed now.

can you upload this large stat validation in the description?
from the integration results small stat, all seems in order

silviodonato · 2021-03-25T14:10:36Z

unassign analysis

sscruz · 2021-03-26T17:07:53Z

hi, validating the new matching with more stats I found an obvious bug in the new implementation, that is fixed now.

can you upload this large stat validation in the description?
from the integration results small stat, all seems in order

sure, sorry for the late reply. this is what I have:
https://sesanche.web.cern.ch/projects/electronmatching/validation/Electron_genPartFlav.png
https://sesanche.web.cern.ch/projects/electronmatching/validation/Electron_genPartIdx.png

mariadalfonso · 2021-03-26T17:52:20Z

+xpog

silviodonato · 2021-03-29T10:25:13Z

merge

mariadalfonso · 2021-03-29T11:36:08Z

@sscruz
can you prepare the backport to 10_6_X ?

note that we need to safeguard the current ongoing v8 production in 10_6_X
(run2_nanoAOD_106Xv1 & ~run2_nanoAOD_devel).toModify(xyzTable , xyxVariable = None)

santocch · 2021-03-30T19:42:36Z

+1

cmsbuild · 2021-03-30T19:42:57Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

Sergio Sanchez Cruz added 2 commits February 22, 2021 17:57

Extending electron matching

c953322

Missing commint

d5148f7

cmsbuild added this to the CMSSW_11_3_X milestone Feb 22, 2021

cmsbuild added analysis-pending code-checks-pending orp-pending pending-signatures tests-pending xpog-pending labels Feb 22, 2021

cmsbuild added code-checks-rejected and removed code-checks-pending labels Feb 22, 2021

Codechecks

bf77de3

cmsbuild added code-checks-pending and removed code-checks-rejected labels Feb 22, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 22, 2021

mariadalfonso mentioned this pull request Feb 22, 2021

features for V9 nanoAOD cms-nanoAOD/cmssw#555

Closed

55 tasks

cmsbuild added tests-started and removed tests-pending labels Feb 22, 2021

cmsbuild added tests-approved and removed tests-started labels Feb 22, 2021

cmsbuild added tests-started and removed tests-approved labels Feb 23, 2021

qliphy mentioned this pull request Feb 23, 2021

Workflow 11634.911 appearing broken in several PR tests #32963

Closed

cmsbuild added tests-started and removed tests-pending labels Mar 23, 2021

cmsbuild added tests-rejected and removed tests-started labels Mar 23, 2021

mariadalfonso mentioned this pull request Mar 23, 2021

move JetMatching package from SimDataFormats into DataFormats #33189

Merged

cmsbuild added tests-started and removed tests-rejected labels Mar 23, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 23, 2021

cmsbuild added xpog-approved and removed xpog-pending labels Mar 26, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 29, 2021

cmsbuild merged commit 393780c into cms-sw:master Mar 29, 2021

cmsbuild added analysis-approved fully-signed and removed analysis-pending pending-signatures labels Mar 30, 2021

sscruz mentioned this pull request Apr 6, 2021

Extension of electron matching in nanoAOD (backport) #33347

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extension of electron matching in nanoAOD #32966

Extension of electron matching in nanoAOD #32966

sscruz commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

mariadalfonso commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

qliphy commented Feb 23, 2021

cmsbuild commented Mar 23, 2021

smuzaffar commented Mar 23, 2021

cmsbuild commented Mar 23, 2021

mariadalfonso commented Mar 24, 2021

silviodonato commented Mar 25, 2021

sscruz commented Mar 26, 2021

mariadalfonso commented Mar 26, 2021

silviodonato commented Mar 29, 2021

mariadalfonso commented Mar 29, 2021

santocch commented Mar 30, 2021

cmsbuild commented Mar 30, 2021

Extension of electron matching in nanoAOD #32966

Extension of electron matching in nanoAOD #32966

Conversation

sscruz commented Feb 22, 2021

PR description:

cmsbuild commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

mariadalfonso commented Feb 22, 2021

cmsbuild commented Feb 22, 2021

Comparison Summary

qliphy commented Feb 23, 2021

cmsbuild commented Mar 23, 2021

Unit Tests

RelVals

RelVals-INPUT

smuzaffar commented Mar 23, 2021

cmsbuild commented Mar 23, 2021

Comparison Summary

mariadalfonso commented Mar 24, 2021

silviodonato commented Mar 25, 2021

sscruz commented Mar 26, 2021

mariadalfonso commented Mar 26, 2021

silviodonato commented Mar 29, 2021

mariadalfonso commented Mar 29, 2021

santocch commented Mar 30, 2021

cmsbuild commented Mar 30, 2021