Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix in puppi photon matching #19587

Merged
merged 7 commits into from Jul 31, 2017
Merged

Conversation

ahinzmann
Copy link
Contributor

@ahinzmann ahinzmann commented Jul 6, 2017

Protect PUPPI photon matching against other PF candidate types that may overlap with photons in angular space.
This reduces the PUPPI MET tail in gamma+jets events.
Validation plots and further details given in the discussion of this PR below.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 6, 2017

A new Pull Request was created by @ahinzmann for master.

It involves the following packages:

CommonTools/PileupAlgos

@perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please review it and eventually sign? Thanks.
@jdolen, @rappoccio, @gkasieczka this is something you requested to watch as well.
@davidlange6 you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Jul 6, 2017 via email

@slava77
Copy link
Contributor

slava77 commented Jul 6, 2017

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 6, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21221/console Started: 2017/07/06 16:32

@@ -186,6 +186,7 @@ void PuppiPhoton::produce(edm::Event& iEvent, const edm::EventSetup& iSetup) {
}
// ------------------------------------------------------------------------------------------
bool PuppiPhoton::matchPFCandidate(const reco::Candidate *iPF,const reco::Candidate *iPho) {
if(iPF->pdgId() != iPho->pdgId()) return false;
double lDR = deltaR(iPF->eta(),iPF->phi(),iPho->eta(),iPho->phi());
for(unsigned int i0 = 0; i0 < pdgIds_.size(); i0++) {
if(std::abs(iPF->pdgId()) == pdgIds_[i0] && lDR < dRMatch_[i0]) return true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the change above apparently at least partially invalidates the logic here, where other pdgIds are checked specifically.
Should the loop over pdgIds_ be removed?

I thought that matching of a photon to an electron were still meaningful, but maybe I don't have a good memory of what is done in PuppiPhoton

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the matching is to make sure it is the vary same PF candidate, so the same PDG ID is required. So electrons can be matched to electrons and photons to photons, but not mixed.
The loop over pdg id allows to configure different dR criteria for different particle types.

@ahinzmann
Copy link
Contributor Author

@slava77 Does some 9_3_X-compatible HT>600 gamma+jet sample happen to exist to repeat the test?

@slava77
Copy link
Contributor

slava77 commented Jul 6, 2017 via email

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 6, 2017

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 6, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 6, 2017

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19587/21221/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /build/cmsbld/jenkins-workarea/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-19587/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016+HARVESTDR2_REMINIAOD_data2016

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 1790655
  • DQMHistoTests: Total failures: 44270
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 1746219
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 93 log files, 14 edm output root files, 23 DQM output files

@ahinzmann
Copy link
Contributor Author

Here are tests on a flat pT>200 GeV gamma+jets sample (test tails of MET distribution) and a flat pT>15 GeV QCD sample (test bulk of MET distribution).

The fix in the matching procedure is a minor effect, since the bug was almost completely fixed by #18341 already
Black= PF MET
Blue= PUPPI MET in CMSSW_9_3_X_2017-07-06-1100
Red= PUPPI MET including fix in matching procedure
met-matching-bug-fix.pdf

However, high MET tails are visible, which are due to three problems.

  1. The outdated spring16 photon ID is used in 93X. An example event with a 2 TeV photon doesn't pass this ID:
    PUPPI MET: pt 1994.9, phi -2.35, sumEt (2080.0)
    MET: pt 45.0, phi -2.47, sumEt (4096.4). rawMET: 68.0, MET with JES up/down: 59.5/30.6
    PV at x,y,z = -0.025, +0.068, +2.647, ndof: 159.3, score: (pt2 of clustered objects) 722936.9
    phot 0: pt 1936.9, supercluster eta +0.106, sigmaIetaIeta 0.009 (full5x5 shower shapes)
    PhotonCutBasedIDLoose 1
    PhotonCutBasedIDTight 1
    cutBasedPhotonID-Spring15-25ns-V1-standalone-loose 1
    cutBasedPhotonID-Spring15-25ns-V1-standalone-medium 0
    cutBasedPhotonID-Spring15-25ns-V1-standalone-tight 0
    cutBasedPhotonID-Spring15-50ns-V1-standalone-loose 1
    cutBasedPhotonID-Spring15-50ns-V1-standalone-medium 0
    cutBasedPhotonID-Spring15-50ns-V1-standalone-tight 0
    cutBasedPhotonID-Spring16-V2p2-loose 0
    cutBasedPhotonID-Spring16-V2p2-medium 0
    cutBasedPhotonID-Spring16-V2p2-tight 0
    mvaPhoID-Spring15-25ns-nonTrig-V2p1-wp90 1
    mvaPhoID-Spring15-50ns-nonTrig-V2p1-wp90 1
    mvaPhoID-Spring16-nonTrig-V1-wp80 0
    mvaPhoID-Spring16-nonTrig-V1-wp90 1

  2. In some cases high pT photons do not end up in the pat::Photon collection, because they pass non of the IDs. In this example event, the photon can be found in the CHS jet collection, but not in the photon collection and not in the PUPPI jet collection:
    jet CHS 0: pt 1197.3 (raw pt 1130.0, matched-calojet pt 1152.3), eta +0.36, btag CSVIVFv2 0.000, CMVAv2 0.000, pileup mva disc -0.32
    0
    constituent 1: pt 810.50, dz(pv) +0.000, pdgId +22, hcal energy fraction 0.00, puppi weight 0.000
    constituent 2: pt 0.25, dz(pv) +0.000, pdgId +22, hcal energy fraction 0.00, puppi weight 0.000

Therefore the PR was adapted to remove the photon ID requirement and use all PF photons and raise the photon pt cut from 10 GeV to 20 GeV to compensate for this. This was found to remove the photon-related MET tails in the photon+jets samples and keep the MET bulk in the QCD sample similar. This can be considered as the best we can do while 2017 egamma IDs are not available. It is better to have PUPPI MET working independent of the egamma IDs, rather than waiting for the last moment when full scale validation of the impact will be difficult.

Black= PF MET
Blue= PUPPI MET in CMSSW_9_3_X_2017-07-06-1100
Red= PUPPI MET removing photon ID requirement and use all PF photons and raising photon pT cut to 20 GeV

high pt Photon+jet samples
met-pt20pfphotons.pdf

low pt QCD sample
met_qcd-pt20pfphotons.pdf

  1. In some cases photons are mis-identified by the PF algorithm as neutral hadrons and identified as PU by the PUPPI algorithm. This an example where a CHS jet exists, but not corresponding PUPPI jet, since this is a misidentified photon:
    jet CHS 0: pt 1215.2 (raw pt 1122.8, matched-calojet pt 1290.2), eta +1.49, btag CSVIVFv2 0.000, CMVAv2 0.000, pileup mva disc -0.51
    3
    constituent 1: pt 1086.00, dz(pv) +0.000, pdgId +130, hcal energy fraction 1.00, puppi weight 0.073
    constituent 2: pt 33.22, dz(pv) +0.000, pdgId +130, hcal energy fraction 1.00, puppi weight 0.000
    constituent 3: pt 1.68, dz(pv) +3.885, pdgId +211, hcal energy fraction 0.00, puppi weight 0.000
    constituent 4: pt 1.11, dz(pv) -0.866, pdgId +211, hcal energy fraction 0.00, puppi weight 0.000
    constituent 5: pt 0.74, dz(pv) +0.000, pdgId +22, hcal energy fraction 0.00, puppi weight 0.000
    constituent 6: pt 0.49, dz(pv) +0.102, pdgId +211, hcal energy fraction 0.00, puppi weight 1.000

This can be fixed by adding a protection to the PUPPI algorithm to not reject high momentum neutral particles. This was implemented by gradually increasing the PUPPI weight from 0 to 1 from 0 to 200 GeV if this is larger than the weight given by the PUPPI algorithm.

Black= PF MET
Blue= PUPPI MET in CMSSW_9_3_X_2017-07-06-1100
Red= PUPPI MET removing photon ID requirement and use all PF photons and raising photon pT cut to 20 GeV and adding protection for high momentum neutrals

high pt Photon+jet samples
met-pt20pfphotonshighpt.pdf

low pt QCD sample
met_qcd-pt20pfphotonshighpt.pdf

@violatingcp tested the same change on top of 80X with high statistics DY and HT>600 gamma+jets.

Red=PF MET
Blue= PUPPI MET in CMSSW_8_0_X
Black= PUPPI MET removing photon ID requirement and use all PF photons and raising photon pT cut to 20 GeV and adding protection for high momentum neutrals

low pT DY samples
dy

high pT gamma+jets
gjets

@slava77 Since this became more than a simple bug fix, we'll have to discuss this in JetMET before this can be merged.

@@ -76,7 +71,25 @@ void PuppiPhoton::produce(edm::Event& iEvent, const edm::EventSetup& iSetup) {
edm::Handle<CandidateView> hPuppiProduct;
iEvent.getByToken(tokenPuppiCandidates_,hPuppiProduct);
const CandidateView *pupCol = hPuppiProduct.product();
for(CandidateView::const_iterator itPho = phoCol->begin(); itPho!=phoCol->end(); itPho++) {
if(usePFphotons_) {
for(CandidateView::const_iterator itPho = pfCol->begin(); itPho!=pfCol->end(); itPho++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @ahinzmann - this can instead be a range based loop

for ( const auto & pho : pfCol ) {...
if pho.pt() < pt_ continue...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this gives me a
"/afs/cern.ch/user/h/hinzmann/stable_13TeV/jetmet/CMSSW_9_3_0_pre1/src/CommonTools/PileupAlgos/plugins/PuppiPhoton.cc: In member function 'virtual void PuppiPhoton::produce(edm::Event&, const edm::EventSetup&)':
/afs/cern.ch/user/h/hinzmann/stable_13TeV/jetmet/CMSSW_9_3_0_pre1/src/CommonTools/PileupAlgos/plugins/PuppiPhoton.cc:75:27: error: 'begin' was not declared in this scope
for(const auto & pho : pfCol) {
"
what's the exact syntax for
const CandidateView *pfCol = hPFProduct.product();

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for ( const auto & pho : *pfCol )

for(CandidateView::const_iterator itPho = pfCol->begin(); itPho!=pfCol->end(); itPho++) {
iC++;
if(itPho->pt() < pt_) continue;
if(abs(itPho->pdgId())!=22) continue;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use std::abs

@cmsbuild
Copy link
Contributor

Pull request #19587 was updated. @perrotta, @cmsbuild, @slava77, @monttj, @davidlange6 can you please check and sign again.

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 28, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/21872/console Started: 2017/07/28 17:01

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19587/21872/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 224 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 2376534
  • DQMHistoTests: Total failures: 41724
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2334644
  • DQMHistoTests: Total skipped: 166
  • DQMHistoTests: Total Missing objects: 0
  • Checked 93 log files, 14 edm output root files, 23 DQM output files

@slava77
Copy link
Contributor

slava77 commented Jul 28, 2017

@davidlange6
Copy link
Contributor

merge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants