Backport of access to tauID and electron discrimination from miniAOD #17292

roger-wolf · 2017-01-26T11:47:30Z

Dear colleagues,
this is a 1:1 backport of the work we have invested last autumn to provide access to tau-ID and electron discrimination on miniAOD in 81X. The relevant PRs have been:

Extension of miniAOD event content to be able to run tau mva isolation discriminator on miniAOD input #15731
Extension of PAT TauPFEssential event content to be able to run anti-electron discriminators in miniAOD input #15747
Extension of miniAOD event content to be able to run tau mva isolation discriminator on miniAOD input #15758
Adding missing variable to be able to run the anti-electron discriminator on present trainings #15995
Extension of infrastructure to readout tau discriminators from miniAOD to anti-electron discriminator #16030

By that time we had no chance to get these developments backported in time to the 80X release series and had announced that we would like to do this once we are set. We have now prepared the first case where tau-ID and electron-discrimination have indeed been produced and can be applied to analyses based only on miniAOD information! This has been provided for the upcoming analyses of this year. It is consequently done in 80X.

The backport introduces necessary additional contents in a few tau-specific DataFormats. No existing contents or members are changed or removed. In standard workflows there is no access to the newly added data members, so the backport does not change standard reconstruction behaviour. It only allows that existing training files may also be accessed from miniAOD in addition to the unchanged standard access from AOD. This project is of great importance for the flexibility of tau analyses in 2016.

Since only information is added we do not expect any change of the default reconstruction behaviour (see discussion of 81X pull requests). We have made the standard tests that we usually do in preparation of our pull requests to verify that this is the case. The additional features have also been successfully tested with unit-tests that had been introduced by the time of the original pull requests to 81X (and backported accordingly). In addition the whole infrastructure has already been used for the production and extensive validation of new MVA weight files, and their application. A detailed report is planed by Alexander Nehrkorn for the TauPOG meeting during the upcoming CMS week.

A small set of modifications on top of this pull request (which is a plain backport as said) would follow to give official access to the new weight files, most probably also related to a new Global Tag request. We would highly appreciate if these modifications could make it into a new 80X analysis release if possible. I'm happy to give a report or just to drop a few words of further clarification during the RECO/AT meeting tomorrow. You can put me on the agenda and let me know in case.

Cheers,
Roger

cmsbuild · 2017-01-26T11:47:49Z

A new Pull Request was created by @roger-wolf (Roger Wolf) for CMSSW_8_0_X.

It involves the following packages:

DataFormats/PatCandidates
PhysicsTools/PatAlgos
RecoTauTag/Configuration
RecoTauTag/RecoTau

@cmsbuild, @cvuosalo, @slava77, @monttj, @davidlange6 can you please review it and eventually sign? Thanks.
@TaiSakuma, @gouskos, @JyothsnaKomaragiri, @imarches, @ahinzmann, @acaudron, @mmarionncern, @rappoccio, @jdolen, @nhanvtran, @gpetruc, @gkasieczka, @schoef, @ferencek, @mverzett, @mariadalfonso, @pvmulder, @cbernet this is something you requested to watch as well.
@davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

slava77 · 2017-01-26T14:13:32Z

@cmsbuild please test

cmsbuild · 2017-01-26T14:13:51Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17454/console Started: 2017/01/26 15:15

cmsbuild · 2017-01-26T16:01:01Z

+1
Tested at: b839f2c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17292/17454/summary.html

cmsbuild · 2017-01-26T16:01:05Z

Comparison job queued.

cmsbuild · 2017-01-26T16:30:04Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17292/17454/summary.html

slava77 · 2017-02-03T09:17:20Z

@roger-wolf
you mention #15758 in the PR description. This is an 80X PR.
I'm guessing that you had in mind #15778

roger-wolf · 2017-02-04T18:54:10Z

Hi Slava,

you are right -- #15758 is already a backport. So I must have mixed this in by incidence. Please note though that we would like to add one single variable (phiAtEcalEntrance_) to TauPFEssentials.h in PatCandidates if possible. This variable was missing for the backport of #15758 and the fix did not appear in time by that time. Since this is only an additional variable in the PAT event contents that is default filled 0 unless the new tauID sequence is run I hope that adding this member is not impossible.

Cheers,
Roger

roger-wolf · 2017-02-09T11:05:59Z

Hi all,

all tests passed all questions answered as far as there were any. Any reasons not to sign this off so that we could continue with a few more commits on top of this?

Thanx a lot!

Cheers,
Roger

slava77 · 2017-02-09T11:41:13Z

Hi Roger,
I haven't checked the backport yet. That's why it's not signed.

The PR description states "this is a 1:1 backport".
I made a quick attempt to add up to the +3,083 lines in this PR and was not successful:

 Extension of miniAOD event content to be able to run tau mva isolation discriminator on miniAOD input #15731 
https://github.com/cms-sw/cmssw/pull/15731	in CMSSW_8_1_X
+81 −43	 in 8
 Extension of PAT TauPFEssential event content to be able to run anti-electron discriminators in miniAOD input #15747 
https://github.com/cms-sw/cmssw/pull/15747	in CMSSW_8_1_X
+116 −41	 in 5

 Extension of miniAOD event content to be able to run tau mva isolation discriminator on miniAOD input #15758 
https://github.com/cms-sw/cmssw/pull/15758	in CMSSW_8_0_X from Sep 8, 2016 ??? 
+145 −22	 in 6
.. this does not belong in here as a reference

 Adding missing variable to be able to run the anti-electron discriminator on present trainings #15995 
https://github.com/cms-sw/cmssw/pull/15995	in CMSSW_8_1_X
+23 −10	in 7
 Extension of infrastructure to readout tau discriminators from miniAOD to anti-electron discriminator #16030 
https://github.com/cms-sw/cmssw/pull/16030	in CMSSW_8_1_X
+607 −24	in 6

.. this clearly doesn't add up
.. there was also
 Infrastructure to create the actual ValueMap pat::Tau-discriminator for tau isolation and ID #15778 
https://github.com/cms-sw/cmssw/pull/15778	in CMSSW_8_1_X
+1,775 −225	in 19
.. 81 + 116 +23+607+1775 = 2602 is still smaller than 3083 in this 80X PR

please check and update the list of already merged 81X/90X PRs in a way that it's clear this is actually a 1:1 backport.

If you plan additional commits on top of this which will be a clear backports of features already in 80X/90X, feel free to add them here

roger-wolf · 2017-02-09T11:48:06Z

Hm,

I actually made the backport based in the PR's to 81X -- Ok I'll re-check and let you know asap.

NB:
since you started the review already (many thanx for the care that you spent on it!) please let me know, whether you would still like me to match the files with the 81X/90X files to indicate that this is a backport.

Cheers,
Roger

slava77 · 2017-02-10T19:08:58Z

RecoTauTag/Configuration/test/QCD-validation.txt

@@ -0,0 +1,5 @@
+/store/relval/CMSSW_8_1_0_pre11/RelValQCD_Pt_80_120_13/GEN-SIM-RECO/81X_mcRun2_asymptotic_Candidate_2016_08_30_11_31_55-v1/00000/24F576D4-4D74-E611-8898-0025905A48BA.root


this seems inappropriate for 80X.
Please check and update to something based on 80X

These files are only used for our internal tests. They are anyhow obsolete, once the validation samples are deleted form relval at CERN. But I've added some more intuitive dummy files for the next commit.

slava77 · 2017-02-10T19:33:23Z

RecoTauTag/RecoTau/plugins/PATTauDiscriminationAgainstElectronMVA6.cc

+      mva_(0),
+      category_output_()
+  {
+    mva_ = new AntiElectronIDMVA6(cfg);


can we use a unique_ptr here as it was introduced in #16030

Ok -- done for the next commit.

slava77 · 2017-02-10T20:16:30Z

RecoTauTag/RecoTau/plugins/PATTauDiscriminationAgainstElectronMVA6.cc

+  std::string moduleLabel_;
+
+  AntiElectronIDMVA6* mva_;
+  float* mvaInput_;


what is this needed for here? it looks unused

has been removed. This is left over garbage that had been collected in 81X and also now in 80X.

cmsbuild · 2017-02-24T15:15:10Z

Pull request #17292 was updated. @cmsbuild, @cvuosalo, @slava77, @monttj, @davidlange6 can you please check and sign again.

slava77 · 2017-02-24T15:24:37Z

@cmsbuild please test

cmsbuild · 2017-02-24T15:24:52Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17952/console Started: 2017/02/24 16:26

cmsbuild · 2017-02-24T17:57:08Z

+1
Tested at: f212de7
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17292/17952/summary.html

cmsbuild · 2017-02-24T17:57:11Z

Comparison job queued.

cmsbuild · 2017-02-24T18:51:24Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17292/17952/summary.html

slava77 · 2017-03-01T03:23:35Z

+1

for #17292 f212de7

the backport of features needed to run from miniAOD is in line with the description and the follow up comments. The backport is a combination of contributions from Adding additional atomistic scripts and configs for s/w validation purposes to RecoTauTag/Configuration #13884, Extension of miniAOD event content to be able to run tau mva isolation discriminator on miniAOD input #15731, Infrastructure to create the actual ValueMap pat::Tau-discriminator for tau isolation and ID #15778, Adding missing variable to be able to run the anti-electron discriminator on present trainings #15995, Extension of infrastructure to readout tau discriminators from miniAOD to anti-electron discriminator #16030, Package of several technical changes: downgrade of edm::Warnings to edm::Info, removal of an unused parameter from cfi file, upward move of jet selection for tau seeding, switch off of production of null taus #17620. Changes are expected in miniAOD as a result of extension of patTau pfEssential.
jenkins tests pass and comparisons with baseline show no differences.

Local tests confirm no change in RECO and changes in patTau output products.
I'm putting this on hold, to be not merged until the rereco release is to be built.

slava77 · 2017-03-01T03:23:45Z

hold

cmsbuild · 2017-03-01T03:23:51Z

Pull request has been put on hold by @slava77
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

slava77 · 2017-04-06T07:06:57Z

@davidlange6
why was this merged?
This is nominally for miniAOD v2

davidlange6 · 2017-04-06T07:09:11Z

ok, i didn't understand that - should I revert?

slava77 · 2017-04-06T07:28:42Z

On 4/6/17 9:09 AM, David Lange wrote: ok, i didn't understand that - should I revert?

#17603 (comment) I think it's more appropriate to revert.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17292 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEdcbkZ7svumMZlvlXh91xeus9jkgJZlks5rtI-XgaJpZM4Lujad>.

roger-wolf added 3 commits January 18, 2017 17:33

adding quick validation suite to 80X tau-pog development branch

d987984

backport of miniAOD access for antie and mvaID from 81X to 80X

36925d5

adding test directory for TAUPOG specific short circiut unit tests

b839f2c

cmsbuild added this to the Next CMSSW_8_0_X milestone Jan 26, 2017

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jan 26, 2017

cmsbuild added tests-started and removed tests-pending labels Jan 26, 2017

cmsbuild added tests-approved and removed tests-started labels Jan 26, 2017

cmsbuild added comparison-available and removed comparison-pending labels Jan 26, 2017

slava77 reviewed Feb 10, 2017

View reviewed changes

cmsbuild added the tests-pending label Feb 24, 2017

cmsbuild added tests-started and removed tests-pending labels Feb 24, 2017

cmsbuild added tests-approved and removed tests-started labels Feb 24, 2017

cmsbuild added comparison-available and removed comparison-pending labels Feb 24, 2017

This was referenced Feb 28, 2017

CMSSW changes requested for spring re-reco of 2016 data #17603

Closed

Developments to provide facilitated access to new TauID information (all high level changes) #17624

Closed

cmsbuild added hold reconstruction-approved and removed reconstruction-pending labels Mar 1, 2017

davidlange6 merged commit f048a8e into cms-sw:CMSSW_8_0_X Apr 6, 2017

davidlange6 mentioned this pull request Apr 6, 2017

Revert "Backport of access to tauID and electron discrimination from miniAOD" #18234

Merged

slava77 mentioned this pull request Apr 10, 2017

Revert "Revert "Backport of access to tauID and electron discrimination from miniAOD"" #18235

Closed

roger-wolf deleted the CMSSW_8_0_X_tau-pog_miniAOD-backport branch November 17, 2017 10:01

roger-wolf restored the CMSSW_8_0_X_tau-pog_miniAOD-backport branch November 23, 2017 13:53

mbluj deleted the CMSSW_8_0_X_tau-pog_miniAOD-backport branch October 10, 2023 10:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport of access to tauID and electron discrimination from miniAOD #17292

Backport of access to tauID and electron discrimination from miniAOD #17292

roger-wolf commented Jan 26, 2017

cmsbuild commented Jan 26, 2017

slava77 commented Jan 26, 2017

cmsbuild commented Jan 26, 2017 •

edited

cmsbuild commented Jan 26, 2017

cmsbuild commented Jan 26, 2017

cmsbuild commented Jan 26, 2017

slava77 commented Feb 3, 2017

roger-wolf commented Feb 4, 2017

roger-wolf commented Feb 9, 2017

slava77 commented Feb 9, 2017

roger-wolf commented Feb 9, 2017 •

edited

slava77 Feb 10, 2017

roger-wolf Feb 23, 2017

slava77 Feb 10, 2017

roger-wolf Feb 23, 2017

slava77 Feb 10, 2017

roger-wolf Feb 23, 2017

cmsbuild commented Feb 24, 2017

slava77 commented Feb 24, 2017

cmsbuild commented Feb 24, 2017 •

edited

cmsbuild commented Feb 24, 2017

cmsbuild commented Feb 24, 2017

cmsbuild commented Feb 24, 2017

slava77 commented Mar 1, 2017

slava77 commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

slava77 commented Apr 6, 2017

davidlange6 commented Apr 6, 2017

slava77 commented Apr 6, 2017 via email

		@@ -0,0 +1,5 @@
		/store/relval/CMSSW_8_1_0_pre11/RelValQCD_Pt_80_120_13/GEN-SIM-RECO/81X_mcRun2_asymptotic_Candidate_2016_08_30_11_31_55-v1/00000/24F576D4-4D74-E611-8898-0025905A48BA.root

Backport of access to tauID and electron discrimination from miniAOD #17292

Backport of access to tauID and electron discrimination from miniAOD #17292

Conversation

roger-wolf commented Jan 26, 2017

cmsbuild commented Jan 26, 2017

slava77 commented Jan 26, 2017

cmsbuild commented Jan 26, 2017 • edited

cmsbuild commented Jan 26, 2017

cmsbuild commented Jan 26, 2017

cmsbuild commented Jan 26, 2017

slava77 commented Feb 3, 2017

roger-wolf commented Feb 4, 2017

roger-wolf commented Feb 9, 2017

slava77 commented Feb 9, 2017

roger-wolf commented Feb 9, 2017 • edited

slava77 Feb 10, 2017

Choose a reason for hiding this comment

roger-wolf Feb 23, 2017

Choose a reason for hiding this comment

slava77 Feb 10, 2017

Choose a reason for hiding this comment

roger-wolf Feb 23, 2017

Choose a reason for hiding this comment

slava77 Feb 10, 2017

Choose a reason for hiding this comment

roger-wolf Feb 23, 2017

Choose a reason for hiding this comment

cmsbuild commented Feb 24, 2017

slava77 commented Feb 24, 2017

cmsbuild commented Feb 24, 2017 • edited

cmsbuild commented Feb 24, 2017

cmsbuild commented Feb 24, 2017

cmsbuild commented Feb 24, 2017

slava77 commented Mar 1, 2017

slava77 commented Mar 1, 2017

cmsbuild commented Mar 1, 2017

slava77 commented Apr 6, 2017

davidlange6 commented Apr 6, 2017

slava77 commented Apr 6, 2017 via email

cmsbuild commented Jan 26, 2017 •

edited

roger-wolf commented Feb 9, 2017 •

edited

cmsbuild commented Feb 24, 2017 •

edited