clean up of old tau discriminators and integration of new ones #13272

roger-wolf · 2016-02-13T11:14:24Z

The pull request is build upon CMSSW_8_0_X_2016-02-09-1100. The modifications that have been made is limited to five files in

RecoTauTag/Configuration (definition of even content)
RecoTauTag/RecoTau (main sequence)
Validation/RecoTau (validation configuration)
PhysicsTools/PatAlgos (high level analysis configuration)
We've checked that the pat configuration runs in unscheduled mode with all RecoTauTagRECO in the outputCommands, i.e. all discriminators have been build and made persostent starting from relval RECO. The limited set of matrix tests (runTheMatrix.py -l limited -i all) passed w/o failures. The usual validation plot that we checked show unsuspicious behaviour (checked by TauPOg ynd ID conveners). We would like this request still to go into CMSSW_8_0_0_pre6 to be built from Monday on if possible.
Thanx a lot!

…from branch 80xReMiniAOD-RemoveOldDisc to here: the following three files should be more or less clear.

…from branch 80xReMiniAOD-RemoveOldDisc to here: in the following two files I commented several lines which where present in the latest IB of 02.09.2016 -- 11:00 but not in the indicated developments branch. Please cross check that these lines can be deleted.

…ill at least slope through a solitary instance of creating tau discriminators from RECO on. I'd like to use this as a lightweighted test that things are OK w/o running this full matrix script all the time. Within the next days this might be extended and potentially moved into some RecoTau place.

…ith conveners during the day

…uring the day

…een light from convenres during the day). PileupWeighted discriminatirs remain in commented for the moment as they might remain of interest for validation purposes.

…ll sequences are run in unscheduled

…tContent to satisfy matrx test 1000.0 (see comment to corresponding lines in RecoTauTag_EventContent_cff)

…ontinued. Note a renaming: MVA6raw --> MVA6Raw

cmsbuild · 2016-02-13T11:14:44Z

A new Pull Request was created by @roger-wolf (Roger Wolf) for CMSSW_8_0_X.

It involves the following packages:

PhysicsTools/PatAlgos
RecoTauTag/Configuration
Validation/RecoTau

@cvuosalo, @monttj, @cmsbuild, @deguio, @slava77, @vadler, @vanbesien, @davidlange6 can you please review it and eventually sign? Thanks.
@TaiSakuma, @imarches, @ahinzmann, @acaudron, @mmarionncern, @rappoccio, @jdolen, @nhanvtran, @schoef, @ferencek, @gpetruc, @mariadalfonso, @pvmulder this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

slava77 · 2016-02-13T17:41:54Z

PhysicsTools/PatAlgos/python/producersLayer1/tauProducer_cfi.py

-        byLoosePileupWeightedIsolation3Hits = cms.InputTag("hpsPFTauDiscriminationByLoosePileupWeightedIsolation3Hits"),
-        byMediumPileupWeightedIsolation3Hits = cms.InputTag("hpsPFTauDiscriminationByMediumPileupWeightedIsolation3Hits"),
-        byTightPileupWeightedIsolation3Hits = cms.InputTag("hpsPFTauDiscriminationByTightPileupWeightedIsolation3Hits"),
+        #byLoosePileupWeightedIsolation3Hits = cms.InputTag("hpsPFTauDiscriminationByLoosePileupWeightedIsolation3Hits"),


why are these commented out and not completely removed?

agreed. I've removed the commented lines. For explanation: there was a request by TauID conveners to keep these discriminators accessible for validation purposes. But validation is not run on PAT therefore your argument applies.

slava77 · 2016-02-13T17:47:39Z

@cmsbuild please test

cmsbuild · 2016-02-13T17:47:58Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11195/console

cmsbuild · 2016-02-13T19:20:41Z

-1

Tested at: 231e24b
I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13272/11195/summary.html

cmsbuild · 2016-02-13T20:01:10Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13272/11195/summary.html

cmsbuild · 2016-02-15T16:06:47Z

Pull request #13272 was updated. @cvuosalo, @monttj, @cmsbuild, @deguio, @slava77, @vadler, @vanbesien, @davidlange6 can you please check and sign again.

roger-wolf · 2016-02-15T16:14:41Z

I've updated the relval files for PAT now from 760 to 800pre6, which should fix this problem. I did not touch the reval section for filesRelValTTbarPileUpFastSimGENSIMDIGIRECO, which contains a comment I can not judge upon and not the SingleMuon Relval section either. Since AODSIM relval files do not exist (anymore?) I'm loading GEN-SIM-RECO instead also for the AODSIM string. On the way I've commited another minor fix.

slava77 · 2016-02-15T16:35:21Z

@hengne
are the AODSIM relval outputs expected to come soon or were they not included in the 800pre6?

cvuosalo · 2016-02-15T19:43:28Z

@cmsbuild please test

cmsbuild · 2016-02-15T19:43:48Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11237/console

cmsbuild · 2016-02-15T21:18:26Z

+1
Tested at: 8bf3eef
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13272/11237/summary.html

cmsbuild · 2016-02-15T21:59:19Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13272/11237/summary.html

cvuosalo · 2016-02-16T17:07:18Z

Jenkins results show a decrease in tau L1 global efficiencies. Is this change expected? Here's an example from workflow 25.0_TTbar.

cvuosalo · 2016-02-16T17:58:51Z

Here are more examples of changes from a test of workflow 25202.0_TTbar_13 with 70 events against baseline CMSSW_8_0_X_2016-02-09-1100.

Here are more efficiency drops:

And here is one example among several of newly filled-in plots for tau offline HLT:

cvuosalo · 2016-02-16T19:25:06Z

As might be expected, an extended test shows the deletion of the obsolete discriminators shrinks Reco event content size and reduces CPU time by a few percent. Details to follow once the questions above are resolved.

roger-wolf · 2016-02-17T13:18:43Z

I've cross checked with the responsible persons for tau DQM and the tau POG and sub-group conveners. This looks like a plotting issue in the corresponding DQM package (https://github.com/cms-tau-pog/cmssw/tree/CMSSW_8_0_X_tau-pog/HLTriggerOffline/Tau/python/Validation) that currently is under investigation. Conveners are aware of the situation and propose to proceed. We'll keep the change in the validation plot in mind and add a fix asa available possibly at some later stage, if this is OK with you.

cvuosalo · 2016-02-17T20:23:04Z

+1

For #13272 8bf3eef

Removing obsolete tau discriminators and integrating a few new ones.

The code changes are satisfactory. Jenkins tests against baseline CMSSW_8_0_X_2016-02-15-1100 show some minor differences discussed above, which will be investigated and fixed, if necessary, in a subsequent PR. An extended test of workflow 25202.0_TTbar_13 with 70 events against baseline CMSSW_8_0_X_2016-02-09-1100 shows similar differences. Reco event content size decreases by 0.2% from the loss of 58 deleted discriminators, which is partly offset by 3 newly added discriminators. Mini-AOD increases in size by 0.6% due to more tau content. CPU time decreases by about 2.3%, partly due to the eliminated discriminators. Memory usage shrinks marginally.
Here are the details about the Mini-AOD size increase:

-----------------------------------------------------------------
   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------

   1935.2 ->      2132.2        197      9.7   0.23     patTaus_slimmedTausBoosted__RECO.
   2215.7 ->      2516.9        301     12.7   0.36     patTaus_slimmedTaus__RECO.
-------------------------------------------------------------
    84181 ->       84679        498             0.6     ALL BRANCHES

deguio · 2016-02-18T08:28:14Z

+1

clean up of old tau discriminators and integration of new ones

roger-wolf added 11 commits February 10, 2016 16:03

adaptations to switch form old to new tau doscriminators -- migrated …

ef3c69c

…from branch 80xReMiniAOD-RemoveOldDisc to here: the following three files should be more or less clear.

clean up of unneccesary module definitions after email consultation w…

7570544

…ith conveners during the day

cleanup of the Eventcontent after email consultation with conveners d…

f840e30

…uring the day

deleting lines which have been commented in previous commit (after gr…

f0b0810

…een light from convenres during the day). PileupWeighted discriminatirs remain in commented for the moment as they might remain of interest for validation purposes.

adding full TauPOG RECO EventContent to outputCommands to make sure a…

09cd459

…ll sequences are run in unscheduled

needed to re-add ByLooseIsolation and ByLooseChargedIsolation to Even…

accea22

…tContent to satisfy matrx test 1000.0 (see comment to corresponding lines in RecoTauTag_EventContent_cff)

first trial of giving the file a better readable structure -- to be c…

f674cbb

…ontinued. Note a renaming: MVA6raw --> MVA6Raw

MVA6raw --> MVA6Raw

ced52be

referting changes in PhysicsTools packages

231e24b

cmsbuild added this to the Next CMSSW_8_0_X milestone Feb 13, 2016

cmsbuild added reconstruction-pending dqm-pending analysis-pending pending-signatures tests-pending orp-pending comparison-pending labels Feb 13, 2016

slava77 reviewed Feb 13, 2016
View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Feb 13, 2016

cmsbuild added tests-rejected and removed tests-started labels Feb 13, 2016

cmsbuild added tests-pending comparison-pending and removed comparison-available tests-rejected labels Feb 15, 2016

cmsbuild added tests-started and removed tests-pending labels Feb 15, 2016

cmsbuild added tests-approved and removed tests-started labels Feb 15, 2016

cmsbuild added comparison-available and removed comparison-pending labels Feb 15, 2016

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Feb 17, 2016

cmsbuild added dqm-approved and removed dqm-pending labels Feb 18, 2016

davidlange6 added a commit that referenced this pull request Feb 18, 2016

Merge pull request #13272 from cms-tau-pog/CMSSW_8_0_X_tau-pog

4f9f6ec

clean up of old tau discriminators and integration of new ones

davidlange6 merged commit 4f9f6ec into cms-sw:CMSSW_8_0_X Feb 18, 2016

mbluj mentioned this pull request Feb 25, 2016

Hlt tau ref producer update #13455

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

clean up of old tau discriminators and integration of new ones #13272

clean up of old tau discriminators and integration of new ones #13272

roger-wolf commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

slava77 Feb 13, 2016

roger-wolf Feb 13, 2016

slava77 commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

cmsbuild commented Feb 15, 2016

roger-wolf commented Feb 15, 2016

slava77 commented Feb 15, 2016

cvuosalo commented Feb 15, 2016

cmsbuild commented Feb 15, 2016

cmsbuild commented Feb 15, 2016

cmsbuild commented Feb 15, 2016

cvuosalo commented Feb 16, 2016

cvuosalo commented Feb 16, 2016

cvuosalo commented Feb 16, 2016

roger-wolf commented Feb 17, 2016

cvuosalo commented Feb 17, 2016

deguio commented Feb 18, 2016

clean up of old tau discriminators and integration of new ones #13272

clean up of old tau discriminators and integration of new ones #13272

Conversation

roger-wolf commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

slava77 Feb 13, 2016

Choose a reason for hiding this comment

roger-wolf Feb 13, 2016

Choose a reason for hiding this comment

slava77 commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

cmsbuild commented Feb 13, 2016

cmsbuild commented Feb 15, 2016

roger-wolf commented Feb 15, 2016

slava77 commented Feb 15, 2016

cvuosalo commented Feb 15, 2016

cmsbuild commented Feb 15, 2016

cmsbuild commented Feb 15, 2016

cmsbuild commented Feb 15, 2016

cvuosalo commented Feb 16, 2016

cvuosalo commented Feb 16, 2016

cvuosalo commented Feb 16, 2016

roger-wolf commented Feb 17, 2016

cvuosalo commented Feb 17, 2016

deguio commented Feb 18, 2016