Update miniAOD AK8 userFloat content. Add PUPPI substructure. Remove CMSTT. #13793

jdolen · 2016-03-21T18:26:25Z

We have the following proposal for the AK8chs jet MiniAOD userfloat content in 80X/81X:

Remove unused run 1 taggers (CMS Top tagger)
Add PUPPI based taggers (PUPPI soft drop + PUPPI Nsubjettiness ) for matched AK8 PUPPI jets
In order to save space, add only PUPPI subjets and userfloats for matched PUPPI jet 4-vector and substructure variables.

Size/timing estimates are given in the slides linked here.
https://indico.cern.ch/event/510570/contribution/7/attachments/1246782/1836484/160321_-_Jet_substructure_MiniAOD_content.pptx.pdf

Cheers,
@rappoccio @ahinzmann @jdolen

cmsbuild · 2016-03-21T18:27:30Z

A new Pull Request was created by @jdolen (James Dolen) for CMSSW_8_1_X.

It involves the following packages:

CommonTools/RecoAlgos
PhysicsTools/PatAlgos
RecoJets/Configuration
RecoJets/JetProducers

@cvuosalo, @monttj, @cmsbuild, @slava77, @vadler, @davidlange6 can you please review it and eventually sign? Thanks.
@rappoccio, @imarches, @makortel, @acaudron, @abbiendi, @mmarionncern, @jhgoh, @ahinzmann, @nhanvtran, @yslai, @schoef, @ferencek, @gpetruc, @mariadalfonso, @pvmulder, @TaiSakuma this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

slava77 · 2016-03-21T19:34:18Z

PhysicsTools/PatAlgos/python/slimming/MicroEventContent_cff.py

        #'keep *_slimmedJetsAK8PFCHSSoftDropSubjets_*_*',
        #'keep *_slimmedJetsCMSTopTagCHSSubjets_*_*',
        'keep *_slimmedJetsAK8PFCHSSoftDropPacked_SubJets_*',
-        'keep *_slimmedJetsCMSTopTagCHSPacked_SubJets_*',
+        'keep *_slimmedJetsAK8PFPuppiSoftDropPacked_SubJets_*',
+        #'keep *_slimmedJetsCMSTopTagCHSPacked_SubJets_*',


please drop commented out lines,
unless there is a good reason to keep.

Also remove 'keep *_caTopTagInfosPAT_*_*' in line 65.

slava77 · 2016-03-21T19:36:33Z

@cmsbuild please test

cmsbuild · 2016-03-21T19:40:16Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/11995/console

cmsbuild · 2016-03-21T21:11:46Z

+1
Tested at: 8574201
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13793/11995/summary.html

cmsbuild · 2016-03-21T22:07:33Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13793/11995/summary.html

ferencek · 2016-03-22T09:59:48Z

PhysicsTools/PatAlgos/python/slimming/applySubstructure_cff.py

-
-    # Add AK8 top tagging variables
-    process.patJetsAK8.tagInfoSources = cms.VInputTag(cms.InputTag("caTopTagInfosPAT"))
+    process.patJetsAK8.userData.userFloats.src += ['ak8PFJetsCHSPrunedMass','ak8PFJetsCHSSoftDropMass']  #,'ak8PFJetsCHSTrimmedMass','ak8PFJetsCHSFilteredMass']
    process.patJetsAK8.addTagInfos = cms.bool(True)


process.patJetsAK8.addTagInfos can now be set to False or, since that is the default value for this switch, this line can be removed.

cvuosalo · 2016-03-23T22:08:03Z

An extended test of workflow 25202.0_TTbar_13 with 70 events and no DQM/Validation step against baseline CMSSW_8_1_X_2016-03-14-1100 shows the following time and memory usage. TopTagCHS modules are removed and replaced by AK8PFPuppi modules. The removed modules had a total time of about 50 ms/evt, while the new modules together only take about 40 ms/evt. Overall, there is no significant change in CPU time for RECO.

Baseline
Max VSIZ 2280.05 on evt 36 ; max RSS 1826.91 on evt 36
Time av 15.1361 s/evt   max 48.7359 s on evt 1
M1 Time av 14.6492 s/evt   max 27.0861 s on evt 36
M8 Time av 14.8417 s/evt   max 27.0861 s on evt 36

PR 13793
Summary for 70 events
Max VSIZ 2276.84 on evt 36 ; max RSS 1765.72 on evt 61
Time av 15.123 s/evt   max 50.7816 s on evt 1
M1 Time av 14.6063 s/evt   max 27.0756 s on evt 36
M8 Time av 14.7225 s/evt   max 27.0756 s on evt 36

  delta/mean delta/orJob     original                   new       module name
  ---------- ------------     --------                  ----       ------------
     removed      -0.00%         0.07 ms/ev ->         0.00 ms/ev patJetPartonMatchCMSTopTagCHS
     removed      -0.01%         1.77 ms/ev ->         0.00 ms/ev pfSecondaryVertexTagInfosCMSTopTagCHSSubjets
     removed      -0.00%         0.08 ms/ev ->         0.00 ms/ev patJetCorrFactorsCMSTopTagCHS
     removed      -0.00%         0.39 ms/ev ->         0.00 ms/ev patJetsCMSTopTagCHS
     removed      -0.00%         0.04 ms/ev ->         0.00 ms/ev caTopTagInfos
     removed      -0.00%         0.03 ms/ev ->         0.00 ms/ev patJetGenJetMatchCMSTopTagCHS
     removed      -0.01%         1.24 ms/ev ->         0.00 ms/ev patJetFlavourAssociationCMSTopTagCHSSubjets
     removed      -0.01%         1.02 ms/ev ->         0.00 ms/ev pfInclusiveSecondaryVertexFinderTagInfosCMSTopTagCHSSubjets
     removed      -0.00%         0.03 ms/ev ->         0.00 ms/ev patJetGenJetMatchCMSTopTagCHSSubjets
     removed      -0.00%         0.07 ms/ev ->         0.00 ms/ev ak8PFJetsCHSTrimmedMass
     removed      -0.01%         1.43 ms/ev ->         0.00 ms/ev patJetsCMSTopTagCHSSubjets
     removed      -0.00%         0.04 ms/ev ->         0.00 ms/ev ak8PFJetsCHSFilteredMass
     removed      -0.00%         0.06 ms/ev ->         0.00 ms/ev selectedPatJetsCMSTopTagCHS
     removed      -0.00%         0.04 ms/ev ->         0.00 ms/ev caTopTagInfosPAT
     removed      -0.00%         0.11 ms/ev ->         0.00 ms/ev selectedPatJetsCMSTopTagCHSSubjets
     removed      -0.00%         0.23 ms/ev ->         0.00 ms/ev pfCombinedSecondaryVertexV2BJetTagsCMSTopTagCHSSubjets
     removed      -0.00%         0.08 ms/ev ->         0.00 ms/ev slimmedJetsCMSTopTagCHSSubjets
     removed      -0.00%         0.13 ms/ev ->         0.00 ms/ev patJetPartonMatchCMSTopTagCHSSubjets
     removed      -0.01%         1.72 ms/ev ->         0.00 ms/ev pfImpactParameterTagInfosCMSTopTagCHSSubjets
     removed      -0.00%         0.04 ms/ev ->         0.00 ms/ev slimmedJetsCMSTopTagCHSPacked
     removed      -0.04%         4.98 ms/ev ->         0.00 ms/ev ak8PFJetsCHSFiltered
     removed      -0.00%         0.13 ms/ev ->         0.00 ms/ev patJetCorrFactorsCMSTopTagCHSSubjets
     removed      -0.29%        41.15 ms/ev ->         0.00 ms/ev ak8PFJetsCHSTrimmed
     removed      -0.00%         0.23 ms/ev ->         0.00 ms/ev pfCombinedInclusiveSecondaryVertexV2BJetTagsCMSTopTagCHSSubjets
      added      +0.00%         0.00 ms/ev ->         0.03 ms/ev patJetGenJetMatchAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.06 ms/ev patJetPartonMatchAK8PFPuppiSoftDrop
      added      +0.00%         0.00 ms/ev ->         0.16 ms/ev pfCombinedInclusiveSecondaryVertexV2BJetTagsAK8PFPuppiSoftDropSubjets
      added      +0.01%         0.00 ms/ev ->         1.31 ms/ev patJetsAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.32 ms/ev patJetPartonMatchAK8Puppi
      added      +0.00%         0.00 ms/ev ->         0.38 ms/ev patJetsAK8PFPuppiSoftDrop
      added      +0.00%         0.00 ms/ev ->         0.07 ms/ev selectedPatJetsAK8PFPuppiSoftDrop
      added      +0.00%         0.00 ms/ev ->         0.16 ms/ev pfCombinedSecondaryVertexV2BJetTagsAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.05 ms/ev ak8PFJetsPuppiSoftDropMass
      added      +0.00%         0.00 ms/ev ->         0.08 ms/ev patJetCorrFactorsAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.08 ms/ev patJetPartonMatchAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.07 ms/ev patJetCorrFactorsAK8PFPuppiSoftDrop
      added      +0.19%         0.00 ms/ev ->        27.16 ms/ev ak8PFJetsPuppi
      added      +0.00%         0.00 ms/ev ->         0.09 ms/ev selectedPatJetsAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.59 ms/ev patJetFlavourAssociationAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.04 ms/ev slimmedJetsAK8PFPuppiSoftDropPacked
      added      +0.00%         0.00 ms/ev ->         0.42 ms/ev pfInclusiveSecondaryVertexFinderTagInfosAK8PFPuppiSoftDropSubjets
      added      +0.06%         0.00 ms/ev ->         7.90 ms/ev NjettinessAK8Puppi
      added      +0.00%         0.00 ms/ev ->         0.03 ms/ev patJetGenJetMatchAK8PFPuppiSoftDrop
      added      +0.00%         0.00 ms/ev ->         0.04 ms/ev patJetGenJetMatchAK8Puppi
      added      +0.00%         0.00 ms/ev ->         0.05 ms/ev slimmedJetsAK8PFPuppiSoftDropSubjets
      added      +0.00%         0.00 ms/ev ->         0.32 ms/ev patJetCorrFactorsAK8Puppi
      added      +0.00%         0.00 ms/ev ->         0.19 ms/ev ak8PFJetsPuppiValueMap
      added      +0.01%         0.00 ms/ev ->         1.07 ms/ev pfImpactParameterTagInfosAK8PFPuppiSoftDropSubjets
      added      +0.01%         0.00 ms/ev ->         1.10 ms/ev pfSecondaryVertexTagInfosAK8PFPuppiSoftDropSubjets
      added      +0.01%         0.00 ms/ev ->         1.20 ms/ev patJetsAK8Puppi
      added      +0.00%         0.00 ms/ev ->         0.67 ms/ev ak8PFJetsPuppiSoftDrop
      added      +0.00%         0.00 ms/ev ->         0.56 ms/ev patJetFlavourAssociationAK8Puppi

cvuosalo · 2016-03-23T22:28:08Z

@jdolen : Reco is ready to approve this PR when the little code fixes requested by @slava77 and @ferencek are completed.

cmsbuild · 2016-03-24T15:47:10Z

Pull request #13793 was updated. @cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please check and sign again.

cmsbuild · 2016-03-24T15:51:59Z

Pull request #13793 was updated. @cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please check and sign again.

cmsbuild · 2016-03-24T15:57:48Z

Pull request #13793 was updated. @cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please check and sign again.

cmsbuild · 2016-03-24T16:08:15Z

Pull request #13793 was updated. @cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please check and sign again.

cvuosalo · 2016-03-24T22:21:02Z

@cmsbuild please test

cmsbuild · 2016-03-24T22:21:32Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12062/console

cmsbuild · 2016-03-24T23:04:15Z

-1

Tested at: 8201e22
I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-13793/12062/summary.html

cvuosalo · 2016-03-25T20:59:41Z

Additional, possibly more accurate, test results running only the PAT step with 1000 events to produce Mini-AOD were obtained with this command:

cmsDriver.py miniAOD-prod -s PAT --eventcontent MINIAODSIM --runUnscheduled --mc --filein /store/relval/CMSSW_8_1_0_pre1/RelValTTbar_13/GEN-SIM-RECO/PU25ns_80X_mcRun2_asymptotic_v6-v1/10000/407BD44A-DCE8-E511-BF44-0025905A60B6.root --conditions 80X_mcRun2_asymptotic_v6 -n 1000 --customise Validation/Performance/TimeMemoryInfo.py

Wall-clock time and memory usage are shown, along with a second timing measure that excludes the first event. Only the largest time changes are listed (see above for full listing of added and removed modules).

Baseline
Max VSIZ 1736.16 on evt 910 ; max RSS 1267.02 on evt 993
Time av 5.51691 s/evt   max 112.822 s on evt 778
M1 Time av 5.48936 s/evt   max 112.822 s on evt 778
M8 Time av 5.51286 s/evt   max 112.822 s on evt 778

PR 13793
Max VSIZ 1704.7 on evt 401 ; max RSS 1273.86 on evt 666
Time av 5.47937 s/evt   max 115.781 s on evt 778
M1 Time av 5.4533 s/evt   max 115.781 s on evt 778
M8 Time av 5.47663 s/evt   max 115.781 s on evt 778

  delta/mean delta/orJob     original                   new       module name
  ---------- ------------     --------                  ----       ------------
     removed      -0.07%         3.35 ms/ev ->         0.00 ms/ev pfSecondaryVertexTagInfosCMSTopTagCHSSubjets
     removed      -0.07%         3.39 ms/ev ->         0.00 ms/ev patJetFlavourAssociationCMSTopTagCHSSubjets
     removed      -0.07%         3.48 ms/ev ->         0.00 ms/ev pfInclusiveSecondaryVertexFinderTagInfosCMSTopTagCHSSubjets
     removed      -0.17%         8.75 ms/ev ->         0.00 ms/ev patJetsCMSTopTagCHSSubjets
     removed      -0.07%         3.70 ms/ev ->         0.00 ms/ev pfImpactParameterTagInfosCMSTopTagCHSSubjets
     removed      -0.32%        16.43 ms/ev ->         0.00 ms/ev ak8PFJetsCHSFiltered
     removed      -2.00%       101.57 ms/ev ->         0.00 ms/ev ak8PFJetsCHSTrimmed
       added      +0.08%         0.00 ms/ev ->         3.89 ms/ev patJetsAK8PFPuppiSoftDropSubjets
       added      +1.02%         0.00 ms/ev ->        51.86 ms/ev ak8PFJetsPuppi
       added      +0.44%         0.00 ms/ev ->        22.64 ms/ev NjettinessAK8Puppi
       added      +0.06%         0.00 ms/ev ->         3.14 ms/ev pfSecondaryVertexTagInfosAK8PFPuppiSoftDropSubjets
       added      +0.08%         0.00 ms/ev ->         4.29 ms/ev patJetsAK8Puppi
   +0.173479      +0.42%       111.30 ms/ev ->       132.44 ms/ev pfCandidateToVertexAssociationPuppi
   -0.134996      -0.80%       320.81 ms/ev ->       280.24 ms/ev puppi
  ---------- ------------     --------                  ----       ------------
Job total:  5.09096 s/ev ==> 5.03398 s/ev

Change in the size of Mini-AOD:

   or, B         new, B      delta, B   delta, %   deltaJ, %    branch 
-----------------------------------------------------------------
    307.6 ->       317.7         10      3.2   0.02     patJets_slimmedJetsAK8__PAT.
     60.6 ->         0.0        -61     OLDO  -0.13     recoCATopJetTagInfos_caTopTagInfosPAT__PAT.
    293.3 ->         0.0       -293     OLDO  -0.63     patJets_slimmedJetsCMSTopTagCHSPacked_SubJets_PAT.
      0.0 ->       302.1        302     NEWO   0.65     patJets_slimmedJetsAK8PFPuppiSoftDropPacked_SubJets_PAT.
-------------------------------------------------------------
    46279 ->       46237        -42            -0.1     ALL BRANCHES

cvuosalo · 2016-03-25T21:17:00Z

+1

For #13793 8201e22

Removing obsolete and unused CMS Top taggers and adding PUPPI-based taggers to Mini-AOD.

#13796 is the 80X version of this PR, and it has already been approved.

The code changes are satisfactory. Jenkins tests for this PR encountered errors in the IB, but Jenkins tests for #13796 show no significant differences, except that DQM plots for the removed taggers are now empty. Extended tests are discussed above in detail, and they show no problems.

rappoccio and others added 3 commits March 17, 2016 14:47

Removing CMS top tagger from miniAOD, replacing with softdrop + puppi

28b22be

Now adding ungroomed PUPPI jets via value maps

3690618

Remove trimming and filtering. Pt cut 170

8574201

cmsbuild added this to the Next CMSSW_8_1_X milestone Mar 21, 2016

cmsbuild added reconstruction-pending analysis-pending pending-signatures tests-pending orp-pending comparison-pending labels Mar 21, 2016

slava77 reviewed Mar 21, 2016
View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Mar 21, 2016

cmsbuild added tests-approved and removed tests-started labels Mar 21, 2016

cmsbuild added comparison-available and removed comparison-pending labels Mar 21, 2016

jdolen mentioned this pull request Mar 21, 2016

Update miniAOD AK8 userFloat content. Add PUPPI substructure. Remove CMSTT. (80X) #13796

Merged

ferencek reviewed Mar 22, 2016
View reviewed changes

Removing commented out code from applySubstructure

9c61f2e

cmsbuild removed comparison-available tests-approved labels Mar 24, 2016

cmsbuild added tests-pending comparison-pending labels Mar 24, 2016

Removing comments and caTopTagInfosPAT

a5c596c

Changing addTagInfos to False

1812ae7

Setting addTagInfos to False for AK8Puppi

8201e22

cmsbuild added tests-started and removed tests-pending labels Mar 24, 2016

cmsbuild added tests-rejected and removed tests-started labels Mar 24, 2016

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 25, 2016

davidlange6 merged commit cae5e3b into cms-sw:CMSSW_8_1_X Mar 28, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update miniAOD AK8 userFloat content. Add PUPPI substructure. Remove CMSTT. #13793

Update miniAOD AK8 userFloat content. Add PUPPI substructure. Remove CMSTT. #13793

jdolen commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

slava77 Mar 21, 2016

ferencek Mar 22, 2016

slava77 commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

ferencek Mar 22, 2016

cvuosalo commented Mar 23, 2016

cvuosalo commented Mar 23, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cvuosalo commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cvuosalo commented Mar 25, 2016

cvuosalo commented Mar 25, 2016

Update miniAOD AK8 userFloat content. Add PUPPI substructure. Remove CMSTT. #13793

Update miniAOD AK8 userFloat content. Add PUPPI substructure. Remove CMSTT. #13793

Conversation

jdolen commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

slava77 Mar 21, 2016

Choose a reason for hiding this comment

ferencek Mar 22, 2016

Choose a reason for hiding this comment

slava77 commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

cmsbuild commented Mar 21, 2016

ferencek Mar 22, 2016

Choose a reason for hiding this comment

cvuosalo commented Mar 23, 2016

cvuosalo commented Mar 23, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cvuosalo commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cmsbuild commented Mar 24, 2016

cvuosalo commented Mar 25, 2016

cvuosalo commented Mar 25, 2016