Update of DeepTauID to ver. 2017v2

[mbluj]

PR description:

This PR updates DeepTauID to version 2017v2. The new version, thanks to revised structure of used DNN, fixes issues of version 2017v1 that is a size of training files and memory consumption are greatly reduced. Possibility to run previous version is maintained.

DeepTauID (2017v2) is added to the tau sequence in the standard MiniAOD workflow.

Performance with 1k ttbar events:

• size of three training files: 2.9+0.85+0.85=4.6MiB (was 21MiB in 2017v1);
• memory consumption: on top of DeepJet: 6.4MiB or 21.5 MiB standalone. The difference between the two comes mainly from executing pat::PackedCandidate::dzError() method which triggers unpacking of track covariance matrix and at smaller extend from initialization of TensorFlow. Full information on memory usage from IGProf can be accessed at this link: http://mdidomen.web.cern.ch/mdidomen/cgi-bin/igprof-navigator/changes_PR_126/DeepTau_memory_usage/;
• CPU timing:
  • Full timing of a simple test workflow is 0.11s/event with DeepTauId::getPredictions 0.088s/event; details at at http://mdidomen.web.cern.ch/mdidomen/cgi-bin/igprof-navigator/changes_PR_126/DeepTauCPU_TT;
  • Full timing of miniAOD workflow is 1.06s/event with DeepTauId::getPredictions 0.082s/event; details at http://mdidomen.web.cern.ch/mdidomen/cgi-bin/igprof-navigator/changes_PR_126/DeepTau_MiniAOD.

In addition the new v2 training overperforms (physics-result-wise) old v1 one as well as other TauIDs, details in slides:

• https://indico.cern.ch/event/816162/contributions/3408556/attachments/1835352/3007877/2019-04-29_DeepTau_ID_v2_developments.pdf
• https://indico.cern.ch/event/818643/contributions/3422014/attachments/1839253/3014630/2019-05-06_DeepTau_ID_v2_WP.pdf

Note 1: this PR is meant to be backported to 10_6_X for UL processing and to 10_2_X for using with current Run-2 samples.
Note 2: this PR is completed by PR to cms-data cms-data/RecoTauTag-TrainingFiles#3 which needs to be propagated to cms-dist.
Note 3: DeepTau 2017v2 is added to MiniAOD workflow with 11_0_X and 10_6_X (UL), while not in the backport to 10_2_X.

PR validation:

Validated with matrix tests; requires cms-data/RecoTauTag-TrainingFiles#3 for unit tests

if this PR is a backport please specify the original PR:

N/A

[mbluj]

Currently tauID in MiniAOD samples is routinely (as part of RelVal campaigns) validated with standalone (python+ROOT-based) Tau POG tools, i.e. tools not integrated with central DQM workflows. We plan to add DeepTauID to these tools which is rather trivial task.
There is also plan to add MiniAOD validation to DQM workflows which will for instance support validation of features added to CMSSW via unitTests.

[cmsbuild]

+1
Tested at: 77ac4c5
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4b3e66/784/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4b3e66/784/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 220 differences found in the comparisons
• DQMHistoTests: Total files compared: 33
• DQMHistoTests: Total histograms compared: 3210222
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3209887
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
• Checked 137 log files, 14 edm output root files, 33 DQM output files

[perrotta]

+1

• New ID provides impressive improvement in physics performance
• CPU time ~0.09s/evt (tested on TT sample), about 9% of the miniAOD processing time for that sample
  • Factor 3 reduction since the first version of this PR
  • It is probably still acceptable for miniAOD production, but a bit too much for routine nanoAOD production: some further reduction should be attempted if that ID will have to be recomputed in the nanoAOD campaigns: github issue Reduce cpu for DeepTauID 2017v2 #27119 has been opened to keep track of the future improvements in cpu performance
• Jenkins tests show the additional IDs in miniAOD outputs as only differences
• This has to be merged together with the external in Training file for DeepTauID 2017v2 cms-data/RecoTauTag-TrainingFiles#3

[fabiocos]

@civanch @santocch could you please check this PR?

[civanch]

+1

[santocch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1