Disabled PCA inputs for deepTau v2

[swozniewski]

PR description:

Problems have been detected with dxy_PCA input variables to DeepTauID which are inconsistent between data and MC. This PR sets input values to 0 and disables the variables this way.
Issue and fix were discussed in https://indico.cern.ch/event/842796/contributions/3541592/attachments/1897386/3130770/2019-08-26_DeepTau_ID_2017v2_DY_MC_vs_Embedded_and_bug_in_definition_of_PCA_variables.pdf (see slide 7 in particular)

Backport to 10_2_X for legacy analyses intended.

PR validation:

compiles

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27878/11650

• This PR adds an extra 28KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27878/11650/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27878/11650/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27878/11652

• This PR adds an extra 32KB to repository

[cmsbuild]

A new Pull Request was created by @swozniewski for master.

It involves the following packages:

RecoTauTag/RecoTau

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[perrotta]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2208/console Started: 2019/08/27 11:46

[perrotta]

@swozniewski @mbluj, in view of the possible discussion at the ORP, could you please let us know which are the plans for this PR and its backports? I.e. are they planned for a given re-miniAOD campaign, or which is the urgency for having them ready for analyses?

Moreover, do you have updated plots as in page 2 of https://indico.cern.ch/event/842796/contributions/3541592/attachments/1897386/3130770/2019-08-26_DeepTau_ID_2017v2_DY_MC_vs_Embedded_and_bug_in_definition_of_PCA_variables.pdf made with the updated id as in this PR?

[peruzzim]

Let me comment here, yes, we will want this backported to 10_2 and 10_6 because it's the version that should replace v2 for Run2 data analysis. And yes, it will be included in a future MINIAOD run. We can discuss at the ORP how to handle this (I guess with the usual run2_miniAOD_devel), probably we will want to switch it on before UL18 starts.

[cmsbuild]

+1
Tested at: 93068c1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-87bf54/2304/summary.html

[cmsbuild]

Comparison job queued.

[mbluj]

@swozniewski @mbluj, in view of the possible discussion at the ORP, could you please let us know which are the plans for this PR and its backports? I.e. are they planned for a given re-miniAOD campaign, or which is the urgency for having them ready for analyses?

Backports of this PR are already prepared:

• Disabled PCA inputs for deepTau v2 (10_6_X) #27882 for 106X for UL reMiniAODv2 (under run2_miniAOD_devel era as mentioned in features for UL re-miniAOD in 10_6_X #27889),
• Disabled PCA inputs for deepTau v2 (backport to 10_2_X) #27879 for 102X for legacy analyses (not added to official workflows) with corresponding PR to nanoAOD repository Update DeepTauID to ver. 2017v2p1 cms-nanoAOD/cmssw#396. The latter development is base of private recipe of Tau POG.

Moreover, do you have updated plots as in page 2 of https://indico.cern.ch/event/842796/contributions/3541592/attachments/1897386/3130770/2019-08-26_DeepTau_ID_2017v2_DY_MC_vs_Embedded_and_bug_in_definition_of_PCA_variables.pdf made with the updated id as in this PR?

To redo those plots a number of samples has to be processed. Such processing with 102X is well advanced, plots can be expected at the end of this week or beginning of next one.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-87bf54/2304/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 110 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2955688
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2955346
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.047 KiB( 33 files compared)
• DQMHistoSizes: changed ( 1325.7 ): 0.047 KiB Physics/NanoAODDQM
• Checked 145 log files, 15 edm output root files, 34 DQM output files

[perrotta]

+1

• From the point of view of the code this PR implements what announced in the PR description: the new deep tau id version v2p1 disables (i.e. sets to 0) the PCA inputs.
• v2p1 replaces the previous v2 everywhere, and the change is propagated wherever needed for mini and nano AOD productions and DQM in cmssw
• Jenkins tests show differences in the tau id in miniAOD, as expected
• Some global comparison data/MC is announced to arrive in short, but the results shown in the presentation linked from the PR description suggest that by disabling the PCA inputs the ids move in the right direction

[peruzzim]

+1

[santocch]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[fabiocos]

+1

[mbluj]

For records: a set of control plots obtained with the fixed DeepTau 2017v2p1, i.e. with PCA inputs disabled, can be found in https://indico.cern.ch/event/847915/contributions/3565600/attachments/1908653/3153065/TauPOG_DeepTauIDv2p1.pdf (for instance see sl. 4-5 and 12-14). The fix improves data/MC agreement as expected for whole Run-2, in particular ~30% disagreement for 2017 is solved.