Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hcalFraction definition in DeepTauID #27173

Conversation

mbluj
Copy link
Contributor

@mbluj mbluj commented Jun 11, 2019

PR description:

This PR fixes issue related to the fact that DeepTauID is trained with samples produced before pat::PackedCandidate::hcalFraction() was redefined in #26506 (for >=CMSSW_10_6_X).
This fix is similar to one for RecoBTag/FeatureTools/plugins/DeepBoostedJetTagInfoProducer.cc
from #26506.

This fixes differences wrt DeepTauID with 10_2_X mentioned in #27148 (comment)

PR validation:

As it is a small and well focused change only a simple test running DeepTauID was executed.

if this PR is a backport please specify the original PR:

N/A

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-27173/10324

  • This PR adds an extra 28KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mbluj for master.

It involves the following packages:

RecoTauTag/RecoTau

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 11, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/874/console Started: 2019/06/11 16:57

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a54410/874/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 94 differences found in the comparisons
  • DQMHistoTests: Total files compared: 33
  • DQMHistoTests: Total histograms compared: 3215458
  • DQMHistoTests: Total failures: 2
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3215122
  • DQMHistoTests: Total skipped: 334
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
  • Checked 137 log files, 14 edm output root files, 33 DQM output files

@perrotta
Copy link
Contributor

@mbluj , since you are fixing, could you please also include here (and in the backport) the fix suggested by the static analyzer in https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a54410/874/llvm-analysis/report-b3bc51.html#EndPath (that I evidently overlooked during previous review)?

@perrotta
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 12, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/899/console Started: 2019/06/12 12:06

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a54410/899/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 90 differences found in the comparisons
  • DQMHistoTests: Total files compared: 33
  • DQMHistoTests: Total histograms compared: 3237932
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3237597
  • DQMHistoTests: Total skipped: 334
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
  • Checked 137 log files, 14 edm output root files, 33 DQM output files

@perrotta
Copy link
Contributor

Please @mbluj confirm that the changes observed in the miniAOD outputs are the expected consequence of the fix, and that they now correspond identically to the outputs of the version of this code run in 10_2_X.

@mbluj
Copy link
Contributor Author

mbluj commented Jun 12, 2019

Please @mbluj confirm that the changes observed in the miniAOD outputs are the expected consequence of the fix, and that they now correspond identically to the outputs of the version of this code run in 10_2_X.

DQM should not show differences as tauIDs are not yet monitored at MiniAOD and only small changes in DeepTauIDs embedded into pat::Taus in slimmedTaus collection are expected. And indeed the only differences reported by DQM tests, as far as I can see, are small differences by TimerService.

It was verified with a standalone tool that the fix makes DeepTauIDs identical between 106X/110X and 102X.

@perrotta
Copy link
Contributor

Please @mbluj confirm that the changes observed in the miniAOD outputs are the expected consequence of the fix, and that they now correspond identically to the outputs of the version of this code run in 10_2_X.

DQM should not show differences as tauIDs are not yet monitored at MiniAOD and only small changes in DeepTauIDs embedded into pat::Taus in slimmedTaus collection are expected. And indeed the only differences reported by DQM tests, as far as I can see, are small differences by TimerService.

Well, the miniAOD event output comparison show differences, see https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_0_X_2019-06-11-2300+a54410/32208/validateJR.html, and this must be expected given the fix

It was verified with a standalone tool that the fix makes DeepTauIDs identical between 106X/110X and 102X.

Great! That was needed just to confirm it.

@mbluj
Copy link
Contributor Author

mbluj commented Jun 12, 2019

Please @mbluj confirm that the changes observed in the miniAOD outputs are the expected consequence of the fix, and that they now correspond identically to the outputs of the version of this code run in 10_2_X.

DQM should not show differences as tauIDs are not yet monitored at MiniAOD and only small changes in DeepTauIDs embedded into pat::Taus in slimmedTaus collection are expected. And indeed the only differences reported by DQM tests, as far as I can see, are small differences by TimerService.

Well, the miniAOD event output comparison show differences, see https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_0_X_2019-06-11-2300+a54410/32208/validateJR.html, and this must be expected given the fix

It was verified with a standalone tool that the fix makes DeepTauIDs identical between 106X/110X and 102X.

Great! That was needed just to confirm it.

Sorry, I did not found these plots. Yes, they generally look as one can expect for comparisons with very low number of events. And as already said it was verified with bigger stats using standalone tools that changes are as expected.

@perrotta
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@fabiocos
Copy link
Contributor

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants