Further preparations of deep tau ID for hlt phase2 (backport to CMSSW_11_2_X) #32832

swozniewski · 2021-02-05T17:26:43Z

PR description:

Backports #32676 and #32838 in order to have a consistent setup for phase2 HLT DeepTauID developments from CMSSW_11_1_X on (together with another backport #32831 to 11_1_X).

PR validation:

Automatically cherry-picked commits from original PR. No particular validation done again.

…that the list of working-points is empty

…olation pT-sum The HGCal work-around is motivated by this presentation: https://indico.cern.ch/event/954050/contributions/4013705/attachments/2103331/3536699/HLTTauTrigger_status_2020Sep16.pdf

- added option to save all input variables to JSON file (this feature allows to compare the input variables in C++ vs python/during the DeepTau training)

…x for accessing the grid of low-level variables

…d" to true in ParameterSet description (true restores the old behaviour of the code)

…Konstantin

…147) * Change default values of number of hits variables to not be negative.… … * Use case in getPuppiWeight, remove unnecessary lines and run code checks * Update puppiweights and remove default_value from deeptau v2 related code * Add newline

cmsbuild · 2021-02-05T17:27:04Z

A new Pull Request was created by @swozniewski for CMSSW_11_2_X.

It involves the following packages:

RecoTauTag/RecoTau

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from Further preparations of deep tau ID for hlt phase2 #32676

slava77 · 2021-02-05T17:51:25Z

@cmsbuild please test

cmsbuild · 2021-02-05T20:25:49Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e6ea02/12742/summary.html
COMMIT: 6644452
CMSSW: CMSSW_11_2_X_2021-02-05-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32832/12742/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2528845
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2528822
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
Checked 151 log files, 37 edm output root files, 36 DQM output files

cmsbuild · 2021-02-05T22:40:26Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_11_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

slava77 · 2021-02-06T15:39:10Z

I removed the +1 until #32837 is resolved or understood as unrelated

slava77 · 2021-02-07T03:30:24Z

RecoTauTag/RecoTau/plugins/DeepTauId.cc

@@ -1013,11 +1039,19 @@ namespace {
    int getPhiTensorIndex(const CellIndex& cellIndex) const { return cellIndex.phi + maxPhiIndex(); }

    bool tryGetCellIndex(double deltaEta, double deltaPhi, CellIndex& cellIndex) const {
-      static auto getCellIndex = [](double x, double maxX, double size, int& index) {
+      static auto getCellIndex = [this](double x, double maxX, double size, int& index) {


Suggested change

static auto getCellIndex = [this](double x, double maxX, double size, int& index) {

const auto getCellIndex = [this](double x, double maxX, double size, int& index) {

to match #32838

added backport of #32838 and updated PR description accordingly

cmsbuild · 2021-02-08T10:33:51Z

Pull request #32832 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

perrotta · 2021-02-08T11:32:08Z

please test

cmsbuild · 2021-02-08T14:04:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e6ea02/12759/summary.html
COMMIT: 1b81df6
CMSSW: CMSSW_11_2_X_2021-02-07-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32832/12759/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2528845
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2528810
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 35 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 151 log files, 37 edm output root files, 36 DQM output files

slava77 · 2021-02-08T14:16:23Z

+1

for #32832 1b81df6

use non-static lambda in DeepTauId::CellGrid::tryGetCellIndex #32838 part is now included, which resolved ASAN problem in DeepTauId #32837 (the reason for earlier removal of the reco +1 for this PR)
tests pass; comparisons show no (relevant) differences

cmsbuild · 2021-02-08T14:16:47Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_11_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-02-11T13:02:01Z

+1

veelken and others added 18 commits February 5, 2021 18:17

pass WPList as ponter instead of by reference, to allow for the case …

b8952e6

…that the list of working-points is empty

added option to apply HGCal work-around when computing the charged is…

d962d06

…olation pT-sum The HGCal work-around is motivated by this presentation: https://indico.cern.ch/event/954050/contributions/4013705/attachments/2103331/3536699/HLTTauTrigger_status_2020Sep16.pdf

- improved checks of input variables

67f9333

- added option to save all input variables to JSON file (this feature allows to compare the input variables in C++ vs python/during the DeepTau training)

added disable_CellIndex_workaround parameter to enable/disable bug-fi…

d599817

…x for accessing the grid of low-level variables

set "disable_hcalFraction_workaround" and"disable_CellIndex_workaroun…

6d3cdc0

…d" to true in ParameterSet description (true restores the old behaviour of the code)

code polishing as suggested by Sebastian

3d77d67

more code polishing, following comments by Sebastian

ee6e7e9

set disable_hcalFraction_workaround to false by default

458c7a1

inverted definition of disable_CellIndex_workaround, as requested by …

b679c34

…Konstantin

apply code checks and automated format

fbad9a8

add mutex to protect set operation

4fa5ec7

better placement of mutex

5b7555d

more elegant method to ensure thread safety

6641d89

follow up on review

0f8c284

fix handling of CellGrids

771f977

follow up on review

5656d91

fix code format

6644452

cmsbuild added this to the CMSSW_11_2_X milestone Feb 5, 2021

cmsbuild added orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 5, 2021

cmsbuild added tests-started and removed tests-pending labels Feb 5, 2021

cmsbuild added tests-approved and removed tests-started labels Feb 5, 2021

cmsbuild added fully-signed reconstruction-approved labels Feb 5, 2021

cmsbuild added pending-signatures reconstruction-pending and removed fully-signed reconstruction-approved labels Feb 6, 2021

slava77 reviewed Feb 7, 2021

View reviewed changes

use non-static lambda in DeepTauId::CellGrid::tryGetCellIndex

1b81df6

cmsbuild added tests-pending and removed tests-approved labels Feb 8, 2021

cmsbuild added tests-started and removed tests-pending labels Feb 8, 2021

cmsbuild added tests-approved and removed tests-started labels Feb 8, 2021

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Feb 8, 2021

cmsbuild added orp-approved and removed orp-pending labels Feb 11, 2021

cmsbuild merged commit 8f24c58 into cms-sw:CMSSW_11_2_X Feb 11, 2021

mbluj deleted the CMSSW_11_2_X_tau-pog_deepTauForHLT_phase2_backport branch October 10, 2023 10:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Further preparations of deep tau ID for hlt phase2 (backport to CMSSW_11_2_X) #32832

Further preparations of deep tau ID for hlt phase2 (backport to CMSSW_11_2_X) #32832

swozniewski commented Feb 5, 2021 •

edited

cmsbuild commented Feb 5, 2021 •

edited

slava77 commented Feb 5, 2021

cmsbuild commented Feb 5, 2021

cmsbuild commented Feb 5, 2021

slava77 commented Feb 6, 2021

slava77 Feb 7, 2021

swozniewski Feb 8, 2021

cmsbuild commented Feb 8, 2021

perrotta commented Feb 8, 2021

cmsbuild commented Feb 8, 2021

slava77 commented Feb 8, 2021

cmsbuild commented Feb 8, 2021

qliphy commented Feb 11, 2021

	static auto getCellIndex = [this](double x, double maxX, double size, int& index) {
	const auto getCellIndex = [this](double x, double maxX, double size, int& index) {

Further preparations of deep tau ID for hlt phase2 (backport to CMSSW_11_2_X) #32832

Further preparations of deep tau ID for hlt phase2 (backport to CMSSW_11_2_X) #32832

Conversation

swozniewski commented Feb 5, 2021 • edited

PR description:

PR validation:

cmsbuild commented Feb 5, 2021 • edited

slava77 commented Feb 5, 2021

cmsbuild commented Feb 5, 2021

Comparison Summary

cmsbuild commented Feb 5, 2021

slava77 commented Feb 6, 2021

slava77 Feb 7, 2021

Choose a reason for hiding this comment

swozniewski Feb 8, 2021

Choose a reason for hiding this comment

cmsbuild commented Feb 8, 2021

perrotta commented Feb 8, 2021

cmsbuild commented Feb 8, 2021

Comparison Summary

slava77 commented Feb 8, 2021

cmsbuild commented Feb 8, 2021

qliphy commented Feb 11, 2021

swozniewski commented Feb 5, 2021 •

edited

cmsbuild commented Feb 5, 2021 •

edited