Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41545

AlexDeMoor · 2023-05-04T16:24:02Z

PR description:

See description of #41275 - new tagger for Run 3 for MiniAOD and NanoAOD, update of DeepJet for Run 3, and removing outdated DeepCSV for Run 3.
Additions are verbatim copies of original PR, modifications have been updated to be in line with latest PNet additions into the same packages.

Collaboration with @AnnikaStein

PR validation:

Compilation, basic checks similar to original PR.

scram b runtests scram build code-checks scram build code-format scram build updateclassversion

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #41275

* added EXOLLPJetHCAL skim (initial version) * added test script test_EXOLLPJetHCAL_SKIM.py * Replace cout with edm::Logverbatim in Geometry/GlobalTrackingGeometryBuilder * Re-enable unit tests for ASAN build * dd4hep phase2 wf * Add wf *.81 for BPH DQM * Added utility script edmFileInPath * improve unt test; use edmFileInPath; convert to bash * Replace std::cout with edm::LogVerbatim in Geometry/HcalEventSetup and Geometry/HcalTestBeamData. Also add some missing files needed for testing * Code check * Remove FWCore TestHelper utility * Add ap_data type for GT interface Update GMT emulator for new ap format Also clean up the config file Update Z0/D0 according to GT Using 0.05 for Z0 and 0.03 for d0. update code format Bug fix in tracker muon matching Thanks to Santi for spotting this. This line was commented out by someone, likely by accident. Update the reference type to SA muons * improve unit test testCreateRandomMisalignment * Add back removed data version * fix bug in computation of barycenters per partition and improve graphical display * get rid of gROOT->Reset() in GeometryComparisonPlotter.cc --------- Co-authored-by: Kiley Kennedy <kiley.elizabeth.kennedy@cern.ch> Co-authored-by: Sunanda <sunanda.banerjee@cern.ch> Co-authored-by: Shahzad Malik Muzaffar <shahzad.malik.muzaffar@cern.ch> Co-authored-by: Norraphat <srimanob@mail.cern.ch> Co-authored-by: AdrianoDee <adriano.diflorio@ba.infn.it> Co-authored-by: Zhenbin Wu <zhenbin.wu@gmail.com> Co-authored-by: cmsbuild <cmsbuild@cern.ch> Co-authored-by: mmusich <marco.musich@cern.ch> Co-authored-by: Andrew David Loeliger <andrew.loeliger@cern.ch>

* Modifications to JetTags for jet selections Modify the JetTags producer for skipping the non selected jets and processing the good ones only * Update ParT TagInfo for jet selection * Add the bool for jet selection

cmsbuild · 2023-05-04T16:24:35Z

A new Pull Request was created by @AlexDeMoor (Alexandre De Moor) for CMSSW_13_1_X.

It involves the following packages:

DataFormats/BTauReco (reconstruction)
PhysicsTools/NanoAOD (xpog)
PhysicsTools/PatAlgos (xpog, reconstruction)
RecoBTag/Configuration (reconstruction)
RecoBTag/FeatureTools (reconstruction)
RecoBTag/ONNXRuntime (reconstruction)

@swertz, @vlimant, @clacaputo, @cmsbuild, @simonepigazzini, @mandrenguyen can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @hatakeyamak, @emilbols, @mbluj, @demuller, @seemasharmafnal, @mmarionncern, @missirol, @ahinzmann, @jdolen, @azotz, @hqucms, @rovere, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @andrzejnovak, @AlexDeMoor, @AnnikaStein, @JyothsnaKomaragiri, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41275

perrotta · 2023-05-05T05:58:22Z

please test

perrotta · 2023-05-05T08:34:02Z

please test
(To be sure that it picks the correct cmsdist PR: maybe I was too fast before...)

cmsbuild · 2023-05-05T13:48:39Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3aff2/32410/summary.html
COMMIT: 4e2ae70
CMSSW: CMSSW_13_1_X_2023-05-04-2300/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41545/32410/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

134.709134.709_RunSinglePh2015B/step2_RunSinglePh2015B.log

Comparison Summary

Summary:

You potentially removed 14 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 1057 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3460672
DQMHistoTests: Total failures: 1679
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3458971
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -3.5940000000000007 KiB( 47 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -0.438 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): -0.264 KiB Physics/NanoAODDQM
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 2 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 15 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12171
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 12171
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 11.106000000000003 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 2.696 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.620 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.401,... ): -0.438 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.5111,... ): -0.264 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.438	2.387	0.051 ( +2.1% )	5.15	5.47	-5.7%	1.498	1.424
2500.311	2.545	2.487	0.058 ( +2.3% )	4.62	4.93	-6.2%	1.891	1.788
2500.312	2.491	2.437	0.054 ( +2.2% )	4.77	5.10	-6.4%	1.883	1.769
2500.33	1.240	1.206	0.034 ( +2.8% )	9.64	10.34	-6.7%	1.670	1.484
2500.331	1.603	1.550	0.054 ( +3.5% )	5.23	5.63	-7.1%	1.836	1.650
2500.332	1.485	1.444	0.041 ( +2.9% )	7.55	8.03	-6.0%	1.872	1.578
2500.401	2.153	2.184	-0.030 ( -1.4% )	5.29	5.23	+1.1%	1.359	1.297
2500.501	1.752	1.768	-0.017 ( -0.9% )	9.01	8.79	+2.5%	1.269	1.180
2500.511	1.144	1.162	-0.018 ( -1.5% )	4.34	4.39	-1.3%	1.562	1.456
2500.5111	1.502	1.520	-0.018 ( -1.2% )	3.81	3.81	+0.2%	1.605	1.466
2500.601	2.014	2.049	-0.035 ( -1.7% )	12.66	12.36	+2.4%	1.170	1.048

AnnikaStein · 2023-05-07T05:08:44Z

Could the tests be restarted (with the usual test parameters)? If it's a timeout only, maybe next run on another machine we are lucky and it works just fine.

simonepigazzini · 2023-05-08T07:08:03Z

please test

simonepigazzini · 2023-05-08T07:09:23Z

let's see. Anyway I think we can be happy given that the changes to NANO are identical to the original PR.
@AnnikaStein can you please confirm that the changes in the DAQ for NANO are as expected?

AnnikaStein · 2023-05-08T10:18:17Z

let's see. Anyway I think we can be happy given that the changes to NANO are identical to the original PR. @AnnikaStein can you please confirm that the changes in the DAQ for NANO are as expected?

@simonepigazzini looking alright, RobustParTAK4 got added for the Run 2 workflows 2500.31 -- 332 (based on the modifier), and the plots for 12.2, 12.4, 12.6 (2500.401 -- 601) exist but are filled with -1 only, as the mini is not rerun again - so these results are all expected, similar to what we had for the other PRs.

btagCMVA is gone from the plots (was empty anyway), DeepCSV which used to be filled in the DQM is now also gone for Run 3

cmsbuild · 2023-05-08T11:51:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e3aff2/32458/summary.html
COMMIT: 4e2ae70
CMSSW: CMSSW_13_1_X_2023-05-07-2300/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41545/32458/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 1061 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3460672
DQMHistoTests: Total failures: 1681
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3458969
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -3.5940000000000007 KiB( 47 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -0.438 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): -0.264 KiB Physics/NanoAODDQM
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 6 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 15 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12171
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 12171
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 11.106000000000003 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 2.696 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.620 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.401,... ): -0.438 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.5111,... ): -0.264 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.438	2.387	0.051 ( +2.1% )	5.21	5.52	-5.6%	1.489	1.478
2500.311	2.545	2.487	0.058 ( +2.3% )	4.68	4.96	-5.7%	1.875	1.868
2500.312	2.491	2.437	0.054 ( +2.2% )	4.83	5.17	-6.6%	1.875	1.857
2500.33	1.240	1.206	0.034 ( +2.8% )	9.84	10.57	-6.9%	1.671	1.661
2500.331	1.603	1.550	0.054 ( +3.5% )	5.27	5.66	-6.9%	1.842	1.814
2500.332	1.485	1.444	0.041 ( +2.9% )	7.53	8.08	-6.9%	1.873	1.874
2500.401	2.153	2.184	-0.030 ( -1.4% )	5.30	5.31	-0.1%	1.347	1.346
2500.501	1.752	1.768	-0.017 ( -0.9% )	9.07	9.05	+0.2%	1.255	1.257
2500.511	1.144	1.162	-0.018 ( -1.5% )	4.40	4.44	-0.9%	1.555	1.553
2500.5111	1.502	1.520	-0.018 ( -1.2% )	3.90	3.85	+1.2%	1.600	1.599
2500.601	2.014	2.049	-0.035 ( -1.7% )	12.76	12.60	+1.3%	1.148	1.148

simonepigazzini · 2023-05-08T12:03:09Z

+1

clacaputo · 2023-05-09T15:15:11Z

+reconstruction

cmsbuild · 2023-05-09T15:15:38Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2023-05-09T19:02:44Z

+1

AnnikaStein and others added 19 commits March 29, 2023 10:12

ParTAK4 V00 for 131X

6ad7665

ParT PUPPI Run 3 WIP

598ec0a

paths to new models (122X training)

d236625

code-checks, code-format

71b0f04

cleanup

98c040e

cleanup

ce6bb4f

rm tests

07aac71

Merge branch 'master' into from-CMSSW_13_1_0_pre2

69c5fe7

restore puppiw compatibility after merging

92ea010

code-format again

fdca404

update custom_jme, nanoDQM

7a383cd

Merge branch 'cms-sw:master' into from-CMSSW_13_1_0_pre2

358a861

ParT from mini

8a8eeaf

replace nominal with robust+fast ParT model

b0c0838

Add the jet selection for ParT AK4 (#6)

a8947fa

* Modifications to JetTags for jet selections Modify the JetTags producer for skipping the non selected jets and processing the good ones only * Update ParT TagInfo for jet selection * Add the bool for jet selection

fix typo: add missing char for CHS,PUPPI nano

948155f

Add modifications respecting comments (#7)

c857a54

fix kwargs for RobustParTAK4 to resolve conflict with tau pog addition

4e2ae70

cmsbuild added this to the CMSSW_13_1_X milestone May 4, 2023

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending xpog-pending labels May 4, 2023

AlexDeMoor changed the title ~~Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41434~~ Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano May 4, 2023

cmsbuild added tests-started and removed tests-pending labels May 5, 2023

cmsbuild added tests-pending and removed tests-started labels May 5, 2023

cmsbuild added tests-started and removed tests-pending labels May 5, 2023

cmsbuild added tests-rejected and removed tests-started labels May 5, 2023

cmsbuild added tests-started and removed tests-rejected labels May 8, 2023

cmsbuild added tests-approved and removed tests-started labels May 8, 2023

cmsbuild added xpog-approved and removed xpog-pending labels May 8, 2023

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels May 9, 2023

cmsbuild added orp-approved and removed orp-pending labels May 9, 2023

cmsbuild merged commit ac69812 into cms-sw:CMSSW_13_1_X May 9, 2023
27 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41545

Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41545

AlexDeMoor commented May 4, 2023

cmsbuild commented May 4, 2023 •

edited

perrotta commented May 5, 2023

perrotta commented May 5, 2023

cmsbuild commented May 5, 2023

AnnikaStein commented May 7, 2023

simonepigazzini commented May 8, 2023

simonepigazzini commented May 8, 2023

AnnikaStein commented May 8, 2023

cmsbuild commented May 8, 2023

simonepigazzini commented May 8, 2023

clacaputo commented May 9, 2023

cmsbuild commented May 9, 2023

rappoccio commented May 9, 2023

Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41545

Backport to 13_1_X of #41275 Introduce (Robust)ParTAK4 jet tagger, DeepJet model update for Run 3, remove DeepCSV from nano #41545

Conversation

AlexDeMoor commented May 4, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented May 4, 2023 • edited

perrotta commented May 5, 2023

perrotta commented May 5, 2023

cmsbuild commented May 5, 2023

RelVals-INPUT

Comparison Summary

NANO Comparison Summary

AnnikaStein commented May 7, 2023

simonepigazzini commented May 8, 2023

simonepigazzini commented May 8, 2023

AnnikaStein commented May 8, 2023

cmsbuild commented May 8, 2023

Comparison Summary

NANO Comparison Summary

simonepigazzini commented May 8, 2023

clacaputo commented May 9, 2023

cmsbuild commented May 9, 2023

rappoccio commented May 9, 2023

cmsbuild commented May 4, 2023 •

edited