[Backport 12_3_X] Update to DNN-based strategy for outside-in seed generation in Muon HLT

[kondratyevd]

This is a backport for #37437.
The backport is needed to integrate the OI seeding upgrade into the V2 HLT menu.
There are no changes to the code w.r.t. original PR.

@missirol

[cmsbuild]

A new Pull Request was created by @kondratyevd (Dmitry Kondratyev) for CMSSW_12_3_X.

It involves the following packages:

• RecoMuon/TrackerSeedGenerator (reconstruction)

@jpata, @cmsbuild, @clacaputo, @slava77 can you please review it and eventually sign? Thanks.
@HuguesBrun, @abbiendi, @Fedespring, @bellan, @sscruz, @jhgoh, @CeliaFernandez, @trocino, @cericeci, @rociovilar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[missirol]

urgent

Requested for HLT development in 12_3_0, see #37437 (comment).

[missirol]

please test

The PR tests are not sensitive to this PR, see #37437 (comment).

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a821c2/23662/summary.html
COMMIT: 4bb8edb
CMSSW: CMSSW_12_3_X_2022-04-05-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37467/23662/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 10 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3697381
• DQMHistoTests: Total failures: 19
• DQMHistoTests: Total nulls: 1
• DQMHistoTests: Total successes: 3697339
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.004 KiB( 48 files compared)
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[JanFSchulte]

Just a reminder that we also need cms-data/RecoMuon-TrackerSeedGenerator#4 for this update. So once it is merged we need a PR to update the version number of the relevant package in this file: https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_3_X/master/data/cmsswdata.txt

For that we first need to know if the memory consumption reported in these profiles is acceptable: #37437 (comment)

[missirol]

I can't really judge this myself right away. This might be worth discussing in a TSG meeting to get better feedback (cc: @silviodonato). Or what is it already discussed in TSG?

If I look at the % values, this looks like a non-negligible increase, although it is a few percent of the total (<5%). At the same time, this looks like a very large increase compared to the previous model (x10 - x20 ?), if I understood correctly:

• Link for old model: DNN-based optimization of seeding in outside-in sequence of Muon HLT #35237 (comment)
• Link for new model: Update to DNN-based strategy for outside-in seed generation in Muon HLT #37437 (comment)

Was this increase expected? By the way, should the model be reviewed by a ML contact, to see if there are ways to improve it?

I don't think this necessarily stops this PR from proceeding, but these questions should be addressed before this is actually integrated in the HLT menu.

[smuzaffar]

test parameters:

• pull_requests = Update tag for RecoMuon-TrackerSeedGenerator to V00-04-00 cmsdist#7755

[smuzaffar]

cmsdist PR to get the new data files is cms-sw/cmsdist#7755

[smuzaffar]

please test

[kondratyevd]

@missirol yes, the size increase is understood, it is simply due to the increased number of weights in the network.
We did not limit the network size during optimization, as we are not aware of what the memory limitation would be.

I suppose, we can re-optimize the network for a given limitation on the size of the model, if needed.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a821c2/23689/summary.html
COMMIT: 4bb8edb
CMSSW: CMSSW_12_3_X_2022-04-05-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/37467/23689/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @mmusich minor changes to SiPixelLorentzAnglePCL modules [12.3.X] #37459
• @missirol Adding option to disable resetting L1 PS counters on ls change [12_3_X] #37462
• @gk199 [Backport of] HCAL finegrain bit fix to account for TDC packing, and SOI alignment #37439
• @eyigitba EMTF Unpacker Update to Add HMT and Displaced Muon Information [12_3_X] #37465
• @fwyzard Update all GPU workflows [12.3.x] #37413
• @fwyzard Update OpenMPI to version 4.1.3 [12.3.x] cmsdist#7742
• @cms-sw Update tag for RecoMuon-TrackerSeedGenerator to V00-04-00 cmsdist#7755

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a821c2/23689/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a821c2/23689/git-merge-result

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3697381
• DQMHistoTests: Total failures: 70
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3697289
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[perrotta]

@cms-sw/reconstruction-l2 would you agree signing this PR, verbatim backport of #37437 ?
By mistake the cmsdist PR which updates the cms-data inputs needed for it was already merged, see cms-sw/cmsdist#7755. Of course, if there are objections to this backport we'll have to revert it before final 12_3_0.

[jpata]

+reconstruction

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_3_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_4_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1