Some clean up on reco muon classes #31789

perrotta · 2020-10-14T15:35:34Z

PR description:

Some cleanup is applied in the MuonReco DataFormat, mostly replacing loop expressions with more readable range based ones. This simplifications, among other things, allowed pinpointing the issue described in #31783 (which is not addressed here, waiting for the input from the muon pog)

Moreover, also a few simplifications in the code are implemented, like avoiding repeated computations of the same quantities or reducing the loop iterations when possible. This is not expected to provide significant improvements in the timing, but it shouldn't hurt too.

The updates in the MuonIdProducer are meant to simplify and speed up the algo in a few points, and fix the name of a method which is only used internally to that producer (and therefore does not need to be propagated elsewhere).

PR validation:

I just checked that the code runs without errors in a typical Run2 workflow.
No change is expected in output: therefore, if any shows up from the automatic comparisons it may suggest some bug/typo in this implementation

@trocino @ArnabPurohit FYI

cmsbuild · 2020-10-14T15:36:01Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-14T15:42:44Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31789/19059

This PR adds an extra 48KB to repository

cmsbuild · 2020-10-14T15:43:13Z

A new Pull Request was created by @perrotta for master.

It involves the following packages:

DataFormats/MuonReco
RecoMuon/MuonIdentification

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@bellan, @abbiendi, @Fedespring, @echapon, @calderona, @HuguesBrun, @jhgoh, @battibass, @cericeci, @rovere, @trocino, @amagitte, @folguera, @rociovilar this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

perrotta · 2020-10-14T15:43:26Z

please test

cmsbuild · 2020-10-14T15:43:36Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-10-14-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9956/console Started: 2020/10/14 17:44

cmsbuild · 2020-10-14T16:25:52Z

-1

Tested at: 56cd1d6

CMSSW: CMSSW_11_2_X_2020-10-14-1100
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4796b4/9956/summary.html

I found follow errors while testing this PR

Failed tests: Build ClangBuild

Build:

I found compilation warning when building: See details on the summary page.

Clang:

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

cmsbuild · 2020-10-14T16:25:54Z

Comparison not run due to Build errors (RelVals and Igprof tests were also skipped)

perrotta · 2020-10-14T16:40:21Z

/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-14-1100/src/DataFormats/MuonReco/src/Muon.cc: In member function 'unsigned int reco::Muon::stationMask(reco::Muon::ArbitrationType) const':
  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-14-1100/src/DataFormats/MuonReco/src/Muon.cc:152:18: warning: unused variable 'rpcMatch' [-Wunused-variable]
        for (auto& rpcMatch : chamberMatch.rpcMatches) {
                  ^~~~~~~~
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-14-1100/src/DataFormats/MuonReco/src/Muon.cc: In member function 'unsigned int reco::Muon::RPClayerMask(reco::Muon::ArbitrationType) const':
  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_11_2_X_2020-10-14-1100/src/DataFormats/MuonReco/src/Muon.cc:237:16: warning: unused variable 'rpcMatch' [-Wunused-variable]
      for (auto& rpcMatch : chamberMatch.rpcMatches) {  // There is clearly something odd here
                ^~~~~~~~

Indeed, this is exactly the compilation warning that allowed me to pinpoint the issue reported in #31783

I can apply a patch here to reproduce the exact previous behavior without getting that warning (" just check if the size of chamberMatch->rpcMatches is above 0, compute once curmask in that case, and add it to the totMask").

But maybe it is better to understand the original intentions of the author of the code and apply a more appropriate fix, if needed

VinInn · 2020-10-14T16:59:07Z

DataFormats/MuonReco/interface/Muon.h

-             segment != chamber->segmentMatches.end();
-             ++segment) {
+      for (auto& chamber : muMatches_)
+        for (auto& segment : chamber.segmentMatches) {


what about:
if (i+chamber.segmentMatches.size()<n ) i+=chamber.segmentMatches.size()) continue;
return chamber.segmentMatches[n-i].t0;

?

VinInn · 2020-10-14T17:01:24Z

DataFormats/MuonReco/src/MuonSelectors.cc

+    for (int detectorIdx = 1; detectorIdx < 3; ++detectorIdx) {
+      float dist = muon.trackDist(stationIdx, detectorIdx, arbitrationType);
+      if (dist < maxChamberDist &&
+          dist / muon.trackDistErr(stationIdx, detectorIdx, arbitrationType) < maxChamberDistPull)


dist < maxCham... * muon.trackDist.....;

avoid a costly div

VinInn · 2020-10-14T17:03:39Z

RecoMuon/MuonIdentification/plugins/MuonIdProducer.cc

@@ -564,7 +564,7 @@ void MuonIdProducer::produce(edm::Event& iEvent, const edm::EventSetup& iSetup)

        for (auto& muon : *outputMuons) {
          if (muon.innerTrack().get() == trackerMuon.innerTrack().get() &&
-              cos(phiOfMuonIneteractionRegion(muon) - phiOfMuonIneteractionRegion(trackerMuon)) > 0) {
+              cos(phiOfMuonInteractionRegion(muon) - phiOfMuonInteractionRegion(trackerMuon)) > 0) {


cos(alpha) >0 if |alpha|<pi/2 once alpha is between -pi and pi....
please used dphi function and avoid calling cos.

VinInn · 2020-10-14T17:09:38Z

RecoMuon/MuonIdentification/plugins/MuonIdProducer.cc

@@ -665,7 +665,7 @@ void MuonIdProducer::produce(edm::Event& iEvent, const edm::EventSetup& iSetup)
      // predict direction based on the muon interaction region location
      // if it's available
      if (muon.isStandAloneMuon()) {
-        if (cos(phiOfMuonIneteractionRegion(muon) - muon.phi()) > 0) {
+        if (cos(phiOfMuonInteractionRegion(muon) - muon.phi()) > 0) {


cmsbuild · 2020-10-20T07:29:34Z

The tests are being triggered in jenkins.
Test Parameters:

ENABLE_BOT_TESTS = PROFILING
CMSSW_11_2_X_2020-10-19-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10115/console Started: 2020/10/20 09:40
CMSSW_11_2_X_2020-10-19-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10122/console Started: 2020/10/20 13:43

cmsbuild · 2020-10-20T10:02:43Z

+1
Tested at: 59b3826
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4796b4/10115/summary.html
CMSSW: CMSSW_11_2_X_2020-10-19-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-20T10:02:51Z

Comparison job queued.

perrotta · 2020-10-20T11:01:02Z

igprof with the automatic tests keeps failing. By the way, if I run the same workflow 23434.21 myself, I get the very same error on opening the input MB file for the pileup: I suspect there is really something related to that file, or its physical location.

However, I run igprof on another PU workflow, 10224, and compared the timing for muonIdProducer without and with this PR. As I anticipated in the PR description, the difference appears to be rather small, at the limit of the significance of the method:

Baseline CMSSW_11_2_X_2020-10-18-2300:

    4.8      46.12  MuonIdProducer::produce(edm::Event&, edm::EventSetup const&) [64]
    4.4      42.18  MuonIdProducer::fillMuonId(edm::Event&, edm::EventSetup const&, reco::Muon&, TrackDetectorAssociator::Direction) [68]

Baseline + this PR:

    4.7      45.47  MuonIdProducer::produce(edm::Event&, edm::EventSetup const&) [64]
    4.3      41.57  MuonIdProducer::fillMuonId(edm::Event&, edm::EventSetup const&, reco::Muon&, TrackDetectorAssociator::Direction) [69]

Nevertheless, I would still be in favor of integrating this PR, if there are no opinions against it

cmsbuild · 2020-10-20T15:51:19Z

+1
Tested at: 59b3826
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4796b4/10122/summary.html
CMSSW: CMSSW_11_2_X_2020-10-19-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-20T15:51:22Z

Comparison job queued.

smuzaffar · 2020-10-20T16:24:35Z

@perrotta , profiling works now

cmsbuild · 2020-10-20T17:07:32Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4796b4/10122/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2544100
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2544077
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

perrotta · 2020-10-21T07:55:57Z

DataFormats/MuonReco/src/Muon.cc

-           rpcMatch != chamberMatch->rpcMatches.end();
-           rpcMatch++) {
-        curMask = 1 << ((chamberMatch->station() - 1) + 4 * (rpcIndex - 1));
+      //    for (auto& rpcMatch : chamberMatch.rpcMatches) { // TO BE FIXED: there is clearly something odd here


@jhgoh please confirm that what implemented here for the rpc matches (which faithfully reproduces what was coded so far in CMSSW) is actually the sought behaviour, and it did not reveal a bug in the the original code instead.
If it is confirmed being ok, I will remove the comment lines left as a warning.

perrotta · 2020-10-22T10:06:08Z

+1

Some code optimization for the muon data format and for the muonId plugin is implemented, without changing the final outcomes of the previous algorithms
Jenkins tests pass and show no differences wrt the baseline; the cpu timing improvement due to this optimization appears to be rather limited, though
Since I got no answer to my comment in Some clean up on reco muon classes #31789 (review), I go ahead with approving this PR (because the original algo is reproduced here, thus no changes in practice for it), but I let the comment lines of warning in the code. github issue Possible bug in the usage of rpcMatches in a reco::Muon #31783 will remain open until I get the final confirmation that the original code was exactly meant to be coded as such, and that there was no mistake in it: at that point, we will remove those comment lines from the release.

cmsbuild · 2020-10-22T10:06:33Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-10-22T12:42:07Z

+1

perrotta added 3 commits October 14, 2020 16:20

Some cleanup in DataFormat/MuonReco/Muon

59e3e8f

Some cleanup in DataFormat/MuonReco/MuonSelectors

0472021

Some cleanup in MuonIdProducer

56cd1d6

cmsbuild added this to the CMSSW_11_2_X milestone Oct 14, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 14, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 14, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 14, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Oct 14, 2020

VinInn reviewed Oct 14, 2020

View reviewed changes

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Oct 20, 2020

cmsbuild added tests-started and removed tests-pending labels Oct 20, 2020

cmsbuild added tests-approved and removed tests-started labels Oct 20, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 20, 2020

perrotta commented Oct 21, 2020

View reviewed changes

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Oct 22, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 22, 2020

cmsbuild merged commit 325b07c into cms-sw:master Oct 22, 2020

perrotta deleted the cleanUpDataFormatMuon branch October 22, 2020 12:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some clean up on reco muon classes #31789

Some clean up on reco muon classes #31789

perrotta commented Oct 14, 2020 •

edited

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

perrotta commented Oct 14, 2020

cmsbuild commented Oct 14, 2020 •

edited

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

perrotta commented Oct 14, 2020

VinInn Oct 14, 2020

VinInn Oct 14, 2020

VinInn Oct 14, 2020 •

edited

VinInn Oct 14, 2020

cmsbuild commented Oct 20, 2020 •

edited

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

perrotta commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

smuzaffar commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

perrotta Oct 21, 2020

perrotta commented Oct 22, 2020

cmsbuild commented Oct 22, 2020

silviodonato commented Oct 22, 2020

Some clean up on reco muon classes #31789

Some clean up on reco muon classes #31789

Conversation

perrotta commented Oct 14, 2020 • edited

PR description:

PR validation:

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

perrotta commented Oct 14, 2020

cmsbuild commented Oct 14, 2020 • edited

cmsbuild commented Oct 14, 2020

cmsbuild commented Oct 14, 2020

perrotta commented Oct 14, 2020

VinInn Oct 14, 2020

Choose a reason for hiding this comment

VinInn Oct 14, 2020

Choose a reason for hiding this comment

VinInn Oct 14, 2020 • edited

Choose a reason for hiding this comment

VinInn Oct 14, 2020

Choose a reason for hiding this comment

cmsbuild commented Oct 20, 2020 • edited

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

perrotta commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

smuzaffar commented Oct 20, 2020

cmsbuild commented Oct 20, 2020

perrotta Oct 21, 2020

Choose a reason for hiding this comment

perrotta commented Oct 22, 2020

cmsbuild commented Oct 22, 2020

silviodonato commented Oct 22, 2020

perrotta commented Oct 14, 2020 •

edited

cmsbuild commented Oct 14, 2020 •

edited

VinInn Oct 14, 2020 •

edited

cmsbuild commented Oct 20, 2020 •

edited