Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

avoid overflowing firstStrip_ in case of approximated clusters #38735

Merged
merged 1 commit into from Jul 21, 2022

Conversation

mmusich
Copy link
Contributor

@mmusich mmusich commented Jul 14, 2022

resolves #38729

PR description:

See #38729 (comment)

PR validation:

Tested running 140.58 successfully under CMSSW_12_5_ASAN_X_2022-07-13-1100

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38735/31048

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

  • DataFormats/SiStripCluster (reconstruction)

@jpata, @cmsbuild, @clacaputo can you please review it and eventually sign? Thanks.
@echabert, @swertz, @robervalwalsh, @rovere, @VinInn, @alesaggio, @threus this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor Author

mmusich commented Jul 14, 2022

test parameters:

  • workflow = 140.58

@mmusich
Copy link
Contributor Author

mmusich commented Jul 14, 2022

@cmsbuild, please test for CMSSW_12_5_ASAN_X

@mmusich mmusich changed the title avoid overflowing firstStrip_ in case of approximated clusters [RFC] avoid overflowing firstStrip_ in case of approximated clusters Jul 14, 2022
@@ -28,7 +28,7 @@ SiStripCluster::SiStripCluster(const SiStripApproximateCluster cluster) : error_
amplitudes_.resize(cluster.width(), cluster.avgCharge());

//initialize firstStrip_
firstStrip_ = cluster.barycenter() - cluster.width() / 2;
firstStrip_ = std::max(0.f, cluster.barycenter() - cluster.width() / 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you can recycle here the same barycenter_ already computed a few lines above...

Copy link
Contributor Author

@mmusich mmusich Jul 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure. I have more doubts though:

  1. given cluster.width() here is a uint8_t

    uint8_t width() const { return width_; }
    should it be divided by 2.f to avoid truncating the half-width of one half-strip?

  2. I am not sure if there's a better (more efficient) way of computing the max since this should be a pretty intensive call in production.

@abaty @mandrenguyen

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

I think Marco is correct on item 1, we should be dividing by a float here. I am not familiar with the most efficient way of implementing item 2, although we could check the timing if it is a strong concern.

Copy link
Contributor

@VinInn VinInn Jul 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plase 0.5f*float(cluster.width())
max is not slower than the above (and much faster than cluster.width() / 2.)

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26240/summary.html
COMMIT: 9330786
CMSSW: CMSSW_12_5_ASAN_X_2022-07-13-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38735/26240/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26240/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26240/git-merge-result

RelVals

  • 140.58140.58_RunHI2018+RunHI2018+RAWPRIMEHI18+RECOHID18APPROXCLUSTERS+HARVESTDHI18/step3_RunHI2018+RunHI2018+RAWPRIMEHI18+RECOHID18APPROXCLUSTERS+HARVESTDHI18.log

@mmusich
Copy link
Contributor Author

mmusich commented Jul 15, 2022

the other problem we have uncovered now is coming from the other "side" of the module, see log.

By adding:

diff --git a/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc b/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc
index dcdf774f0ef..58d1cc53b97 100644
--- a/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc
+++ b/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc
@@ -42,6 +42,8 @@ std::vector<float> SiStripClusterInfo::stripNoisesRescaledByGain() const {
   std::vector<float> results;
   results.reserve(width());
   for (size_t i = 0, e = width(); i < e; i++) {
+    std::cout << "i: " << i << " firstStrip: " << firstStrip() << " getting strip:" << firstStrip() + i << std::endl;
+
     results.push_back(siStripNoises_->getNoise(firstStrip() + i, detNoiseRange) /
                       siStripGain_->getStripGain(firstStrip() + i, detGainRange));
   }

I get:

i: 0 firstStrip: 757 getting strip:757
i: 1 firstStrip: 757 getting strip:758
i: 2 firstStrip: 757 getting strip:759
i: 3 firstStrip: 757 getting strip:760
i: 4 firstStrip: 757 getting strip:761
i: 5 firstStrip: 757 getting strip:762
i: 6 firstStrip: 757 getting strip:763
i: 7 firstStrip: 757 getting strip:764
i: 8 firstStrip: 757 getting strip:765
i: 9 firstStrip: 757 getting strip:766
i: 10 firstStrip: 757 getting strip:767
i: 11 firstStrip: 757 getting strip:768
=================================================================
==3626964==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f8e8c3717ff at pc 0x7f8f1bb0beba bp 0x7ffe3fda2080 sp 0x7ffe3fda2078
READ of size 1 at 0x7f8e8c3717ff thread T0

so it can happen that firstStrip() can exceed the total number of strips in the module...
I am not sure how to solve this, because one could clamp between 0.f and (total number of strips - cluster width), but the SiStripCluster is agnostic w.r.t. the DetId and the total number of strips depends on the DetId...

@mmusich
Copy link
Contributor Author

mmusich commented Jul 15, 2022

with bd09b48 the failing workflow runs, but it doesn't address the case of the 4 APV modules:

I am not sure how to solve this, because one could clamp between 0.f and (total number of strips - cluster width), but the SiStripCluster is agnostic w.r.t. the DetId and the total number of strips depends on the DetId...

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38735/31068

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

Pull request #38735 was updated. @jpata, @cmsbuild, @clacaputo can you please check and sign again.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38735/31128

  • This PR adds an extra 16KB to repository

@cmsbuild
Copy link
Contributor

Pull request #38735 was updated. @jpata, @cmsbuild, @clacaputo can you please check and sign again.

@clacaputo
Copy link
Contributor

@cmsbuild, please test for CMSSW_12_5_ASAN_X

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26321/summary.html
COMMIT: 5895157
CMSSW: CMSSW_12_5_ASAN_X_2022-07-18-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38735/26321/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_ProduceSystematicMisalignment had ERRORS
---> test testAlignmentOfflineValidation had ERRORS
---> test test_PixelBaryCentreTool had ERRORS
---> test testPedeCampaign had ERRORS
and more ...

@mmusich
Copy link
Contributor Author

mmusich commented Jul 19, 2022

-1
Failed Tests: UnitTests

Failures are unrelated, already seen in the IB: https://cmssdt.cern.ch/SDT/cgi-bin/showBuildLogs.py/el8_amd64_gcc10/www/mon/12.5.ASAN-mon-11/CMSSW_12_5_ASAN_X_2022-07-18-1100?utests

@clacaputo
Copy link
Contributor

+reconstruction

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

please test
(As it was changed since #38735 (comment))

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26349/summary.html
COMMIT: 5895157
CMSSW: CMSSW_12_5_X_2022-07-20-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38735/26349/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7fab06/140.58_RunHI2018+RunHI2018+RAWPRIMEHI18+RECOHID18APPROXCLUSTERS+HARVESTDHI18

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3723682
  • DQMHistoTests: Total failures: 7276
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3716384
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 212 log files, 45 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

@qliphy
Copy link
Contributor

qliphy commented Jul 21, 2022

+1

@cmsbuild cmsbuild merged commit d24ce62 into cms-sw:master Jul 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Out of bounds read in SiStripMonitorCluster::analyze
7 participants