Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[10.0.x] remove memory leak in PVValidation and fix ini file #21428

Merged
merged 4 commits into from Nov 28, 2017

Conversation

mmusich
Copy link
Contributor

@mmusich mmusich commented Nov 22, 2017

The purpose of this PR is to remove the large memory leak introduced in c3db41b.
As the cloned recHit was not destroyed after being used, running over a large sample of data was resulting in jobs killed because of memory exhaustion.
For comparison here is the RSS performance measured by running over 10k events of run 305967 of /HLTPhysics/Run2017F-TkAlMinBias-PromptReco-v1/ALCARECO

alltrends_vs_cmssw_10_0_0_pre1_vs_this_fix

I profit of this PR also to clean-up two other (relatively minor) memory leaks as well as to change the nomenclature in the example ini file, which is now broken due to fb07740.

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/PR-21428/2145

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

Alignment/OfflineValidation

@ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @lpernie can you please review it and eventually sign? Thanks.
@mschrode, @ghellwig, @mmusich, @tocheng, @tlampen this is something you requested to watch as well.
@davidlange6, @slava77 you are the release manager for this.

cms-bot commands are listed here

@davidlange6
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 22, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/24609/console Started: 2017/11/22 14:48

@ghellwig
Copy link

@mmusich sorry, I overlooked the example ini file

@mmusich
Copy link
Contributor Author

mmusich commented Nov 22, 2017

@ghellwig, no problem, I don't mind (as long as it works out of the box :) )

@VinInn
Copy link
Contributor

VinInn commented Nov 22, 2017

why cloning in first place?

@mmusich
Copy link
Contributor Author

mmusich commented Nov 22, 2017

@VinInn, admittedly no good reason - just bad copy paste of other code from my side... fixing

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

Pull request #21428 was updated. @ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @lpernie can you please check and sign again.

@mmusich mmusich changed the title [10.0.0.x] remove memory leak in PVValidation and fix ini file [10.0.x] remove memory leak in PVValidation and fix ini file Nov 22, 2017
@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@davidlange6
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 22, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/24634/console Started: 2017/11/22 19:08

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-21428/24634/summary.html

There are some workflows for which there are errors in the baseline:
10824.0 step 5
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 27
  • DQMHistoTests: Total histograms compared: 2717769
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2717592
  • DQMHistoTests: Total skipped: 176
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.13000000006 KiB( 23 files compared)
  • Checked 111 log files, 8 edm output root files, 27 DQM output files

@mmusich
Copy link
Contributor Author

mmusich commented Nov 27, 2017

@arunhep @lpernie, as the backport to 94x of this PR (#21429) is already fully signed, any objection to sign also this one?

@lpernie
Copy link
Contributor

lpernie commented Nov 27, 2017

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2)

@davidlange6
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit d5344cb into cms-sw:master Nov 28, 2017
@@ -186,6 +186,8 @@ PrimaryVertexValidation::~PrimaryVertexValidation()
{
// do anything here that needs to be done at desctruction time
// (e.g. close files, deallocate resources etc.)
if (theTrackFilter_) delete theTrackFilter_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future reference, it would be much better to change theTrackFilter_ and theTrackClusterizer_ to be std::unique_ptr rather than doing explicit deletes. Also, there is no need to test for nullptr before calling delete since, internally, delete does the same check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Dr15Jones, thanks for the comment. I'll keep that in mind for the next iteration.
As I got inspiration from,

if (theTrackFilter) delete theTrackFilter;
if (theTrackClusterizer) delete theTrackClusterizer;

I think same comment applies also there: maybe worth opening an issue?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

@mmusich mmusich deleted the 1000x_PVvalidationHotFix branch November 29, 2017 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants