-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mu fcs hit clear #301
Mu fcs hit clear #301
Conversation
update from upstream on Friday, June 11, 2021
Updated StarGeo.xml to be able to acces this in dev2021
…er, heatsinks (draft) )
… rotation of middle disk
This includes material changes
…in PEEK bases, removed unnecessary code
…plified cooling tube connectors)
… add a Clear on StMuFcsHit to clean up memory - may effect all TClonesArrays
@@ -28,6 +28,11 @@ class StMuFcsHit : public TObject { | |||
unsigned short ns, unsigned short ehp, unsigned short dep, unsigned short ch, | |||
float e); | |||
~StMuFcsHit(); | |||
|
|||
virtual void Clear (Option_t * opt=""){ | |||
TObject::Clear(opt); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TObject::Clear() seems to be empty... At least
https://root.cern.ch/doc/master/classTObject.html#a0ccdc7dd6ee02ee79c46bfab0e1be291
shows only links to reimplementations. I don't know what this line do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point, it is not needed. I will remove
@@ -242,7 +242,7 @@ void StMuDstMaker::clearArrays() | |||
__NMTDARRAYS__+__NFGTARRAYS__; | |||
|
|||
for ( int i=0; i<ezIndex; i++) { | |||
mAArrays[i]->Clear(); | |||
mAArrays[i]->Clear("C"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change affect all classes in MuDst Arrays. I guess we can try..
Test job to see the memory leak (was showing 273,940 bytes lost in 5 events from StMuFcsHit - biggest and last item in the valgrind report):
From what I see, this patch does resolve the memory leak issue. I do not know whether the "C" argument in -Gene |
valgrind also reported another similar, but smaller memory leak. It appears that
This form of I guess you could create a
I've been using a run for my tests that doesn't have many detectors in it, so I may be missing other collections with the same issue. I'll try processing a typical production run and see what it shows. -Gene |
Just looking for other TArray data members of MuDst classes, I see there's one called
-Gene |
This is exactly why I think adding the "C" option should be added - I always thought clearing the contained objects was the default. So the code you show here was maybe never even called since we did not use "C" option previously. |
Other leaks when not calling destructors found from running valgrind on data with other detectors:
Next up is -Gene |
Can you send me your valgrind report file? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
StMuFcsHit.h line 47. mData is deferenced w/out checking to see that it is not a nullptr.
It would be cleaner for StMuFcsHit (and StEvent/StFcsHit) to hold an instance of TArrayS, rather than constantly having to call destruct on TArrayS during "Clear".
I agree. However, Akio and I were unsure about this (maybe it is causing the other issue with MuDst) - i.e. can TClonesArray hold an object of unknown size? |
Hi,
On Feb 16, 2022, at 1:35 PM, klendathu2k ***@***.******@***.***>> wrote:
You are right. Data members in a TClonesArray need to be of fixed size.
TArray may violate this.
I don't think TArray violates this because its data members are fixed. However, one of its members is the fArray pointer to a separate memory block for the array. Keeping a handle on that array is the ultimate concern here.
Daniel, valgrind log is here:
~genevb/public/log.valgrind2022
…-Gene
|
OK commit 2a7e328 has updates for StMuEvent (and StEventSummary) and StMuETofHeader. I didnt run a full valgrind but I did test to make sure the ::Clear(...) was being called. I also changed StMuFcsHit to use a TArrayS instead of a TArrayS* |
Okey. I see that TArray does not violate TConesArray... I guess its same for TRefArray in StMuFcsCuster and this is nothing to do with StMuFcsCluster reading crash... |
I ran valgrind and now it only shows MuDst leaks with the STL vectors in StMuETofHeader, which is surprising given that you do clear the vectors, and I confirmed that the Clear() is called, though for some reason it only appeared to be called 4 times in 5 events, so perhaps just one event gets lost? Anyhow, this is really small and won't add up to more than a few MB over 50k events even. I can try to track it down later this evening, but I'm fine with moving forward as it is now. Thanks for the work, @jdbrice . -Gene |
I got it.... STL vectors have both a capacity (actual memory allocation) and a size (in-use portion of the allocation). While clear() sets the size to zero, capacity remains at the discretion of the STL implementation. The best bet for reducing the capacity to zero is to call clear(), and then call shrink_to_fit(), which brings brings the capacity down to fit the size (though my impression is that this isn't guaranteed, it has the highest probability of achieving the effect; there is an erase() function, but it too may or may not actually reduce the capacity). Of course deleting the vector guarantees that the allocation is freed, but that's not an option here. Anyhow, I printed the capacities of the two vectors before and after the clear() and indeed they are unchanged. I then tried shrink_to_fit() and the capacity goes to zero! Yay! I am running in valgrind now and I am quite optimistic this memory leak will be gone. As for STL maps, there doesn't seem to be this size vs. capacity business, and clear() does the appropriate job. And valgrind shows no leak from them. So in the end, it appears that what we want is:
-Gene |
Well, valgrind still reported on the two vectors in StMuETofHeader as memory leaks, but says of them:
I'll take memory leaks of size 0 bytes for the win any day :-) -Gene |
allocation to zero
Agreed :) |
Freeing the memory and allocating again for every event is potentially a more expensive operation than reusing the same chunk of memory. This is probably a reason why we have zero calls to |
On Feb 18, 2022, at 10:47 AM, Dmitri Smirnov ***@***.******@***.***>> wrote:Freeing the memory and allocating again for every event is potentially a more expensive operation than reusing the same chunk of memory. This is probably a reason why we have zero calls to std::vector::shrink_to_fit() in our code.
Agreed, Dmitri, which is why STL is designed to do all that memory management behind the scenes. A better approach would probably be to reuse the same StMuETofHeader instances (not use "new StMuETofHeader()", but instead use a "fill" function of some sort, only doing "new" to extend the allocation within the TClonesArray if necessary), but that'll take a little more careful work to do the bookkeeping. In the existing approach to TClonesArrays in the MuDst, the simplest and cleanest way forward is to free and re-allocate.
Thanks,
-Gene
|
@ullrich-bnl Can you comment or approve these changes |
@ullrich-bnl? please? |
Attempt to modify TClonesArray cleanup to prevent memory leak on StMuFcsHit