New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leak in Simulation.unload() caused by collision pipeline #3318
Comments
Could this be caused by |
Thanks again @ScheiklP for these investigations ! Normally the reset function is not supposed to have any cleanup results as it is only there to set again (re-set) the component in a previous state. It is the "cleanup" that is called before removal/deletion of the component from the graph. |
Most of these components also do not have a I adapted the The code: #include <sofa/defaulttype/VecTypes.h>
#include <sofa/component/collision/detection/algorithm/BruteForceBroadPhase.h>
#include <sofa/component/collision/detection/algorithm/BVHNarrowPhase.h>
#include <sofa/component/collision/detection/algorithm/DefaultPipeline.h>
#include <sofa/component/collision/detection/intersection/NewProximityIntersection.h>
#include <sofa/component/collision/response/contact/DefaultContactManager.h>
#include <sofa/component/collision/geometry/PointModel.h>
#include <sofa/simulation/graph/DAGNode.h>
#include <sofa/simulation/graph/DAGSimulation.h>
#include <sofa/simulation/graph/init.h>
#include <sofa/simulation/DefaultAnimationLoop.h>
#include <sofa/component/statecontainer/MechanicalObject.h>
#include <sofa/core/objectmodel/Context.h>
#include <sofa/simulation/Node.h>
#include <sofa/simulation/Simulation.h>
#include <SofaComponentAll/initSofaComponentAll.h>
#include <sofa/helper/system/FileRepository.h>
#include <sofa/helper/logging/LoggingMessageHandler.h>
#include <sofa/core/logging/PerComponentLoggingMessageHandler.h>
#include <sofa/helper/BackTrace.h>
using namespace sofa::defaulttype;
using sofa::simulation::Node;
using sofa::simulation::graph::DAGNode;
using sofa::helper::system::DataRepository;
// collision pipeline
using sofa::component::collision::detection::algorithm::DefaultPipeline;
using sofa::component::collision::detection::algorithm::BruteForceBroadPhase;
using sofa::component::collision::detection::algorithm::BVHNarrowPhase;
using sofa::component::collision::detection::intersection::NewProximityIntersection;
using sofa::component::collision::response::contact::DefaultContactManager;
using sofa::component::collision::geometry::PointCollisionModel;
// mechanical object
using sofa::component::statecontainer::MechanicalObject;
using sofa::defaulttype::StdVectorTypes;
using sofa::type::Vec;
using sofa::core::behavior::MechanicalState;
using sofa::core::State;
using sofa::core::objectmodel::New;
using sofa::core::objectmodel::Data;
using sofa::simulation::DefaultAnimationLoop;
int main(int argc, char** argv)
{
sofa::helper::logging::MessageDispatcher::addHandler(&sofa::helper::logging::MainLoggingMessageHandler::getInstance());
sofa::helper::logging::MessageDispatcher::addHandler(&sofa::helper::logging::MainPerComponentLoggingMessageHandler::getInstance());
sofa::helper::logging::MainLoggingMessageHandler::getInstance().activate();
sofa::helper::BackTrace::autodump();
sofa::simulation::graph::init();
sofa::simulation::setSimulation(new sofa::simulation::graph::DAGSimulation());
for (int i = 0; i < 1000; i++) {
Node::SPtr groot = sofa::simulation::getSimulation()->createNewGraph("root");
groot->setGravity({ 0,0,0 });
groot->setDt(0.02);
DefaultAnimationLoop::SPtr animationLoop = New<DefaultAnimationLoop>();
groot->addObject(animationLoop);
// collision pipeline
DefaultPipeline::SPtr collisionPipeline = New<DefaultPipeline>();
collisionPipeline->setName("Collision Pipeline");
groot->addObject(collisionPipeline);
// collision detection system
BruteForceBroadPhase::SPtr broadPhaseDetection = New<BruteForceBroadPhase>();
broadPhaseDetection->setName("Broad Phase Collision Detection");
groot->addObject(broadPhaseDetection);
BVHNarrowPhase::SPtr narrowPhaseDetection = New<BVHNarrowPhase>();
narrowPhaseDetection->setName("Narrow Phase Collision Detection");
groot->addObject(narrowPhaseDetection);
// component to detection intersection
NewProximityIntersection::SPtr detectionProximity = New<NewProximityIntersection>();
detectionProximity->setName("Detection Proximity");
detectionProximity->setAlarmDistance(10.0);
detectionProximity->setContactDistance(0.5);
groot->addObject(detectionProximity);
// contact manager
DefaultContactManager::SPtr contactManager = New<DefaultContactManager>();
contactManager->setName("Contact Manager");
contactManager->setDefaultResponseType("PenalityContactForceField");
groot->addObject(contactManager);
// Node 1
Node::SPtr firstNode = New<DAGNode>();
firstNode->setName("node 1");
// Node 2
Node::SPtr secondNode = New<DAGNode>();
secondNode->setName("node 2");
// mechanical objects
typedef MechanicalObject< Vec3dTypes > MechanicalObject3d;
MechanicalObject3d::SPtr firstMechanicalObject = New<MechanicalObject3d>();
firstMechanicalObject->setTranslation(0,0,0);
firstMechanicalObject->setRotation(0,0,0);
firstMechanicalObject->setScale(1,1,1);
MechanicalObject3d::SPtr secondMechanicalObject = New<MechanicalObject3d>();
secondMechanicalObject->setTranslation(1,1,1);
secondMechanicalObject->setRotation(0,0,0);
secondMechanicalObject->setScale(1,1,1);
PointCollisionModel<sofa::defaulttype::Vec3Types>::SPtr firstPointCollisionModel = New<PointCollisionModel<sofa::defaulttype::Vec3Types>>();
PointCollisionModel<sofa::defaulttype::Vec3Types>::SPtr secondPointCollisionModel = New<PointCollisionModel<sofa::defaulttype::Vec3Types>>();
firstNode->addObject(firstMechanicalObject);
firstNode->addObject(firstPointCollisionModel);
groot->addChild(firstNode);
secondNode->addObject(secondMechanicalObject);
secondNode->addObject(secondPointCollisionModel);
groot->addChild(secondNode);
// Init the scene
sofa::simulation::getSimulation()->init(groot.get());
firstMechanicalObject->resize(50);
Data<MechanicalObject3d::VecCoord>& firstdpositions = *firstMechanicalObject->write(sofa::core::VecId::position());
MechanicalObject3d::VecCoord& firstpositions = *firstdpositions.beginEdit();
for (int i = 0; i < 50; i++) {
for (int j = 0; j < 3; j++){
firstpositions[i][j] = 0.0;
}
}
firstdpositions.endEdit();
secondMechanicalObject->resize(50);
Data<MechanicalObject3d::VecCoord>& seconddpositions = *secondMechanicalObject->write(sofa::core::VecId::position());
MechanicalObject3d::VecCoord& secondpositions = *seconddpositions.beginEdit();
for (int i = 0; i < 50; i++) {
for (int j = 0; j < 3; j++){
secondpositions[i][j] = 1.0;
}
}
seconddpositions.endEdit();
for (int j = 0; j < 50; j++) {
sofa::simulation::getSimulation()->animate(groot.get(), groot->getDt());
}
sofa::simulation::getSimulation()->unload(groot.get());
}
sofa::simulation::graph::cleanup();
return 0;
}
}
|
I echo @damienmarchal , thanks a lot @ScheiklP for these interesting investigations! your last comment means that it could come from the collisionmodel binding, right? |
I don't know this part of the code, but it is highly suspicious: virtual void addIntersectors(TIntersectionClass* object)
{
new TIntersectorClass(object);
}
|
Thanks @alxbilger ! :) I have a commit here, that puts the created pointers into a map, so that they get cleaned up, if they are created multiple times. What do you think? Should this still be a PR? |
Hi @hugtalbot , EDIT: |
I don't know enough this part of the code to answer you. For sure it sounds better, but I advise more investigations. |
Thanks again for the investigation. I see you are using the raw c++ code to make your scene. Instead of doing that the "hard way" you can use "simplapi" which mimick in c++ de python one. It is in sofa/simulation/graph/simpleapi.h and there are exemple in the code base. |
Does this not look suspicious to you guys? Line 99 in 471a3df
|
Indeed it does. :D |
alright ;) I quickly search for |
@ScheiklP Here is a branch with a simple object tracker: https://github.com/alxbilger/sofa/tree/objecttracker It counts the number of creation and destruction for each class managed with the The diff is here: https://github.com/sofa-framework/sofa/compare/master...alxbilger:sofa:objecttracker?expand=1 For example, there are a couple of components in caduceus where I see a difference between nb of allocations and destructions. |
@alxbilger Thanks!
So that looks ok, right? At least the difference is more or less constant. :D Could it be a vector/map that constantly receives new values but is not cleanup up with |
I don't think so because components are destroyed in the unload
…On Fri, Sep 23, 2022, 16:43 Paul Scheikl ***@***.***> wrote:
@alxbilger <https://github.com/alxbilger> Thanks!
Before unload
Total Base: 31970 31936 Diff: 34
After unload
Total Base: 31970 31939 Diff: 31
Before unload
Total Base: 32002 31968 Diff: 34
After unload
Total Base: 32002 31971 Diff: 31
Before cleanup
Total Base: 32002 32000 Diff: 2
Total Base: 32002 32000 Diff: 2
After cleanup
Total Base: 32002 32001 Diff: 1
So that looks ok, right? At least the difference is more or less constant.
:D
But the tracker only tracks actual objects, right? So Creators, Factories,
and others that do not inherit from base are not tracked.
Could it be a vector/map that constantly receives new values but is not
cleanup up with reset/cleanup?
Similar to what was happening in sofa-framework/SofaPython3#304
<sofa-framework/SofaPython3#304>
—
Reply to this email directly, view it on GitHub
<#3318 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACQVHUANK4QVMDDWVFQIBHTV7W6ZRANCNFSM6AAAAAAQR3UB3M>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@alxbilger not everything. Creators, factories etc are not destroyed during unload
|
I see what you mean. That's why I had confidence in https://github.com/alxbilger/sofa/blob/918cd66008f586575c92c1e068f5c267a952b936/Sofa/framework/Core/src/sofa/core/collision/IntersectorFactory.h#L95. At least now we know that there are no leak in the components themselves. The track continues |
With such serious contribution and bug tracking... sound like the return of valgrind in our workflow :) My contribution: ==91659== 5,152 (32 direct, 5,120 indirect) bytes in 1 blocks are definitely lost in loss record 12,668 of 12,789
==91659== at 0x4849013: operator new(unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==91659== by 0x3EA66C2D: createOutputVector<sofa::component::collision::geometry::PointCollisionModel<sofa::defaulttype::StdVectorTypes<sofa::type::Vec<3, double>, sofa::type::Vec<3, double>, double> >, sofa::component::collision::geometry::PointCollisionModel<sofa::defaulttype::StdVectorTypes<sofa::type::Vec<3, double>, sofa::type::Vec<3, double>, double> > > (Intersection.h:48)
==91659== by 0x3EA66C2D: beginIntersect (Intersection.inl:60)
==91659== by 0x3EA66C2D: sofa::core::collision::MemberElementIntersector<sofa::component::collision::geometry::TPoint<sofa::defaulttype::StdVectorTypes<sofa::type::Vec<3u, double>, sofa::type::Vec<3u, double>, double> >, sofa::component::collision::geometry::TPoint<sofa::defaulttype::StdVectorTypes<sofa::type::Vec<3u, double>, sofa::type::Vec<3u, double>, double> >, sofa::component::collision::detection::intersection::LocalMinDistance>::beginIntersect(sofa::core::CollisionModel*, sofa::core::CollisionModel*, sofa::core::collision::DetectionOutputVector*&) (Intersection.inl:54)
==91659== by 0x3E97FA7E: sofa::component::collision::detection::algorithm::BVHNarrowPhase::addCollisionPair(std::pair<sofa::core::CollisionModel*, sofa::core::CollisionModel*> const&) (BVHNarrowPhase.cpp:73)
==91659== by 0x53714C0: sofa::core::collision::NarrowPhaseDetection::addCollisionPairs(sofa::type::vector<std::pair<sofa::core::CollisionModel*, sofa::core::CollisionModel*>, sofa::type::CPUMemoryManager<std::pair<sofa::core::CollisionModel*, sofa::core::CollisionModel*> > > const&) (NarrowPhaseDetection.cpp:41)
==91659== by 0x3E99415F: sofa::component::collision::detection::algorithm::DefaultPipeline::doCollisionDetection(sofa::type::vector<sofa::core::CollisionModel*, sofa::type::CPUMemoryManager<sofa::core::CollisionModel*> > const&) (DefaultPipeline.cpp:238)
==91659== by 0x4BBD0D6: sofa::simulation::PipelineImpl::computeCollisionDetection() (PipelineImpl.cpp:117)
==91659== by 0x4B68E62: sofa::simulation::CollisionVisitor::processCollisionPipeline(sofa::simulation::Node*, sofa::core::collision::Pipeline*) (CollisionVisitor.cpp:78)
==91659== by 0x4B690BC: runVisitorTask<sofa::simulation::CollisionVisitor, sofa::simulation::Node, sofa::core::co
Sounds like createOutputModel is a good first spot to look at as it allocate an output vector with new... and that this one is returned to finally be stored in a map (eg: line 50 in NarrowPhaseDection).... which is not deleted when destructing object. |
The 5mb indirectly lost also match the plot quite nicely I'll have a look at that next week. :) |
I suggest using the memory sensitization tools provided by GCC or Clang - it is less hassle than valgrind. It also provides more detail regarding memory leaks. To do this, build Sofa with the additional flag: Here is the output when running the example (CPP) scene provided by @ScheiklP. The 15 object(s) that are detected are come from the number of unloads performed (I decreased it to 15 from the original 1000 used in the scene by @ScheiklP)
|
The intended behavior of this is "if the vector is empty, or it's a nullpointer, remove it from the map. If it is not a nullpointer, also release the vector", right? Since the |
It was indeed the |
good work @ScheiklP . Just for the records, you can directly put the keyword: "fix #PRnumber" inside the comment of your PR. Thus when it is merged, the issue is automatically closed with a link to the PR. |
Problem
Hi,
I discovered a memory leak that seems to be connected to the collision detection pipeline / models.
The scene contains 2 nodes with
MechanicalObject
andPointCollisionModel
.I repeatedly load and unload the scene, first with no collision models, then one collision model, and then two collision models.
-> each graph has 3 segments.
No components of the collision pipeline:
-> No memory increase, even with two collision models
NewProximity
-> Some memory increase with one and no collision models, STRONG increase with two collision models
LocalMinDistance
-> Some memory increase with one and no collision models, STRONG increase with two collision models
As you can see, there is a memory increase with no collision models and one collision model,
but with two collision models, the increase is much stronger. This is why I suspect the leak is somewhere in the pipeline, not the
collision models themselves.
Environment
Context
mprof run --python python load_unload_env.py
mprof plot ...
I guess @damienmarchal and @hugtalbot would be the right people to annoy with another leak. :D
Cheers,
Paul
The text was updated successfully, but these errors were encountered: