double free or corruption (out) in teb_local_planner when -mf16c optimization flag is used

[VRichardJP]

Hi

I have encountered a double free or corruption (out) while using teb_local_planner. The error has been reported on the teb_local_planner github page here, but it seems the error might be caused by g2o library. The problem arises when -mf16c flag is used (which is brought by -march=native). When using clang the crash occurs immediately on startup (with gcc the bug is still there but more difficult to trigger).

Here I reported the valgrind dump below (crash on program startup with clang++ and -mf16c flag). The crash seems not to be related to teb_local_planner but looks more like a g2o or Eigen library issue:

==4330== Memcheck, a memory error detector
==4330== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==4330== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==4330== Command: /home/dev/workspace/install/teb_local_planner/lib/teb_local_planner/test_optim_node __name:=test_optim_node __log:=/home/dev/.ros/log/840700a0-5743-11ec-9eb2-0242ac110002/test_optim_node-1.log
==4330== 
process[rviz-2]: started with pid [4353]
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-dev'
==4330== Invalid free() / delete / delete[] / realloc()
==4330==    at 0x4C32D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4330==    by 0x6ECE6FC: g2o::EdgeSE2PointXYCalib::~EdgeSE2PointXYCalib() (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6EC710A: ??? (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x40108D2: call_init (dl-init.c:72)
==4330==    by 0x40108D2: _dl_init (dl-init.c:119)
==4330==    by 0x40010C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==4330==    by 0x2: ???
==4330==    by 0x1FFEFFFE3E: ???
==4330==    by 0x1FFEFFFE94: ???
==4330==    by 0x1FFEFFFEAC: ???
==4330==  Address 0x304ac7c0 is 16 bytes inside a block of size 128 alloc'd
==4330==    at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4330==    by 0x4EE2FA9: handmade_aligned_malloc (Memory.h:88)
==4330==    by 0x4EE2FA9: aligned_malloc (Memory.h:164)
==4330==    by 0x4EE2FA9: Eigen::aligned_allocator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > >::allocate(unsigned long, void const*) (Memory.h:742)
==4330==    by 0x4EE1777: allocate (alloc_traits.h:301)
==4330==    by 0x4EE1777: _M_allocate (stl_vector.h:172)
==4330==    by 0x4EE1777: std::vector<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::aligned_allocator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > > >::_M_fill_insert(__gnu_cxx::__normal_iterator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >*, std::vector<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::aligned_allocator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > > > >, unsigned long, Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const&) (vector.tcc:505)
==4330==    by 0x6ECFB54: g2o::BaseMultiEdge<2, Eigen::Matrix<double, 2, 1, 0, 2, 1> >::resize(unsigned long) (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6ECE21B: g2o::EdgeSE2PointXYCalib::EdgeSE2PointXYCalib() (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6ED5A9F: g2o::HyperGraphElementCreator<g2o::EdgeSE2PointXYCalib>::construct() (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6A6B36E: g2o::Factory::registerType(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, g2o::AbstractHyperGraphElementCreator*) (in /opt/ros/melodic/lib/libg2o_core.so)
==4330==    by 0x6EC710A: ??? (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x40108D2: call_init (dl-init.c:72)
==4330==    by 0x40108D2: _dl_init (dl-init.c:119)
==4330==    by 0x40010C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==4330==    by 0x2: ???
==4330==    by 0x1FFEFFFE3E: ???
==4330== 
==4330== Invalid free() / delete / delete[] / realloc()
==4330==    at 0x4C32D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4330==    by 0x6EE723C: g2o::EdgeSE2TwoPointsXY::~EdgeSE2TwoPointsXY() (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6EC745E: ??? (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x40108D2: call_init (dl-init.c:72)
==4330==    by 0x40108D2: _dl_init (dl-init.c:119)
==4330==    by 0x40010C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==4330==    by 0x2: ???
==4330==    by 0x1FFEFFFE3E: ???
==4330==    by 0x1FFEFFFE94: ???
==4330==    by 0x1FFEFFFEAC: ???
==4330==  Address 0x304ae280 is 16 bytes inside a block of size 128 alloc'd
==4330==    at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4330==    by 0x4EE2FA9: handmade_aligned_malloc (Memory.h:88)
==4330==    by 0x4EE2FA9: aligned_malloc (Memory.h:164)
==4330==    by 0x4EE2FA9: Eigen::aligned_allocator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > >::allocate(unsigned long, void const*) (Memory.h:742)
==4330==    by 0x4EE1777: allocate (alloc_traits.h:301)
==4330==    by 0x4EE1777: _M_allocate (stl_vector.h:172)
==4330==    by 0x4EE1777: std::vector<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::aligned_allocator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > > >::_M_fill_insert(__gnu_cxx::__normal_iterator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >*, std::vector<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> >, Eigen::aligned_allocator<Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > > > >, unsigned long, Eigen::Map<Eigen::Matrix<double, -1, -1, 0, -1, -1>, 0, Eigen::Stride<0, 0> > const&) (vector.tcc:505)
==4330==    by 0x6EE7954: g2o::BaseMultiEdge<4, Eigen::Matrix<double, 4, 1, 0, 4, 1> >::resize(unsigned long) (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6EE6D8B: g2o::EdgeSE2TwoPointsXY::EdgeSE2TwoPointsXY() (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6ED56AF: g2o::HyperGraphElementCreator<g2o::EdgeSE2TwoPointsXY>::construct() (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x6A6B36E: g2o::Factory::registerType(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, g2o::AbstractHyperGraphElementCreator*) (in /opt/ros/melodic/lib/libg2o_core.so)
==4330==    by 0x6EC745E: ??? (in /opt/ros/melodic/lib/libg2o_types_slam2d.so)
==4330==    by 0x40108D2: call_init (dl-init.c:72)
==4330==    by 0x40108D2: _dl_init (dl-init.c:119)
==4330==    by 0x40010C9: ??? (in /lib/x86_64-linux-gnu/ld-2.27.so)
==4330==    by 0x2: ???
==4330==    by 0x1FFEFFFE3E: ???
==4330==

I am not familiar with the g2o internal implementation. Does this rings any bell to you?

[wangzheqie]

这是来自QQ邮箱的自动回复邮件。你好，我已经收到你的邮件，尽快处理。

[VRichardJP]

When Debug is used instead of RelWithDebInfo the program is stopped by some Eigen data alignment assert (still clang, still -mf16c). This might be the root cause of the double free in release mode:

test_optim_node: /usr/include/eigen3/Eigen/src/Core/MapBase.h:191: void Eigen::MapBase<Derived, 0>::checkSanity(typename Eigen::internal::enable_if<(Eigen::internal::traits<OtherDerived>::Alignment > 0), void*>::type) const [with T = Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> >; Derived = Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> >; typename Eigen::internal::enable_if<(Eigen::internal::traits<OtherDerived>::Alignment > 0), void*>::type = void*]: Assertion `( ((internal::UIntPtr(m_data) % internal::traits<Derived>::Alignment) == 0) || (cols() * rows() * innerStride() * sizeof(Scalar)) < internal::traits<Derived>::Alignment ) && "data is not aligned"' failed.
==29581== 
==29581== Process terminating with default action of signal 6 (SIGABRT): dumping core
==29581==    at 0x760AFB7: raise (raise.c:51)
==29581==    by 0x760C920: abort (abort.c:79)
==29581==    by 0x75FC489: __assert_fail_base (assert.c:92)
==29581==    by 0x75FC501: __assert_fail (assert.c:101)
==29581==    by 0x54B3253: void Eigen::MapBase<Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> >, 0>::checkSanity<Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> > >(Eigen::internal::enable_if<(Eigen::internal::traits<Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> > >::Alignment>(0)), void*>::type) const (MapBase.h:190)
==29581==    by 0x54AFDA1: Eigen::MapBase<Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> >, 0>::MapBase(double*, long, long) (MapBase.h:176)
==29581==    by 0x54AB20B: Eigen::MapBase<Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> >, 1>::MapBase(double*, long, long) (MapBase.h:281)
==29581==    by 0x54A62EE: Eigen::Map<Eigen::Matrix<double, 2, 3, 0, 2, 3>, 32, Eigen::Stride<0, 0> >::Map(double*, long, long, Eigen::Stride<0, 0> const&) (Map.h:150)
==29581==    by 0x54CB221: g2o::BaseBinaryEdge<2, double, teb_local_planner::VertexPose, teb_local_planner::VertexPose>::linearizeOplus(g2o::JacobianWorkspace&) (base_binary_edge.hpp:161)
==29581==    by 0x54D1522: g2o::BlockSolver<g2o::BlockSolverTraits<-1, -1> >::buildSystem() (block_solver.hpp:492)
==29581==    by 0x7E31D58: g2o::OptimizationAlgorithmLevenberg::solve(int, bool) (in /opt/ros/melodic/lib/libg2o_core.so)
==29581==    by 0x7E2A1DC: g2o::SparseOptimizer::optimize(int, bool) (in /opt/ros/melodic/lib/libg2o_core.so)
==29581== 
==29581== HEAP SUMMARY:
==29581==     in use at exit: 9,540,124 bytes in 11,852 blocks
==29581==   total heap usage: 50,961 allocs, 39,109 frees, 12,144,750 bytes allocated
==29581== 
==29581== LEAK SUMMARY:
==29581==    definitely lost: 127 bytes in 1 blocks
==29581==    indirectly lost: 0 bytes in 0 blocks
==29581==      possibly lost: 588,583 bytes in 5,524 blocks
==29581==    still reachable: 8,951,414 bytes in 6,327 blocks
==29581==         suppressed: 0 bytes in 0 blocks
==29581== Rerun with --leak-check=full to see details of leaked memory
==29581== 
==29581== For counts of detected and suppressed errors, rerun with: -v
==29581== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
[test_optim_node-2] process has died [pid 29581, exit code -6, cmd valgrind /home/dev/workspace/install/teb_local_planner/lib/teb_local_planner/test_optim_node __name:=test_optim_node __log:=/home/dev/.ros/log/f8b79c8a-57c9-11ec-ad25-0242ac110002/test_optim_node-2.log].
log file: /home/dev/.ros/log/f8b79c8a-57c9-11ec-ad25-0242ac110002/test_optim_node-2*.log

I don't know if it is a Eigen issue or g2o...

[RainerKuemmerle]

Do you have the same flags on g2o and TEB?

[VRichardJP]

I am compiling TEB from source with all compilation flags set at project level, so I think it does apply to Eigen, g2o and other libraries as well. The g2o library provided by the ros-melodic-libg2o package.

[RainerKuemmerle]

The flags then apply to the headers of g2o but not to the libraries/binaries of g2o from the ROS package. There have been issues reported in the past caused by different compilation flags.

…

Am 20.12.2021 um 00:57 schrieb Vincent Richard ***@***.***>:  I am compiling TEB from source with all compilation flags set at project level, so I think it does apply to Eigen, g2o and other libraries as well. The g2o library provided by the ros-melodic-libg2o package. — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.

[VRichardJP]

I have tried to uninstall ros-melodic-libg2o and compile g2o from source as well using the same env CXXFLAGS='-O3 -march=native' for both TEB and g2o. I still get the double free or corruption (out) error