Question : Why Fast-DDS depending on specific commit of foonathan-memory instead of release? [13682]

[dotChris90]

Hello together,

it's not an urgent question but just for curiosity. I saw that Fast-DDS depends on foonathan-memory - moreover it depends not on a release of foonathan-memory but on a specific commit id ^^'

Is there a reason for that? i ask that question from Conan Center side because of the following issue - conan-io/conan-center-index#6731. because conan as package manager should reference proper released versions of a lib - we decided to reference foonathan-memory 0.7.0 directly. here we need to add the build option FOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE = OFF then examples .

then it works. :)

Since the conan package currently using 0.7.0 foonathan-memory and official build 0.6.2 we have a gap and i currently searching what are differences and if fast-dds is planning to upgrade from 0.6.2 to 0.7.0? plus - do you guys also have to set FOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE = OFF?

[tparys]

We've been building Fast-DDS against foonathan-memory 0.7 for a while, and we don't have this issue.

If memory serves, foonathan-memory has a memory debugger tool "nodesize_dbg" which to be needs to run and generate src/container_node_sizes_impl.hpp. If FOONATHAN_MEMORY_BUILD_TOOLS is not switched on, this header is not generated, and incorrect memory allocations result.

You might want to turn it off after the library is built, else will become an unnecessary dependency when using the library.

It's also a pain for cross compilation, as running this debug application on a foreign architecture requires some trickery. But that's probably more of a bug report for foonathan instead of eProsima.

[EduPonz]

Hi @tparys ,

We are actually planning on taking it a bit further than 0.7.0 (eProsima/foonathan_memory_vendor#49), specially cause we also considered all the things you listed above. We just haven't got to it yet (vacations and such), but we will in the upcoming weeks!

[foonathan]

It's also a pain for cross compilation, as running this debug application on a foreign architecture requires some trickery. But that's probably more of a bug report for foonathan instead of eProsima.

Yeah, I don't like the solution either. However, there is no other way to get the container node size information. If it's too annoying, you can completely disable their generation and use conservative constants instead (or maybe you don't need them at all?).

[dotChris90]

Thanks for pointing this all out. to be honest i just asking this all because i am huge fan of packages. :) and dependency management. Fast-DDS is the best DDS implementation i found so far. ;) That's why want to bring it to Conan Center.

Just one small question into direction of our eProsima engineers ;) with what cmake build options do you build foonathan/memory? Reason : i am pretty sure when you using the colcon build the default options are overwritten like :

FOONATHAN_MEMORY_MEMORY_RESOURCE:STRING=foonathan_comp::memory_resource
FOONATHAN_MEMORY_MEMORY_RESOURCE_HEADER:STRING=<foonathan/pmr.hpp>

One question also to @foonathan - this means : if you want to cross compile (and i think DDS is a topic especially for embedded devices so yes cross compiling) you always have to disable the sizecheck? I currently analyze from conan perspective what default options etc. would be required to have easy to compile normal build examples and also cross compile examples.

[foonathan]

if you want to cross compile (and i think DDS is a topic especially for embedded devices so yes cross compiling) you always have to disable the sizecheck?

No, you just need to ensure that the container node sizes are computed correctly (or not use the facility at all). I've originally disabled them completely when cross compiling to prevent exactly those errors, but people have submitted various solutions for different targets that work, which I cannot verify.

[MiguelCompany]

@dotChris90

it depends not on a release of foonathan-memory but on a specific commit id

This is not exactly true. If you build Fast DDS on a system that has foonathan-memory already installed, that version will be used. The specific commit id is the one we use on our CI via foonathan_memory_vendor package. This is to comply with this quality declaration

@foonathan Doing a v0.7.1 release would be nice.

[foonathan]

I can do a new release at the end of the week.

[foonathan]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

[dotChris90]

@MiguelCompany yes you are totally right. ;) if the system has already this library present it wont be linked against different version. But for embedded areas with cross compilation requirements .... it is not that easy. and i actually think Fast-DDS is "the" best implementation of DDS especially for embedded areas like embedded linux and so.

@EduPonz small question - the current master is already using this latest foonathan version? if so i would try out the conanfiles and next version i will bring into artifactory with this new reference on latest foonathan version.

@foonathan great work! ;)

[large-cat]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

Hi:
I am cross compiling using v0.7-1. and i encountered a warning in cmake:

CMake Warning at src/CMakeLists.txt:108 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

And I encountered the same error as above when running the Fast-DDS:

[foonathan::memory] Allocator foonathan::memory::memory_pool (at 0x2b8296d8) received invalid size/alignment 56, max supported is 48
terminate called after throwing an instance of 'foonathan::memory::bad_node_size'
  what():  allocation node size exceeds supported maximum of allocator
Aborted

So, what is this emulator? Should I define it myself? If I need to define it myself, how should I define it?
thank you。

[EduPonz]

@EduPonz small question - the current master is already using this latest foonathan version? if so i would try out the conanfiles and next version i will bring into artifactory with this new reference on latest foonathan version.

Not yet. We have eProsima/foonathan_memory_vendor#49 to use the latest master (the new tag foonathan just created), but we still want the PR to be verified by the ROS 2 guys, so it'll most probably take a couple of days until we can merge it in.

[dotChris90]

@EduPonz thanks for the update - not an issue! :)

[dotChris90]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

Hi:
I am cross compiling using v0.7-1. and i encountered a warning in cmake:

CMake Warning at src/CMakeLists.txt:108 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

And I encountered the same error as above when running the Fast-DDS:

[foonathan::memory] Allocator foonathan::memory::memory_pool (at 0x2b8296d8) received invalid size/alignment 56, max supported is 48
terminate called after throwing an instance of 'foonathan::memory::bad_node_size'
  what():  allocation node size exceeds supported maximum of allocator
Aborted

So, what is this emulator? Should I define it myself? If I need to define it myself, how should I define it?
thank you。

Did you try it with cmake -DFOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE=OFF ..?

[MiguelCompany]

@large-cat

So, what is this emulator? Should I define it myself? If I need to define it myself, how should I define it?

Some details here

[foonathan]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

Hi:
I am cross compiling using v0.7-1. and i encountered a warning in cmake:

CMake Warning at src/CMakeLists.txt:108 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

And I encountered the same error as above when running the Fast-DDS:

[foonathan::memory] Allocator foonathan::memory::memory_pool (at 0x2b8296d8) received invalid size/alignment 56, max supported is 48
terminate called after throwing an instance of 'foonathan::memory::bad_node_size'
  what():  allocation node size exceeds supported maximum of allocator
Aborted

This is maybe more a question towards @MiguelCompany, but why does it work without node sizes? If node sizes are unavailable it just shouldn't compile, instead of giving wrong sizes. How are you constructing the memory_pool in this instance? @large-cat can you show the allocation call stack and where the memory_pool at 0x2b... is constructed?

Did you try it with cmake -DFOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE=OFF ..?

This just disables the check! It still tries to allocate a node with size 56 even though each node only has size 48.

[large-cat]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

Hi:
I am cross compiling using v0.7-1. and i encountered a warning in cmake:

CMake Warning at src/CMakeLists.txt:108 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

And I encountered the same error as above when running the Fast-DDS:

[foonathan::memory] Allocator foonathan::memory::memory_pool (at 0x2b8296d8) received invalid size/alignment 56, max supported is 48
terminate called after throwing an instance of 'foonathan::memory::bad_node_size'
  what():  allocation node size exceeds supported maximum of allocator
Aborted

So, what is this emulator? Should I define it myself? If I need to define it myself, how should I define it?
thank you。

Did you try it with cmake -DFOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE=OFF ..?

I use cmake -DFOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE=OFF now, and fast-DDS can work normally. But are there any other risks?

[large-cat]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

Hi:
I am cross compiling using v0.7-1. and i encountered a warning in cmake:

CMake Warning at src/CMakeLists.txt:108 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

And I encountered the same error as above when running the Fast-DDS:

[foonathan::memory] Allocator foonathan::memory::memory_pool (at 0x2b8296d8) received invalid size/alignment 56, max supported is 48
terminate called after throwing an instance of 'foonathan::memory::bad_node_size'
  what():  allocation node size exceeds supported maximum of allocator
Aborted

This is maybe more a question towards @MiguelCompany, but why does it work without node sizes? If node sizes are unavailable it just shouldn't compile, instead of giving wrong sizes. How are you constructing the memory_pool in this instance? @large-cat can you show the allocation call stack and where the memory_pool at 0x2b... is constructed?

Did you try it with cmake -DFOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE=OFF ..?

This just disables the check! It still tries to allocate a node with size 56 even though each node only has size 48.

I reported this wring when compiling fast-DDS(v22.3.4). And I can compile it。

[large-cat]

New release: https://github.com/foonathan/memory/releases/tag/v0.7-1

Hi:
I am cross compiling using v0.7-1. and i encountered a warning in cmake:

CMake Warning at src/CMakeLists.txt:108 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable

And I encountered the same error as above when running the Fast-DDS:

[foonathan::memory] Allocator foonathan::memory::memory_pool (at 0x2b8296d8) received invalid size/alignment 56, max supported is 48
terminate called after throwing an instance of 'foonathan::memory::bad_node_size'
  what():  allocation node size exceeds supported maximum of allocator
Aborted

This is maybe more a question towards @MiguelCompany, but why does it work without node sizes? If node sizes are unavailable it just shouldn't compile, instead of giving wrong sizes. How are you constructing the memory_pool in this instance? @large-cat can you show the allocation call stack and where the memory_pool at 0x2b... is constructed?

Did you try it with cmake -DFOONATHAN_MEMORY_CHECK_ALLOCATION_SIZE=OFF ..?

This just disables the check! It still tries to allocate a node with size 56 even though each node only has size 48.

Hi
My problem has been solved by the way @MiguelCompany said . I pointed the cmake variable CMAKE_CROSSCOMPILING_EMULATOR to my qemu-aarch64 path and it work now.

[large-cat]

@large-cat

So, what is this emulator? Should I define it myself? If I need to define it myself, how should I define it?

Some details here

Hi,
I solved foonathan_memory's problem in your way. But there was a strange phenomenon when I used Fast-DDS v2.3.4, and it will not happen when I use v2.3.3. When I use the HelloWorldExample in Fast-DDSv2.3.4 to test. If I repeatedly close and start publisher (or subscriber) during the test, subscriber (or publisher) will have a segment fault. But it will not happen in v2.3.3, and i use Fast-DDSv2.3.3 in my program now.
So finally, i hope this information can help you improve the Fast-DDS.
Thanks for your help.

[MiguelCompany]

@large-cat Would you mind checking with the head of branch 2.3.x? There are some bugfixes there that could help solve your issue.

[MiguelCompany]

why does it work without node sizes?

@foonathan I missed this question! Sorry to answer so late!

The code for finding the node size starts here and then goes into the impl folder. You could follow the lead from there.

Basically, when FOONATHAN_MEMORY_NO_NODE_SIZE is defined, the size of a custom node type will be used. If not, the ones on this folder are used.

We have plans to extend this mechanism, so that C++17 node_type is used when available.

In the mean time, if the user has knowledge on the implementation of the std library, could use the mechanism described here

[foonathan]

Ah, this could explain things. Are you sure that your approximation is not buggy?

I'm getting the occasional issue about invalid node sizes, because people set up their cross compiling incorrectly, apparently used your implementation, and crash. E.g. foonathan/memory#103 (comment)

It seems like the node size for std::map<GUID_t, uint16_t> on AArch 64 is wrong, but I can't do cross-compiling to reproduce it.

[MiguelCompany]

@foonathan
I've been playing around with this here, and I think I have finally found the bug. It seems to only be a problem with libstdc++.

Due to alignment issues, the code here is wrong, and the mentioned enumeration should be included.

[MiguelCompany]

@dotChris90 @tparys @large-cat @foonathan
I just created PR #2229, which should allow running Fast DDS when the node_size_dbg tool cannot be run.

[dotChris90]

@MiguelCompany @foonathan thanks you two for investigating. Had some private issues to fix last weeks. But after PR is merged will look if i can patch the two packages in conan center since i think cross compilation is a common thing in embedded area so guess everybody will appreciate. Once again. Thank to you two and have a good week ✌️😊

[MiguelCompany]

@dotChris90 @tparys @large-cat Both #2229 (and backports), as well as eProsima/foonathan_memory_vendor#49 have been merged. Could you check?

[JLBuenoLopez]

I am going to proceed and close this issue. It has already been solved.