Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large message pub sub revision #51

Merged
merged 5 commits into from
Aug 27, 2016
Merged

Conversation

wjwwood
Copy link
Member

@wjwwood wjwwood commented Jul 8, 2016

Replacement for #47 and closes #36.

I tested this and it seems to work as expected, with the caveats about OS X mentioned in #36 (comment).

This requires my FastCDR branch (eProsima/Fast-CDR#7) and the feature branch of Fast-RTPS (https://github.com/eProsima/Fast-RTPS/tree/feature/dynamicmemory_revision) for which there is no pull request yet.

@richiprosima richiprosima added the in progress Actively being worked on (Kanban column) label Jul 8, 2016
@richiprosima
Copy link
Contributor

I've updated Fast-RTPS/dynamicmemory_revision. Tests and examples return to compile successfully on Linux. Pending to check on Windows.
I started a PR on Fast-RTPS(Fast-RTPS#59)
Also I've updated this PR to new Fast-RTPS changes.

I've tested image_tools from ros2 (sending the moving hamburguers) and it is working on my Ubuntu. We will do more testing.

Locator_t multicast_locator;
multicast_locator.set_IP4_address("239.255.0.1");
multicast_locator.port = 7600;
subscriberParam.multicastLocatorList.push_back(multicast_locator);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might be causing a problem on my Linux VM now (but strangely not before):

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
  what():  set_option: No such device

Thread 1 "cam2image__rmw_" received signal SIGABRT, Aborted.
0x00007ffff410a418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff410a418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff410c01a in __GI_abort () at abort.c:89
#2  0x00007ffff474384d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff47416b6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff4741701 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff4741919 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff33fa442 in boost::throw_exception<boost::system::system_error> (e=...) at /usr/include/boost/throw_exception.hpp:69
#7  0x00007ffff33f76c2 in boost::asio::detail::do_throw_error (err=..., location=0x7ffff34f9056 "set_option")
    at /usr/include/boost/asio/detail/impl/throw_error.ipp:38
#8  0x00007ffff33f756b in boost::asio::detail::throw_error (err=..., location=0x7ffff34f9056 "set_option")
    at /usr/include/boost/asio/detail/throw_error.hpp:42
#9  0x00007ffff3489626 in boost::asio::basic_socket<boost::asio::ip::udp, boost::asio::datagram_socket_service<boost::asio::ip::udp> >::set_option<boost::asio::ip::detail::socket_option::multicast_request<0, 35, 41, 20> > (this=0x9ecf38, option=...)
    at /usr/include/boost/asio/basic_socket.hpp:820
#10 0x00007ffff348171d in eprosima::fastrtps::rtps::UDPv4Transport::OpenInputChannel (this=0xa02890, locator=...)
    at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/transport/UDPv4Transport.cpp:122
#11 0x00007ffff345d8fe in eprosima::fastrtps::rtps::ReceiverResource::ReceiverResource (this=0x7fffffff9550, transport=..., 
    locator=...) at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/rtps/network/ReceiverResource.cpp:26
#12 0x00007ffff345847f in eprosima::fastrtps::rtps::NetworkFactory::BuildReceiverResources (this=0xa02810, local=...)
    at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/rtps/network/NetworkFactory.cpp:70
#13 0x00007ffff3461ec7 in eprosima::fastrtps::rtps::RTPSParticipantImpl::createReceiverResources (this=0xa02620, Locator_list=..., 
    ApplyMutation=true) at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:635
#14 0x00007ffff345fad8 in eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl (this=0xa02620, PParam=..., guidP=..., 
    par=0x67c2c0, plisten=0xa01aa8)
    at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:180
#15 0x00007ffff346956f in eprosima::fastrtps::rtps::RTPSDomain::createParticipant (PParam=..., listen=0xa01aa8)
    at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/rtps/RTPSDomain.cpp:145
#16 0x00007ffff346b654 in eprosima::fastrtps::Domain::createParticipant (att=..., listen=0x0)
    at /home/william/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/Domain.cpp:121
#17 0x00007ffff71b1a24 in rmw_create_node (name=0xa13d30 "cam2image", domain_id=87)
    at /home/william/ros2_ws/src/eProsima/ROS-RMW-Fast-RTPS-cpp/rmw_fastrtps_cpp/src/functions.cpp:587
#18 0x00007ffff75ea2d6 in rcl_node_init (node=0x67ee40, name=0xa13d30 "cam2image", options=0x7fffffff9db0)
    at /home/william/ros2_ws/src/ros2/rcl/rcl/src/rcl/node.c:109
#19 0x00007ffff7b36b6c in rclcpp::node::Node::Node (this=0xa13d00, node_name="cam2image", 
    context=std::shared_ptr (count 9, weak -1) 0x7fffffff9e90, use_intra_process_comms=false)
    at /home/william/ros2_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/node.cpp:90
#20 0x00007ffff7b36606 in rclcpp::node::Node::Node (this=0xa13d00, node_name="cam2image", use_intra_process_comms=false)
    at /home/william/ros2_ws/src/ros2/rclcpp/rclcpp/src/rclcpp/node.cpp:34
#21 0x0000000000431307 in __gnu_cxx::new_allocator<rclcpp::node::Node>::construct<rclcpp::node::Node, char const (&) [10]> (
    this=0x7fffffffa017, __p=0xa13d00) at /usr/include/c++/5/ext/new_allocator.h:120
#22 0x000000000042fd7a in std::allocator_traits<std::allocator<rclcpp::node::Node> >::construct<rclcpp::node::Node, char const (&) [10]> (__a=..., __p=0xa13d00) at /usr/include/c++/5/bits/alloc_traits.h:530
---Type <return> to continue, or q <return> to quit---
#23 0x000000000042e27e in std::_Sp_counted_ptr_inplace<rclcpp::node::Node, std::allocator<rclcpp::node::Node>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char const (&) [10]> (this=0xa13cf0, __a=...) at /usr/include/c++/5/bits/shared_ptr_base.h:522
#24 0x000000000042cadd in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<rclcpp::node::Node, std::allocator<rclcpp::node::Node>, char const (&) [10]> (this=0x7fffffffa248, __a=...) at /usr/include/c++/5/bits/shared_ptr_base.h:617
#25 0x000000000042b54e in std::__shared_ptr<rclcpp::node::Node, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<rclcpp::node::Node>, char const (&) [10]> (this=0x7fffffffa240, __tag=..., __a=...) at /usr/include/c++/5/bits/shared_ptr_base.h:1097
#26 0x0000000000429ce8 in std::shared_ptr<rclcpp::node::Node>::shared_ptr<std::allocator<rclcpp::node::Node>, char const (&) [10]> (
    this=0x7fffffffa240, __tag=..., __a=...) at /usr/include/c++/5/bits/shared_ptr.h:319
#27 0x0000000000427d38 in std::allocate_shared<rclcpp::node::Node, std::allocator<rclcpp::node::Node>, char const (&) [10]> (
    __a=...) at /usr/include/c++/5/bits/shared_ptr.h:620
#28 0x0000000000425bbf in std::make_shared<rclcpp::node::Node, char const (&) [10]> () at /usr/include/c++/5/bits/shared_ptr.h:636
#29 0x0000000000424308 in rclcpp::node::Node::make_shared<char const (&) [10]> ()
    at /home/william/ros2_ws/install_isolated/rclcpp/include/rclcpp/node.hpp:83
#30 0x000000000042141a in main (argc=8, argv=0x7fffffffa6d8)
    at /home/william/ros2_ws/src/ros2/demos/image_tools/src/cam2image.cpp:95

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have this issue on my Mac.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restarting my VM addressed the issue... That might be a case we want to handle more gracefully in the future.

@wjwwood
Copy link
Member Author

wjwwood commented Jul 8, 2016

Other than the hiccup I ran into where I had to restart my VM, this works for me.

+1 to test it and then merge it after the Fast-RTPS feature branch has been merged.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 25, 2016

@wjwwood to look at why CI cannot rebase this: http://ci.ros2.org/job/ci_linux/1702/consoleFull

@wjwwood wjwwood mentioned this pull request Aug 25, 2016
@wjwwood wjwwood force-pushed the large_message_pub_sub_revision branch from 080c5e4 to 5ccdc92 Compare August 25, 2016 18:13
@wjwwood
Copy link
Member Author

wjwwood commented Aug 25, 2016

Ok, I had to squash the commits, there was something very weird in the history of this branch. I backed up the previous state to the large_message_pub_sub_revision_backup branch.

Here is CI:

  • Linux:
    • Build Status
  • OS X:
    • Build Status
  • Windows:
    • Build Status

@richiprosima
Copy link
Contributor

There are compilation errors on OSX and Windows. I will solve these tomorrow.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 25, 2016

@richiprosima there is something else going on... The history on these branches are pretty messed up, even after I tried to squash it and rebase it. I think until we can figure that out, I'm going to look at #54 to see if I can get it into merging shape instead. Then rebuild this pr on top of that, hopefully cherry-picking only the meaningful changes in this pr.

@richiprosima
Copy link
Contributor

@wjwwood Ok. I will wait until you update #54 branch

@wjwwood
Copy link
Member Author

wjwwood commented Aug 25, 2016

Actually, I found some other issue in my local workspace. I think this one is back on track with 001bdce and a73213a.
New CI:

  • Linux:
    • Build Status
  • OS X:
    • Build Status
  • Windows:
    • Build Status

@wjwwood
Copy link
Member Author

wjwwood commented Aug 25, 2016

Sorry, I had local API changes I forgot about, so I have to remove those changes for now in 6608f36.

CI, hopefully just one more time:

  • Linux:
    • Build Status
  • OS X:
    • Build Status
  • Windows:
    • Build Status

@wjwwood
Copy link
Member Author

wjwwood commented Aug 25, 2016

@richiprosima were these the two test failures you were talking about?

http://ci.ros2.org/job/ci_linux/1705/testReport/

@richiprosima
Copy link
Contributor

@wjwwood Yes, they are.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 26, 2016

This doesn't see to be a problem on Windows:

Build Status

I get a green build after fixing the Windows compiler warnings.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 26, 2016

Ok, those remaining test failures are due to the fact that we do not wait for the parameter server to be ready before requesting something with the client as part of the test. Which is what you theorized @richiprosima. I'm going to put in a sleep in our client part of tests with a todo to remove it and an issue to track the problem so that we can move forward with this.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 26, 2016

ros2/rclcpp#249

@wjwwood
Copy link
Member Author

wjwwood commented Aug 26, 2016

New CI with ros2/system_tests#159 in place.

CI:

  • Linux:
    • Build Status
  • OS X:
    • Build Status
  • Windows:
    • Build Status

@wjwwood
Copy link
Member Author

wjwwood commented Aug 26, 2016

The only issue on the latest Linux build was an uncrustify issue, which I just solved. So I think it should be good to merge if that is also the only thing on Windows and OS X.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 27, 2016

Ran another Linux job to verify:

Build Status

The UDP6 test that is failing within FastRTPS on OS X is not new, so I'll ignore that one for now.

@wjwwood wjwwood merged commit e336a8b into master Aug 27, 2016
@richiprosima richiprosima removed the in progress Actively being worked on (Kanban column) label Aug 27, 2016
@wjwwood
Copy link
Member Author

wjwwood commented Aug 27, 2016

Thanks for helping us with this @richiprosima.

@wjwwood
Copy link
Member Author

wjwwood commented Aug 30, 2016

@SantiagoMunoz I tried to preserve the history, but at some point I think you guys merged an out-of-date version of master into this branch and I couldn't easily keep the old commits. I can try to redo the history and cherry-pick from the backup branch:

https://github.com/eProsima/ROS-RMW-Fast-RTPS-cpp/commits/large_message_pub_sub_revision_backup

@SantiagoMunoz
Copy link

SantiagoMunoz commented Aug 31, 2016

@wjwwood No worries, the important matter is that the changes are merged. Im happy to see things working now, thanks for the effort!

ahcorde pushed a commit to erlerobot/ROS-RMW-Fast-RTPS-cpp that referenced this pull request Sep 13, 2016
* use dynamical memory allocation to allow large, unbounded messages

* small fixes

* address linting issues

* remove change that was made against local API changes

* fix Windows warnings

Conflicts:
	rmw_fastrtps_cpp/include/rmw_fastrtps_cpp/TypeSupport_impl.h
	rmw_fastrtps_cpp/src/functions.cpp
@dirk-thomas dirk-thomas deleted the large_message_pub_sub_revision branch October 31, 2016 17:13
@dirk-thomas
Copy link
Member

See #108 fixing a regression of this patch which led to wake up unnecessarily.


CustomParticipantInfo* impl = static_cast<CustomParticipantInfo*>(node->data);

std::map<std::string,std::set<std::string>>unfiltered_topics;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the set here imply that topics with the same type are only counted once here? This could be the reason for #127.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Magic numbers in serialization
5 participants