Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dry] message generation extrmely slow #183

Closed
dirk-thomas opened this issue Mar 2, 2013 · 4 comments
Closed

[dry] message generation extrmely slow #183

dirk-thomas opened this issue Mar 2, 2013 · 4 comments
Assignees
Labels

Comments

@dirk-thomas
Copy link
Member

The message generation in dry packages takes a very long time (reported by @hershwg).

@ghost ghost assigned dirk-thomas Mar 2, 2013
@dirk-thomas
Copy link
Member Author

Step-by-step use case provided by Dave:

sudo emacs /opt/ros/groovy/share/roscpp/rosbuild/scripts/genmsg_cpp.py

then make it look like this:
............

import msg_gen

import sys

if len(sys.argv) > 1:
import os.path
import cProfile
outname = '/tmp/genmsg_cpp.profile.' + os.path.basename( sys.argv[1] )
cProfile.run('msg_gen.generate_messages(sys.argv)', outname)
exit

if name == "main":
msg_gen.generate_messages(sys.argv)

............
(Not very proper, but it got me a quick result without needing to understand how to set up the test environment)

Now set up the target messages:

cd ~/groovy_stacks # ...or wherever you like to check out dry packages
git clone git@github.com:ros-interactive-manipulation/object_manipulation.git
cd object_manipulation/object_manipulation_msgs

now do the regular message building step for dry packages:

make

Since step 1 was to edit the installed python script in-place, the regular message build does the profiling now. It leaves its result files in /tmp/genmsg_cpp.profile.*

Now to process the results, you need more python:

cd /tmp # ... since that is where the files are...
ipython

inside ipython I did:

import pstats
p = pstats.Stats('genmsg_cpp.profile.PlaceAction.msg')
p.strip_dirs().sort_stats('cumulative').print_stats(20)

Once you have the "p" stats object, there are lots of ways to process it. I read this page: http://docs.python.org/2/library/profile.html to learn what I know about python profiling.

@dirk-thomas
Copy link
Member Author

For genmsg_cpp.py object_manipulation_msgs/msg/PickupAction.msg it takes 95s for me...

dirk-thomas added a commit to ros/ros that referenced this issue Mar 2, 2013
@dirk-thomas
Copy link
Member Author

After these "tiny" optimizations the time is down to 1.3s...

@dirk-thomas
Copy link
Member Author

The same will be the case for service...

@dirk-thomas dirk-thomas reopened this Mar 2, 2013
PierrickKoch referenced this issue in PierrickKoch/robotpkg Feb 21, 2014
Changes since 1.8.15:

1.9.46 (2013-06-18)
-------------------

* add dependency on roslisp
  (`#240 <https://github.com/ros/ros_comm/issues/240>`_)
* fix crash in bag migration
  (`#239 <https://github.com/ros/ros_comm/issues/239>`_)
* add CMake function roslaunch_add_file_check()
  (`#241 <https://github.com/ros/ros_comm/issues/241>`_)
* fix rosnode_ping to check if new node uri is valid before using it
  (`#235 <https://github.com/ros/ros_comm/issues/235>`_)

1.9.45 (2013-06-06)
-------------------
* improve handling of UDP transport, when fragmented packets are lost or arive
  out-of-order the connection is not dropped anymore, onle a single message is
  lost (`#226 <https://github.com/ros/ros_comm/issues/226>`_)
* fix missing generation of constant definitions for services
  (`ros/gencpp#2 <https://github.com/ros/gencpp/issues/2>`_)
* fix restoring thread context when callback throws an exception
  (`#219 <https://github.com/ros/ros_comm/issues/219>`_)
* fix calling PollManager::shutdown() repeatedly
  (`#217 <https://github.com/ros/ros_comm/issues/217>`_)
* add missing run_depend on python-yaml
* allow configuration of ports for XML RPCs and TCP ROS
* fix race condition where rospy subscribers do not connect to all publisher
* fix closing and deregistering connection when connect fails
  (`#128 <https://github.com/ros/ros_comm/issues/128>`_)
* fix log level of RosOutHandler
  (`#210 <https://github.com/ros/ros_comm/issues/210>`_)
* added option '--duration' to 'rosbag play'
  (`#121 <https://github.com/ros/ros_comm/issues/121>`_)
* fix missing newlines in rosbag error messages
  (`#237 <https://github.com/ros/ros_comm/issues/237>`_)
* fix flushing for tools like 'rosbag compress'
  (`#237 <https://github.com/ros/ros_comm/issues/237>`_)
* add warnings for obviously wrong environment variables ROS_HOSTNAME and
  ROS_IP (`#134 <https://github.com/ros/ros_comm/issues/134>`_)
* fix exception on netifaces.ifaddresses()
  (`#211 <https://github.com/ros/ros_comm/issues/211>`_,
  `#213 <https://github.com/ros/ros_comm/issues/213>`_)
  (regression from 1.9.42)
* modify rosnode_ping to check for changed node uri if connection is refused
  (`#221 <https://github.com/ros/ros_comm/issues/221>`_)
* allow passing arguments to add_rostest(ARGS ...)
 (`#232 <https://github.com/ros/ros_comm/issues/232>`_)
* modified roslaunch $(find PKG) to consider path behind it for resolve
  strategy (`#233 <https://github.com/ros/ros_comm/pull/233>`_)
* add boolean attribute 'subst_value' to rosparam tag in launch files
  (`#218 <https://github.com/ros/ros_comm/issues/218>`_)
* add command line parameter to print out launch args
* fix missing import in arg_dump.py
* fix template syntax for signal_.template addCallback() to work with Intel
  compiler

1.9.44 (2013-03-21)
-------------------
* fix install destination for dll's under Windows
* fix various issues on Windows
  (`#189 <https://github.com/ros/ros_comm/issues/189>`_)
* fix 'roslaunch --files' with non-unique anononymous ids
  (`#186 <https://github.com/ros/ros_comm/issues/186>`_)
* fix ROS_MASTER_URI for Windows

1.9.43 (2013-03-13)
-------------------
* implement process killer for Windows
* fix exports of message filter symbols for Windows

1.9.42 (2013-03-08)
-------------------
* improve speed of message generation in dry packages
  (`#183 <https://github.com/ros/ros_comm/issues/183>`_)
* fix roscpp service call deadlock
  (`#149 <https://github.com/ros/ros_comm/issues/149>`_)
* fix freezing service calls when returning false
  (`#168 <https://github.com/ros/ros_comm/issues/168>`_)
* fix error message publishing wrong message type
  (`#178 <https://github.com/ros/ros_comm/issues/178>`_)
* fix missing explicit dependency on pthread
  (`#135 <https://github.com/ros/ros_comm/issues/135>`_)
* fix compiler warning about wrong comparison of message md5 hashes
  (`#165 <https://github.com/ros/ros_comm/issues/165>`_)
* make dependencies on rospy optional by refactoring RosStreamHandler to
  rosgraph (`#179 <https://github.com/ros/ros_comm/issues/179>`_)
* added option '--duration' to 'rosrun rosbag play'
  (`#121 <https://github.com/ros/ros_comm/issues/121>`_)
* add error message to rosbag when using same in and out file
  (`#171 <https://github.com/ros/ros_comm/issues/171>`_)
* fix handling spaces in folder names
  (`ros/catkin#375 <https://github.com/ros/catkin/issues/375>`_)
* replace custom code with Python module netifaces
  (`#130 <https://github.com/ros/ros_comm/issues/130>`_)
* make dependencies on rospy optional by refactoring RosStreamHandler to
  rosgraph (`#179 <https://github.com/ros/ros_comm/issues/179>`_)
* add option --skip-log-check
  (`#133 <https://github.com/ros/ros_comm/issues/133>`_)
* update API doc to list raised exceptions in config.py
* fix invocation of Python scripts under Windows
  (`#54 <https://github.com/ros/ros_comm/issues/54>`_)
* fix usage of rosservice from within a launch file
* fix missing run_depend on rosbag
  (`#179 <https://github.com/ros/ros_comm/issues/179>`_)

1.9.41 (2013-01-24)
-------------------
* allow sending data exceeding 2GB in chunks
  (`#4049 <https://code.ros.org/trac/ros/ticket/4049>`_)
* update getParam() doc
  (`#1460 <https://code.ros.org/trac/ros/ticket/1460>`_)
* add param::get(float)
  (`#3754 <https://code.ros.org/trac/ros/ticket/3754>`_)
* update inactive assert when publishing message with md5sum *, update related
  tests (`#3714 <https://code.ros.org/trac/ros/ticket/3714>`_)
* fix ros master retry timeout
  (`#4024 <https://code.ros.org/trac/ros/ticket/4024>`_)
* fix inactive assert when publishing message with wrong type
  (`#3714 <https://code.ros.org/trac/ros/ticket/3714>`_)
* improve performance of $(find ...)

1.9.40 (2013-01-13)
-------------------
* add colorization for rospy log output
  (`#3691 <https://code.ros.org/trac/ros/ticket/3691>`_)
* fix socket polling under Windows
  (`#3959 <https://code.ros.org/trac/ros/ticket/3959>`_)
* fix bagsort script (`#42 <https://github.com/ros/ros_comm/issues/42>`_)
* fix dependent packages by pass LOG4CXX include dirs and libraries along
* fix usage of variable arguments in vFormatToBuffer() function
* add colorization for rospy log output
  (`#3691 <https://code.ros.org/trac/ros/ticket/3691>`_)
* fix 'roslaunch --pid=' when pointing to ROS_HOME but folder does not exist
  (`#43 <https://github.com/ros/ros_comm/issues/43>`_)
* fix 'roslaunch --pid=' to use shell expansion for the pid value
  (`#44 <https://github.com/ros/ros_comm/issues/44>`_)
* fix output of 'rossrv --help'
  (`#3979 <https://code.ros.org/trac/ros/ticket/3979>`_)
* add support for boolean in 'rostopic -p'
  (`#3948 <https://code.ros.org/trac/ros/ticket/3948>`_)
* add checks for pip packages and rosdep
* fix check for catkin_pkg
* fix for thread race condition causes incorrect graph connectivity analysis
contradict pushed a commit to contradict/ros_comm that referenced this issue Aug 12, 2016
rqt_launch) Add all-node-stop button.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant