Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

urdfdom* packages in Indigo/Trusty are in conflict with system ones #4633

Closed
j-rivero opened this issue Jun 5, 2014 · 96 comments

Comments

Projects
None yet
@j-rivero
Copy link
Contributor

commented Jun 5, 2014

The full story can be read at https://bitbucket.org/osrf/sdformat/issue/63/gzsdf-segfaults-in-trusty.

The short story, in our efforts to move upstream package to official Ubuntu repositories, the first version already using them is Trusty. Indigo is still shipping ros-indigo-urdfdom*, which I believe should not be needed. Unfortunately, they have the same name but are not ABI compatible, so people with software compiled against system urdfdom libraries, will find a problem if source ROS setup.bash and try to run binary.

IMHO, the way to go would be try to stop using ros-indigo-urdfdom* packages and use the system ones. Not sure about the implications of this for ROS (I imagine that a full recompile of all packages depending on urdfdom is needed. I'm also not sure if a simple change in rosdep can make this transition easy) but please, feel free to propose any other solution you find easier (try to mimic exactly the urdfdom system version in ros-indigio-urdfdom could also work).

As a side note, sorry, I should have detected this problem in the beta phase. I will implement a testing mechanism for this kind of ABI incompatibilities in the OSRF jenkins.

@j-rivero j-rivero changed the title urdfdom* package in Indigo/Trusty are in conflict with system ones urdfdom* packages in Indigo/Trusty are in conflict with system ones Jun 5, 2014

@mikeferguson

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2014

@j-rivero I hit this about an hour ago, and had just found the ticket on sdformat about 20 minutes ago. As @jonbinney pointed out on the sdformat ticket, this will pretty much block the use of gazebo+ros+urdf. I suppose one or the other has to be recompiled -- either we setup ROS Indigo to use the upstream urdfdom, or we have to compile gazebo/sdformat as ROS packages again.

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 5, 2014

One question, is there an official deb for urdfdom on saucy?

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 5, 2014

If there is not (I cannot find one) then we must do several things:

  • Import a custom built version of urdfdom which installs to /usr for Saucy into the ROS deb repo
  • Remove urdfdom as a ROS package in Indigo
  • Add a rosdep rule for urdfdom for Ubuntu
  • Rebuild all debs on the ROS build farm which use urdfdom

Also, this makes building on other platforms pretty painful. Currently when building from source, a version of urdfdom is fetched as a ROS package and built as part of the ROS from source build. Once we do the above, any system without urdfdom in the package manager will neither get it from our from source build nor from the package manager. Which means we need to add "create a Homebrew formula for urdfdom" to this list. This will also impact everyone building ros on OS's like Gentoo, Fedora, Arch, Debian, and non-i386/x86-64 Ubuntu systems.

@j-rivero

This comment has been minimized.

Copy link
Contributor Author

commented Jun 5, 2014

@mikeferguson you can trick a little bit the LD_LIBRARY_PATH, check the sdformat bug out for the example. The only point is not to mix software compiled against ROS urdfdom and system urdfdom.

@wjwwood no, saucy has no urdfdom packages. Trusty is the first having them, together with some other known friends: http://qa.debian.org/developer.php?login=jrivero@osrfoundation.org.

I can take care of the homebrew or any other work needed in any other platform. Feel free to assign it to me.

@tfoote

This comment has been minimized.

Copy link
Member

commented Jun 5, 2014

@isucan @davetcoleman FYI

The version in Trusty is 0.2.10 and in indigo it's 0.3.0. I believe I remember a discussion about 0.3.0 being required for something but I don't know what.

@tfoote

This comment has been minimized.

Copy link
Member

commented Jun 5, 2014

Also @hsu FYI

@j-rivero

This comment has been minimized.

Copy link
Contributor Author

commented Jun 5, 2014

@tfoote 0.3.0 in Indigo but the library file name and soname are 0.2?

(trusty)jrivero@nium ~ $ readelf -a  /opt/ros/indigo/lib/liburdfdom_model.so.0.2  | grep SONAME
 0x000000000000000e (SONAME)             Library soname: [liburdfdom_model.so.0.2]

Probably it is not ABI compatible so any reason not to use a proper soname/soversion?

Could be a short-term fix to not affect to all distributions, patch ros-indigo-urdfdom*, use 0.3 soversion and launch the whole re-release process?

@davetcoleman

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

The biggest change I know of for urdfdom is in urdfdom_headers where a undocumented group name member variable was removed from links. This would cause the ABI compatibility. This change has caused changes in Rviz, so it might not be easy to undo it. I also put some minor changes into urdfdom that I would be sad to see not released into indigo, since I have been waiting for them to be released for a year now (since Hydro).

@davetcoleman

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

The rviz changes has a compiler if statement for differing versions of urdfdom
ros-visualization/rviz@09bc0e0#diff-deb887957b4c886cf99b6fa13089be45R653

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 6, 2014

@davetcoleman what changes are in master of urdfdom which are not in 0.2.3 which you would like to see in Indigo.

We are hoping that there is an ABI compatible way to have 0.2.3 in /usr AND the the features you want.

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 6, 2014

Let me be more specific, currently in Trusty:

  • urdfdom 0.2.10
  • urdfdom_headers 0.2.3

Currently in Indigo:

  • urdfdom 0.3.0
  • urdfdom_headers 0.3.0

Now, I believe that the changes in urdfdom_headers between 0.2.3 and 0.3.0 are trivial and not required for Indigo (@isucan may disagree?), therefore we can roll back the version of that to 0.2.3 for Indigo (allowing us to remove urdfdom_headers from ROS and just use Trusty's). The question for @davetcoleman is:

What changes between 0.2.10 and 0.3.0 of urdfdom are important for Indigo?

ros/urdfdom@0.2.10...0.3.0

If we can determine that then we can see if we can back port those changes to a 0.2.11 of urdfdom which uses 0.2.3 urdfdom_headers and can be released into Trusty as a patch release.

@davetcoleman

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

I would like to see in Indigo this very minor change, which is abi -compatible: ros/urdfdom@cc286e3

It just drastically reduces the amount of noise when using MoveIt in debug logging level.

I also have some trivial commits to README.md. Everything else I am not involved in so don't know, but I don't see (reading through the code just now) any changes to header files so perhaps we can backport all the changes?

@scpeters

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

After discussing with @wjwwood , we think the next step should be to create a branch called 0.2 and cherry pick @davetcoleman 's commits to it in order to make a patch release to send into ubuntu.

@hsu as a maintainer, can you create the 0.2 branch? I will make pull requests to it, but only a maintainer can create branches.

@hsu

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

branch created
On Jun 6, 2014 2:32 PM, "Steven Peters" notifications@github.com wrote:

After discussing with @wjwwood https://github.com/wjwwood , we think
the next step should be to create a branch called 0.2 and cherry pick
@davetcoleman https://github.com/davetcoleman 's commits to it in order
to make a patch release to send into ubuntu.

@hsu https://github.com/hsu as a maintainer, can you create the 0.2
branch? I will make pull requests to it, but only a maintainer can create
branches.


Reply to this email directly or view it on GitHub
#4633 (comment).

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 6, 2014

I think back porting all the changes from 0.3.0 to 0.2.10 is possible, we will just have to revert any changes which require 0.3.0 of urdfdom_headers.

@scpeters

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

@hsu Thanks, but I forgot to ask that the branch be based of of the 0.2.10 tag. Can you rebase it?

@hsu

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

doh. rebasing.

On Fri, Jun 6, 2014 at 2:49 PM, Steven Peters notifications@github.com
wrote:

@hsu https://github.com/hsu Thanks, but I forgot to ask that the branch
be based of of the 0.2.10 tag. Can you rebase it?


Reply to this email directly or view it on GitHub
#4633 (comment).

@hsu

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2014

recreated the branch from 0.2.10

On Fri, Jun 6, 2014 at 2:51 PM, John Hsu hsu@osrfoundation.org wrote:

doh. rebasing.

On Fri, Jun 6, 2014 at 2:49 PM, Steven Peters notifications@github.com
wrote:

@hsu https://github.com/hsu Thanks, but I forgot to ask that the
branch be based of of the 0.2.10 tag. Can you rebase it?


Reply to this email directly or view it on GitHub
#4633 (comment).

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 7, 2014

So what is the strategy then, do we remove urdfdom and urdfdom_headers from Indigo and just rely on the Trusty deb (then push 0.2.11 forward into Trusty) or should we leave them in Indigo, but just change the version to 0.2.10 and then 0.2.11 when it is released, assuming that they are ABI compatible?

@wjwwood

This comment has been minimized.

Copy link
Member

commented Jun 7, 2014

So, after discussing it, this will be the actions we take:

  • Merge the pull request ros/urdfdom#37 and release 0.2.11 ASAP
  • @j-rivero should figure out if we can get 0.2.11 upstream into Trusty ASAP
    • These two items are in the critical path, because if we cannot get 0.2.11 into Trusty then we need to keep urdfdom in the ROS distribution, otherwise we can drop it and rely on the on in Trusty

If we get 0.2.11 into Trusty:

  • Remove ros-indigo-urdfdom{_headers}
  • Add a rosdep rule for urdfdom{_headers}
  • Rerun bloom on all packages which directly depend on urdfdom or urdfdom_headers
    • This is required to update the control file references, e.g. liburdfdom instead of ros-indigo-urdfdom
  • Create a deb for urdfdom{_headers} on Saucy and import it into the ROS package repo
  • Create a Homebrew formula for urdfdom{_headers} and add a rosdep rule

Otherwise we will:

  • Either release 0.2.11 or 0.3.1 (not sure which is required yet) of urdfdom{_headers} into Indigo
  • Assert that this fixes the issue, i.e. the deb in Trusty and the ros-indigo deb are compatible

Unfortunately this means we cannot take action on this until we know about getting 0.2.11 into Trusty.

So I'm sorry to say that for people affected by this, you will have to build from source in the mean time. @jonbinney @mikeferguson

@mikeferguson

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2014

@wjwwood not a problem to build from source right now -- I just wanted to make it clear this should be considered a real blocker for releasing Indigo beyond beta.

Have any of the fedora crowd seen this ticket? I haven't seen any input from them -- I'm not sure who all that entails, but perhaps @wjwwood or @tfoote could link in the right people so they are aware of this upcoming change (which will likely break their releases)

@tfoote

This comment has been minimized.

Copy link
Member

commented Jun 7, 2014

@cottsay FYI

@cottsay

This comment has been minimized.

Copy link
Member

commented Jun 7, 2014

Thanks for bringing me in, @tfoote and @mikeferguson.

Fedora has urdfdom{-headers} 0.2.9 in f19 and f20 proper:

I haven't hit any problems in RPM packaging because urdfdom{-headers}-devel is never installed at compile time. I would need to talk to Rich, but we might be able to update to version 3 if need be. I can't find any downstream packages that would have API/ABI dependencies.

We could almost certainly release 0.2.11. Just give us at least a week of testing before Indigo release if we need to release an update.

@j-rivero

This comment has been minimized.

Copy link
Contributor Author

commented Jun 7, 2014

I've started the first part of the process that is to get the patch merged into Ubuntu development distro (Utopic). You can find the full procedure process here: https://wiki.ubuntu.com/StableReleaseUpdates#Procedure

https://code.launchpad.net/~j-rivero/ubuntu/trusty/urdfdom/urdfdom_ros_patch/+merge/222426

@jonbinney

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2014

For anyone else who needs to build gazebo for source for now, I've created a couple of git repos for gazebo2.2.2 and sdformat1.4.11 plus package.xml's. Using this rosinstall (https://gist.github.com/jonbinney/942e110bbd4fbe7044fc) you can catkin_make_isolated a workspace that includes gazebo, sdformat, and gazebo_ros_pkgs.

It might not be the most beautiful way of doing it, but it should work for a week or two while this gets resolved.

@tfoote tfoote added the critical label Jun 14, 2014

@tfoote

This comment has been minimized.

Copy link
Member

commented Jun 14, 2014

So in reviewing my understanding of the state of things I don't think we can wait for movement upstream. They may not want to take the patch.

To get going what we can do is:

  • create a urdfdom{_headers} debian package (backport the trusty ones) for saucy.
  • remove urdfdom* from Indigo using pure debian packages
  • run bloom on all packages depending directly on urdfdom*
  • Optionally make a debian patch release with cleaned up debugging code which will be overridden when/if a new upstream version is released.

@j-rivero could you look at creating Saucy versions of urdfdom?

@NikolausDemmel

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2014

Have urdfdom_headers, urdfdom and console_bridge been packaged for homebrew yet?

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 1, 2014

srdfdom patched in #4926

@scpeters

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2014

@NikolausDemmel yes they have been packaged for homebrew:

ros/homebrew-deps@7057811

#4884

As discussed in osrf/homebrew-simulation#22

@j-rivero

This comment has been minimized.

Copy link
Contributor Author

commented Jul 1, 2014

@mikeferguson @jonbinney it could be that you still have the ros-indigo-urdfdom package installed and we are reproducing this same bug again?

After sourcing setup.sh ldd on the affected binaries should display that urdfdom is linked against system version and not the one in ROS.

*update: * I've just seen that the latest version of ros-indigo-urdfdom is already a transition package:

ii  ros-indigo-urdfdom                              0.3.0-2trusty-20140624-0141- amd64                        A temporary pass through to the urdfdom rosdep. You should depend on liburdfdom-dev directly.
@NikolausDemmel

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2014

@scpeters: thanks for the links. I noticed the keys in rosdep, but wasn't aware of the new Tap.

@jonbinney

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2014

@j-rivero it appears to be linking against the correct urdfdom:

jbinney@h-> gzsdf print nao.urdf 
*** Error in `gzsdf': free(): invalid pointer: 0x0000000001ea4b70 ***
Aborted (core dumped)

jbinney@h-> which gzsdf
/usr/bin/gzsdf

jbinney@h-> ldd /usr/bin/gzsdf | grep urdf                                                                                                
        liburdfdom_model.so.0.2 => /usr/lib/x86_64-linux-gnu/liburdfdom_model.so.0.2 (0x00007f1cb8542000)

jbinney@h-> dpkg -S /usr/lib/x86_64-linux-gnu/liburdfdom_model.so.0.2
liburdfdom-model0.2:amd64: /usr/lib/x86_64-linux-gnu/liburdfdom_model.so.0.2

Can you try the nao urdf i posted above and see if gzsdf print works for you with the ros-shadow fixed debs?

@j-rivero

This comment has been minimized.

Copy link
Contributor Author

commented Jul 2, 2014

You were totally right @jonbinney, there is a different error on sdformat and I'm afraid that lives in the Ubuntu official package that needs patching. Details and workaround in the sdformat issue tracker.

To avoid waiting the patch in Ubuntu, that will take a bit of time, one option could be:

  • Generate a patched sdformat-1.4.11-1osrf1 and host it in ROS repo
  • Generate new gazebo2 (2.2.2-5) which depends on 1.4.11-1osrf1 (or greater) and update the package in ROS.
@j-rivero

This comment has been minimized.

Copy link
Contributor Author

commented Jul 7, 2014

The status of the fix for this latest issue with sdformat can be found in #4933

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 7, 2014

sdformat is being tracked separately: All elements in this comment are checked off: #4633 (comment)

Once they build I will plan to merge #4830

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 8, 2014

#4830 closed and packages cleared from the repos.

@tfoote tfoote closed this Jul 8, 2014

@syrnick

This comment has been minimized.

Copy link

commented Jul 9, 2014

Installing ros-indigo-desktop-full fails right now. http://answers.ros.org/question/185659/package-ros-indigo-urdfdom-headers-has-no-installation-candidate/

How long should it take to converge and rebuild the packages?

@kscottz

This comment has been minimized.

Copy link

commented Jul 9, 2014

I just tried to install desktop-full from apt-get and ran into the same issue. Looks like it is getting resolved. I can test this in the wild if it helps.

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 9, 2014

The public repository should not have updated yet. I'm trying to track down what updated in the repository.

@tfoote tfoote reopened this Jul 9, 2014

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 9, 2014

The NYC mirror ran out of space. It's being remedied at the moment. Hopefully with that back in place things will recover.

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 10, 2014

It looks also like the staged updates for urdfdom were pulled to the public mirrors incorrectly. We're debugging why. But indigo is close to a clean build so we will sync as soon as the buildfarm finishes.

@tfoote

This comment has been minimized.

Copy link
Member

commented Jul 10, 2014

WIth the new sync i've verified that you can now install correctly from packages.ros.org.

@tfoote tfoote closed this Jul 10, 2014

severin-lemaignan pushed a commit to severin-lemaignan/robotpkg that referenced this issue Aug 18, 2014

Anthony Mallet
[graphics/ros-srdfdom] Update to 0.2.7
Changes since 0.2.5:
0.2.7 (2014-07-01)
------------------
* fixing dependencies for ros/rosdistro#4633
* added travis build status indicator in README.md
* added travis support
* use FindTinyXML from cmake_module
* Contributors: Dave Coleman, Dave Hershberger, Ioan Sucan, Tully Foote

0.2.6 (2013-07-19)
------------------
* fix incorrect tag name

severin-lemaignan pushed a commit to severin-lemaignan/robotpkg that referenced this issue Aug 18, 2014

Anthony Mallet
[sysutils/ros-console-bridge] Update to 0.4.2
Changes since 0.2.7:

0.4.2 (2014-07-18)
------------------
* update maintainers

0.4.1 (2014-06-25)
------------------
* update find_package for console bridge to reflect 3rdparty status (`#8
<https://github.com/ros/rosconsole_bridge/issues/8>`_)

0.4.0 (2014-06-24)
------------------
* rename variables within rosconsole macros (`ros/ros_comm#442
<https://github.com/ros/ros_comm/issues/442>`_)
* convert to use console bridge from upstream debian package
(`ros/rosdistro#4633 <https://github.com/ros/rosdistro/issues/4633>`_)

0.3.4 (2014-02-15)
------------------
* Allows non-ros components to utilize the prefix functionality of log4j
* Contributors: Dave Coleman, Ioan A Sucan

0.3.3 (2013-08-21)
------------------
* fixing catkin as a buildtool dependency
* Contributors: Dirk Thomas, Tully Foote

0.3.2 (2013-04-12)
------------------
* changes due to renames in console_bridge
* Contributors: Ioan Sucan

0.3.1 (2013-03-13 21:24)
------------------------
* fix deps
* update e-mail address
* Contributors: Ioan Sucan

Dropped patch-aa applied upstream.
@mryellow

This comment has been minimized.

Copy link

commented Mar 29, 2015

I'm having an issue with srdfdom wanting urdfdom_py as a ros dependency in debian jessie.

A little confused by how I'd approach doing it as a source install with the system dep mixed in (seeing the code is inside urdfdom repo)

Should urdfdom_py also be released as a separate system dep as part of this change?

@tfoote

This comment has been minimized.

Copy link
Member

commented Mar 30, 2015

@mryellow this looks like a new topic. You should ask a question on http://answers.gazebosim.org

@mryellow

This comment has been minimized.

Copy link

commented Mar 30, 2015

No problem, though it might be part of this whole deployment change....

Seems there is no way to get urdfdom_py without messing up these system deps, taking liburdfdom-dev back to source and hacking some kind of urdfdom_py proxy package. Which isn't ideal by any means.

From my perspective, I'm more likely to take srdfdom back 4 days to when it didn't use urdfdom_py... Not ideal either.

edit:

ok ok, they can coexist... https://github.com/ros-gbp/urdfdom_py-release from source worked along-side this setup, not sure where I took a wrong turn before. Does require a tweak to srdfdom package.xml, no depend for urdfdom_py. Sorry for the mess.

@j-rivero j-rivero referenced this issue Feb 10, 2018

Closed

dartsim 6.3.0 (new formula) #23903

4 of 4 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.