dist-upgrade shadow-fixed rostest

[davetcoleman]

I'm not sure where to post this, but running dist-upgrade after switching the ROS source path to shadow-fixed in Docker gives the following error:

Errors were encountered while processing:
 /var/cache/apt/archives/ros-kinetic-rostest_1.12.4-0xenial-20160920-001830-0700_amd64.deb
�[91mE�[0m�[91m: Sub-process /usr/bin/dpkg returned an error code (1)�[0m�[91m

To investigate, I started up an interactive Docker ros:kinetic-ros-base, switched to shadow-fixed:

echo "deb http://packages.ros.org/ros-shadow-fixed/ubuntu `lsb_release -cs` main" | tee /etc/apt/sources.list.d/ros-latest.list

Then ran

apt-get update
apt-get dist-upgrade

And I get another (similar) error with rostest:

apt-get install ros-kinetic-rostest
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  ros-kinetic-rostest
1 upgraded, 0 newly installed, 0 to remove and 132 not upgraded.
Need to get 28.5 kB of archives.
After this operation, 10.2 kB of additional disk space will be used.
Get:1 http://packages.ros.org/ros-shadow-fixed/ubuntu xenial/main amd64 ros-kinetic-rostest amd64 1.12.4-0xenial-20160920-001830-0700 [28.5 kB]
Fetched 28.5 kB in 0s (159 kB/s)               
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 44107 files and directories currently installed.)
Preparing to unpack .../ros-kinetic-rostest_1.12.4-0xenial-20160920-001830-0700_amd64.deb ...
Unpacking ros-kinetic-rostest (1.12.4-0xenial-20160920-001830-0700) over (1.12.2-0xenial-20160603-153503-0700) ...
dpkg: error processing archive /var/cache/apt/archives/ros-kinetic-rostest_1.12.4-0xenial-20160920-001830-0700_amd64.deb (--unpack):
 unable to move aside './opt/ros/kinetic/share/rostest/nodes/hztest' to install new version: Invalid cross-device link
Errors were encountered while processing:
 /var/cache/apt/archives/ros-kinetic-rostest_1.12.4-0xenial-20160920-001830-0700_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Key line: unable to move aside './opt/ros/kinetic/share/rostest/nodes/hztest' to install new version: Invalid cross-device link

Any thoughts what is going on?

The work around is to remove rostest then reinstall it to get the latest shadow-fixed version.

Thanks!

[dirk-thomas]

I can't reproduce the problem. This looks like some sort of file corruption, not a packaging problem. Can you try to:

• remove just the file /opt/ros/kinetic/share/rostest/nodes/hztest and then retry dist-upgrade or
• remove the package and then try to install it again

[davetcoleman]

That's really surprising you can't reproduce it since I'm build off of a OSRF Docker image that should be the same assuming you have pulled the latest images (I did today). I just ran:

docker run -it --name kinetic-ros-base ros:kinetic-ros-base

Then ran the commands mentioned above.

I did already try removing the package and installing it again, as I mentioned above, and that did work.

Trying your other suggested work around:

rm -rf /opt/ros/kinetic/share/rostest/nodes/hztest

Also worked. I can certainly add this hack to my Docker build file (in fact I already did) but something else seems fishy. I've reproduced the bug locally 4 times now, and on Dockerhub many times also.

[dirk-thomas]

I didn't do the installation in Docker but on my host system which worked fine.

Within the Docker container I can confirm that the problem happens for me too. I double checked the Debian package and it looks good.

The old package had installed /opt/ros/kinetic/share/rostest/nodes/hztest/hztest and the new package wants to install /opt/ros/kinetic/share/rostest/nodes/hztest. apt seems to have a problem doing the update.

Maybe you should ask your question on answers.ros.org to get the attention from more people. Maybe someone has an idea what the difference in Docker is.

[wjwwood]

I believe this is a docker specific issue. Lot's of hits when you search for the error on Google, e.g.:

moby/moby#4824
moby/moby#19309
NVIDIA/nvidia-docker#38

It seems that the Linuxbrew project and the nvidia-docker project have run into this as well in their docker images. Based on the error it seems to be that apt is trying to move a file and then hard link it across two filesystems that don't support that. It's probably some issue with the filesystem abstraction docker uses for /tmp or a similar mount point.

[davetcoleman]

Thanks for looking into this guys! I guess doing a dist-upgrade is dangerous on Docker images because of the hard link bug.

The problem we're having is that the kinetic-ros-base image gets really out of date from ROS shadow-fixed (2 months currently) so it can't be trusted for our CI. I'll move the issue to the ROS Docker github but maybe just stop using OSRF's images in favor of our own.