Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rosbash] zsh completion error #83

Closed
neurovertex opened this issue Jun 1, 2015 · 14 comments
Closed

[rosbash] zsh completion error #83

neurovertex opened this issue Jun 1, 2015 · 14 comments
Labels

Comments

@neurovertex
Copy link

I recently updated all my ROS packages to 1.11.8, I started having issues with command completion in zsh. Trying to rosrun a program (package = mipex_sensor, program = "mipex_node"), I type rosrun mip, which autocompletes the package to mipex_sensor as expected, then I type mip again and I get :

rosrun mipex_sensor mipfind: `[my catkin workspace]/devel/lib/mipex_sensor\n[my catkin workspace]/devel/lib/mipex_sensor': No such file or directory

I can see the error comes from the completion script executing find with a would-be-multiline argument which ends up single-line because the \n is uninterpreted. After fiddling around I find the source of the issue : /opt/ros/indigo/share/rosbash/roszsh, _roscomplete_search_dir(), line 329 :

opts=`find $pkgdir $catkin_package_libexec_dir ${=1} -print0 | tr '\000' '\n' | sed -e "s/.*\/\(.*\)/\1/g"`

with some debugging, I find that the problem comes from $catkin_package_libexec_dir. Strangely enough, echo'ing it shows a newline, but when passed as arg to find, the newline turns back into a \n. Or maybe it's correctly passed with a newline but still as a single string (as if quoted) ? I'm not sure. Regardless, I came up with this workaround :

opts=`find $pkgdir $( echo -en "$catkin_package_libexec_dir" ) ${=1} -print0 | tr '\000' '\n' | sed -e "s/.*\/\(.*\)/\1/g"`

Hope that helps :p. Keep up the awesome work

Distro: Arch Linux
Version: 1.11.8

@neurovertex neurovertex changed the title zsh completion error [rosbash] zsh completion error Jun 1, 2015
@dirk-thomas
Copy link
Member

The completion seems to work fine for me with the following example:

rosrun rospy_[tab]
# completes to: rosrun rospy_tutorials 

rosrun rospy_tutorials [tab]
# completion shows all scripts as expected

Can you please try it with this example. If that works for you it is likely related to something in your custom package.

@neurovertex
Copy link
Author

rosrun turtlesim tufind: `/opt/ros/indigo/lib/turtlesim\n/opt/ros/indigo/lib/turtlesim\n/opt/ros/indigo/lib/turtlesim': No such file or directory

I reckon the oddity must come from catkin_find:

catkin_find --libexec turtlesim                    
/opt/ros/indigo/lib/turtlesim
/opt/ros/indigo/lib/turtlesim
/opt/ros/indigo/lib/turtlesim

So I dug a bit further in the code and tried to understand why it'd return several times the same path. I found the actual search function, find_in_workspaces (line 90 in /opt/ros/indigo/lib/python2.7/site-packages/catkin/find_in_package.py), I added a few prints to debug and observed this :

catkin_find --without-underlays --libexec turtlesim
    [workspace 1]/devel
        lib : [workspace 1]/devel/lib
        libexec : [workspace 1]/devel/libexec
    [workspace 2]/devel
        lib : [workspace 2]/devel/lib
        lib : [workspace 2]/devel/lib
        libexec : [workspace 2]/devel/libexec
    /opt/ros/indigo
        lib : /opt/ros/indigo/lib
        lib : /opt/ros/indigo/lib
        lib : /opt/ros/indigo/lib
        libexec : /opt/ros/indigo/libexec
/opt/ros/indigo/lib/turtlesim
/opt/ros/indigo/lib/turtlesim
/opt/ros/indigo/lib/turtlesim

Where 'lib' is searched through one more time for each consecutive workspace. So I found that this (line 120):

        for workspace in (_workspaces or []):
            if 'libexec' in search_dirs:
                search_dirs.insert(search_dirs.index('libexec'), 'lib')

adds "lib" to the search_dirs list for every workspace iterated through, so when it reaches the third one it'll search in its subdirectory /lib three times. That seem very weird to me and very probably an error.
Maybe you don't have the error because the location of rospy_ is the first one of your search path ?
Nevertheless, I've found that moving the "if 'libexec' ..." block before the loop instead of inside it fixes the issue.

Should I move this issue to the catkin repository?

@dirk-thomas
Copy link
Member

Can you please apply the changes from the references PR (just moving the two lines a few lines up) to your local Python code and try if that already solves the problem of the zsh completion for you?

@dirk-thomas
Copy link
Member

The completion also works for me for your turtlesim example without that patch. Can you please test the exact example I mentioned? May be something else is different on your system.

Please also provide more information about your environment, e.g. ROS_PACKAGE_PATH, CMAKE_PREFIX_PATH.

@neurovertex
Copy link
Author

It's a tricky bug to reproduce. See I just reinstalled the packages to check if it was still happening before installing the latest revision, and it doesn't with my own packages anymore, but still with the others. That's because I migrated from catkin_make to catkin build, which apparently changed my variables. Here they are :

ROS_PACKAGE_PATH=[my workspace]/src/mipex_sensor:[my workspace]/src/infotaxis:[my workspace]/src/khepera_node:/opt/ros/indigo/share:/opt/ros/indigo/stacks
CMAKE_PREFIX_PATH=[workspace]/devel:/opt/ros/indigo

Maybe completion works for you because its workspace is the first in CMAKE_PREFIX_PATH ? (where catkin_find returns only one result)
Anyway, I installed the patched version, and it seems to work perfectly :p.

@dirk-thomas
Copy link
Member

I have tried it with a single as well as two overlayed workspaces but was unable to reproduce your problem.

Can you please try to provide a reproducible example. I am not convinced that the zsh completion does not need fixing.

@neurovertex
Copy link
Author

Well I don't really know what to provide I haven't already. Given the code, catkin_find should return several times the same path for any package it find in a part of the path that is not the first one. Are you trying catkin_find or directly autocomplete ? maybe there's a difference between our configs for zsh that makes yours handle the multiline variable better than mine ? That'd be surprising but at that point ...
You can try export CMAKE_PREFIX_PATH="/foo:/bar:$CMAKE_PREFIX_PATH" then catkin_find --libexec "some package" (it has to be --libexec). If that doesn't work, I don't know what will.

If you want to foolproof zsh completion against other bugs that'd produce a multiline variable, I think replacing --without-underlays with --first-only in roszsh, or even safer, adding | head -n 1 to the command should do the trick.

@dirk-thomas
Copy link
Member

You could e.g. share the rosinstall file(s) for your workspace(s) (you can create them using vcs export). Also the command sequence how you built them as well as the values of the environment variables based on that are valuable.

So simply the steps for someone else to reproduce your exact setup. Otherwise it is not clear to me that you problem is completely fixed.

@neurovertex
Copy link
Author

Ok so I reverted catkin to the unpatched version (0.6.14) and here's a complete transcript of what I did :

[neurovertex]$ /tmp % mkdir -p workspace/src 
[neurovertex]$ /tmp % cd workspace/src 
[neurovertex]$ /tmp/workspace/src % source /opt/ros/indigo/setup.zsh 
[neurovertex]$ /tmp/workspace/src % echo $ROS_PACKAGE_PATH; echo $CMAKE_PREFIX_PATH                                     
/opt/ros/indigo/share:/opt/ros/indigo/stacks
/opt/ros/indigo
[neurovertex]$ /tmp/workspace/src % catkin_init_workspace 
Creating symlink "/tmp/workspace/src/CMakeLists.txt" pointing to "/opt/ros/indigo/share/catkin/cmake/toplevel.cmake"
[neurovertex]$ /tmp/workspace/src % cd ..         
[neurovertex]$ /tmp/workspace % catkin_make                 
Base path: /tmp/workspace
Source space: /tmp/workspace/src
Build space: /tmp/workspace/build
Devel space: /tmp/workspace/devel
Install space: /tmp/workspace/install
####
#### Running command: "cmake /tmp/workspace/src -Wno-dev -DPYTHON_EXECUTABLE=/usr/bin/python2 -DPYTHON_INCLUDE_DIR=/usr/include/python2.7 -DPYTHON_LIBRARY=/usr/lib/libpython2.7.so -DCATKIN_DEVEL_PREFIX=/tmp/workspace/devel -DCMAKE_INSTALL_PREFIX=/tmp/workspace/install -G Unix Makefiles" in "/tmp/workspace/build"
####
-- The C compiler identification is GNU 5.1.0
-- The CXX compiler identification is GNU 5.1.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /tmp/workspace/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/indigo
-- This workspace overlays: /opt/ros/indigo
-- Found PythonInterp: /usr/bin/python2 (found version "2.7.10") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python2
-- Using default Python package layout
-- Found PY_em: /usr/lib/python2.7/site-packages/em.pyc  
-- Using empy: /usr/lib/python2.7/site-packages/em.pyc
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /tmp/workspace/build/test_results
-- Found gtest: gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.6.14
-- BUILD_SHARED_LIBS is on
WARNING: Package "libg2o" does not follow the version conventions. It should not contain leading zeros (unless the number is 0).
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:
    PYTHON_INCLUDE_DIR
    PYTHON_LIBRARY
-- Build files have been written to: /tmp/workspace/build
####
#### Running command: "make -j4 -l4" in "/tmp/workspace/build"
####
[neurovertex]$ /tmp/workspace % source devel/setup.zsh
[neurovertex]$ /tmp/workspace % echo $ROS_PACKAGE_PATH; echo $CMAKE_PREFIX_PATH
/tmp/workspace/src:/opt/ros/indigo/share:/opt/ros/indigo/stacks
/tmp/workspace/devel:/opt/ros/indigo                                                  
[neurovertex]$ /tmp/workspace % rosrun turtlesim<TAB> find: `/opt/ros/indigo/lib/turtlesim\n/opt/ros/indigo/lib/turtlesim': No such file or directory
1 [neurovertex]$ /tmp/workspace % catkin_find --libexec turtlesim                                                        :(
/opt/ros/indigo/lib/turtlesim
/opt/ros/indigo/lib/turtlesim
[neurovertex]$ /tmp/workspace % catkin_find --libexec move_base
/opt/ros/indigo/lib/move_base
/opt/ros/indigo/lib/move_base

@dirk-thomas
Copy link
Member

Thank you for the example. Based on that another fix would be to invoke catkin_find with the option --first-only in the following cases:

But with the option libexec the result is always expected to be a single folder so the catkin fix seems to be the better approach.

To be more defensive about it I have modified these too:

@mitchellwills
Copy link

Any chance of there being an indigo release soon to push this fix?

@dirk-thomas
Copy link
Member

Since the actual patch is in catkin it will take a while since we usually wait for several changes to accumulate since it takes a lot of resources to rebuild all packages of a ROS distro on all platforms. So it will likely be a few weeks until the change will be available from public Debian packages.

Until then you could apply the patch from ros/catkin#713 locally to have it working immediately.

@mintar
Copy link

mintar commented Jun 22, 2015

Argh, I just spent over an hour tracking this bug down before finding this issue. @dirk-thomas , perhaps it would be a good idea to sent a notification of this to ros-users to save others the hassle? After all, the last Indigo update broke rosrun + roslaunch for every ROS zsh user.

BTW, here's some step-by-step instructions for applying the patch from ros/catkin#713 locally, as @dirk-thomas suggested:

git clone -b indigo-devel git://github.com/ros/catkin.git /tmp/catkin
sudo cp /tmp/catkin/python/catkin/find_in_workspaces.py /opt/ros/indigo/lib/python2.7/dist-packages/catkin/

@mintar
Copy link

mintar commented Jun 22, 2015

Well, actually it wasn't the last update that broke this. If I read my apt logs correctly, it's already broken for a month now. So perhaps we don't need a ros-users announcement; everyone who cares already has fixed it or given up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants