-
Notifications
You must be signed in to change notification settings - Fork 280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
catkin_make run_tests - can't import Python code #841
Comments
Please provide a reproducible example to get feedback on the problem. From the output I would just guess that the |
Please check this example with two packages: https://github.com/ZdenekM/rostest-test-repo. Dependency is there. |
Thank for the example. I can reproduce the problem now. This can even be reduced to a single package generating the Python message and having a test trying to import the Python module. This fails since There are a couple of workarounds possible:
I don't think there is a way to fix this problem other then the workarounds described above. |
Thank you for your detailed answer. The first approach does not work for me but the second does - great! Maybe, it would be nice to document this somewhere. |
Oh I was hit by this problem as well, and just wanted to confirm what ZdenekM experienced: the first solution did not work, the second did. A work-around yes, but a really annoying one. |
Ran into this same issue and narrowed it down to |
I should add that commenting that line out and sourcing I could make a new issue for this or perhaps this issue could be reopened to explore a solution? |
Please provide the complete steps to reproduce the problem (what are you sourcing before building the workspace, what exactly is in your workspace, etc.). If the problem is reproducible with that information it is probably best to create a new ticket. |
Sure, using @ZdenekM's repo https://github.com/ZdenekM/rostest-test-repo, here are my steps to reproduce the failure: Open a new shell without any ROS scripts sourced or environment variables set and run the following commands (or paste to a script and run it all at once)
Expected Result: The test.py script errors with And to reproduce the success, do the following after the above:
Expected result: Test passes without errors. |
Hmm, a new twist, however. If I run |
Doing the following from a new terminal leads to the tests succeeding. The only difference is sourcing the system
Addition: it doesn't seem to matter where I source |
I tried your first block of commands (with a single source of the Kinetic setup) and I works fine for me (tests passing). Please try the instructions again starting with a clean directory and environment. The additional source command can't make any difference since neither of the commands in between even uses ROS. |
Hmm, I'll look into this more but it was very reproducible between after deleting |
Did you use a "fresh" shell? |
Sorry, is that question asking, "fresh" vs. "bash" or "sh"? I was using bash, I just meant starting a new terminal with a new shell. Didn't realize there was a "fresh" shell, my bad. |
I meant opening a new terminal to ensure that the environment has not been modified in the meantime. |
Ah, yes. I did just that each time. It's strange, I don't get this error in a fresh kinetic Docker container but I can repeatedly on my host, so there must be something odd there which causes the |
Wild guessed:
Maybe look at the output of |
@NikolausDemmel for the win! Just excluding the one line in my .bashrc where I set (prepend) my PYTHONPATH fixes it. Do catkin/ROS rely on having an unset PYTHONPATH before |
What is it set to and does the issue occur with any other value? |
It seems that having a trailing colon on my The line in my .bashrc is Setting it to the same path but without the trailing colon causes the tests to pass again. |
Evidently this causes the current directory to be added to the PYTHONPATH: https://stackoverflow.com/questions/33615156/why-does-pythonpath-with-trailing-colon-add-current-directory-to-sys-path |
Here's the difference in my
Note the trailing colon disappears after the second source |
Tests that depend on the artifact generated in build process don't run without sourcing the setup in the workspace (probably hitting ros/catkin#841).
Was hit by this again. Seems to me there are several versions of this problem. My most confusing version of the problem has been whenever a wrong PYTHONPATH ends up being cached in build/catkin_generated/setup_cached.sh. In this case it is not solved by sourcing devel/setup.bash between build and test. When running "rostest ..." directly this is not a problem, but whenever cmake is controlling the test execution, the cached PYTHONPATH seems to be used. And the testing fails. I solve it by touching a CMakeLists.txt, this seems to rebuild the cache. I have tried to reproduce this reliably, but in vain. |
I've fixed this and posed inside #974 but also posting here for reference. I discovered that the issue comes from a combination of what users are sourcing (e.g source devel/setup.bash vs source devel/setup.sh) and how the environment variable CATKIN_SETUP_UTIL_ARGS is used. The entire story is explained. Hold tight. If you want a tl;dr solution, simply wipe your devel/ build/ install/ directories and have this set in your ~/.bashrc OR if you want just a one off do ' Users most of the time use setup.bash, which doesn't pass down the --extend parameter anyway, so running source devel/setup.bash --extend, so that won't work as it is just a passthrough script which will call setup.sh. That's where the magic happens. setup.sh however does respect the --extend parameter and it also respects the env var CATKIN_SETUP_UTIL_ARGS which is why the above solution works. But what does --extend do and why does it cause things to work? Looking inside _setup_util.py inside your devel directory, you will see that it can take an --extend param (discussed above). If its set, args.extend will be set to true, and so it will avoid the routine _rollback_env_variables. This function fucks everything up as it explicitly states (and does) " For each catkin workspace in CMAKE_PREFIX_PATH remove the first entry from env[NAME] matching workspace + subfolder. Well, if you look in the file you will see that CMAKE_PREFIX_PATH is most likely of the sort "$yourworkspace/devel/:$opt/ros/kinetic/" or something. The problem is that your workspace is first, and unless extend is set then it trims it off of the CMAKE_PREFIX_PATH, thus preventing your libraries from being found. This file generates an environment cache file which sticks around unless you add/remove new catkin packages and so if --extend isn't set, your path will continuously just not contain the proper $PYTHONPATH which includes your custom modules. Now, this can all be completely avoided by doing @dirk-thomas solution in the linked issue because catkin_make_isolated will just take the env var from your existing shell, which is most likely correct because it wouldn't trim off your custom module path in its generated env cache. I can't be fucked to make a PR to fix this workflow because the bashrc work around is fine for me, but if someone would do that I would be grateful that the default behaviour is changed. Proof of it working when the --extend is passed in via env vars:
Proof of it not working when the env var is not set:
|
I ran into this issue (
in my EDIT I couldn't get this to work reliably. @TheDash 's solution doesn't work for me either. I don't understand why this issue is closed. |
@Achllle The ticket is closed since the originally reported problem was resolved. That there are other comments on thread with different cases isn't a reason to keep the ticket open. For any different case where the workarounds don't work consider to open a separate ticket with complete reproducible steps. |
@dirk-thomas The original problem which is the exact same problem I have got resolved using a partial solution (running |
Please see #986 for a proposed patch which avoid the need for the mentioned workarounds. |
That already exists: #974 |
Not sure if this is my fault or a bug. I prepared a test repository with minimal example: https://github.com/ZdenekM/rostest-test-repo. The thing is that there are two packages - one with a custom message and the other with rostest where I would like to use the message. When I run test like this
rostest test_package test.test
it succeeds - but it fails when I runcatkin_make run_tests
(see whole output below - it seems that messages are generated) withImportError: No module named my_msgs.msg
. What might be wrong?The text was updated successfully, but these errors were encountered: