python3 compatability issue

[tfoote]

From @focs at ros-gbp/geometry2-release#1

Hi,
I'm trying to compile ros on my gentoo system. I got stuck while trying to compile the tf2 package:

./src/catkin/bin/catkin_make_isolated --pkg tf2 --force-cmake

The error is:

[100%] Building CXX object CMakeFiles/tf2_py.dir/src/tf2_py.cpp.o
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:62:1: error: invalid conversion from 'const char_' to 'Py_ssize_t {aka long int}' [-fpermissive]
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp: In function 'PyObject_ transform_converter(const TransformStamped_)':
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:109:103: error: 'PyString_FromString' was not declared in this scope
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp: In function 'PyObject_ allFramesAsYAML(PyObject_, PyObject_)':
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:186:59: error: 'PyString_FromString' was not declared in this scope
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp: In function 'PyObject* allFramesAsString(PyObject_, PyObject_)':
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:192:61: error: 'PyString_FromString' was not declared in this scope
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp: In function 'PyObject* setTransform(PyObject_, PyObject_)':
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:388:105: error: 'PyString_AsString' was not declared in this scope
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp: At global scope:
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:457:1: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp: In function 'void init_tf2()':
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:479:67: error: 'PyString_FromString' was not declared in this scope
/home/fox/ros_catkin_ws/src/tf2/src/tf2_py.cpp:496:43: error: 'Py_InitModule' was not declared in this scope
make[2]: *** [CMakeFiles/tf2_py.dir/src/tf2_py.cpp.o] Error 1
make[1]: *** [CMakeFiles/tf2_py.dir/all] Error 2

I think it is partially related to the python2-3 problem. I tried to force python 2 by:

./src/catkin/bin/catkin_make_isolated --pkg tf2 --force-cmake --cmake-args "-DPYTHON_LIBRARIES=/usr/lib64/libpython2.7.so -DPYTHON_INCLUDE_PATH=/usr/include/python2.7/"

So it should take the python2 library and header but there is no change on the output. Maybe this is not the correct way to spesifie the library.

Let me know if I can provide more info.

Thanks

[allenh1]

Just wondering if there was any progress on this. As you know, I've been working on the Gentoo installation for some time now. Desktop_Full is failing on package 182 of 230, tf2_py, and this is the error I am getting. I'm almost 100% sure this is a python 3 compatibility issue (I'm running Python-3.3.6, and every single problem I've run into with compiling the source is python related... Am I supposed to be using 2.7?). @tfoote

<== Finished processing package [181 of 230]: 'smach_ros'

==> Processing catkin package: 'tf2_py'
==> Building with env: '/opt/ros/jade/env.sh'
Makefile exists, skipping explicit cmake invocation...
==> make cmake_check_build_system in '/home/allenh1/catkin_ws/build_isolated/tf2_py'
-- Using CATKIN_DEVEL_PREFIX: /home/allenh1/catkin_ws/devel_isolated/tf2_py
-- Using CMAKE_PREFIX_PATH: /opt/ros/jade
-- This workspace overlays: /opt/ros/jade
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using default Python package layout
-- Using empy: /usr/lib64/python3.3/site-packages/em.py
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/allenh1/catkin_ws/build_isolated/tf2_py/test_results
-- Found gtest: gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-3.3
-- catkin 0.6.14
-- Using these message generators: gencpp;geneus;genlisp;genpy
-- Configuring done
CMake Warning (dev) at CMakeLists.txt:95 (add_dependencies):
Policy CMP0046 is not set: Error on non-existent dependency in
add_dependencies. Run "cmake --help-policy CMP0046" for policy details.
Use the cmake_policy command to set the policy and suppress this warning.

The dependency target "tf2_msgs_gencpp" of target "tf2_py" does not exist.
This warning is for project developers. Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /home/allenh1/catkin_ws/build_isolated/tf2_py
==> make -j8 -l8 in '/home/allenh1/catkin_ws/build_isolated/tf2_py'
Scanning dependencies of target tf2_py
[100%] Building CXX object CMakeFiles/tf2_py.dir/src/tf2_py.cpp.o
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:62:1: error: invalid conversion from ‘const char_’ to ‘Py_ssize_t {aka long int}’ [-fpermissive]
};
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp: In function ‘PyObject_ transform_converter(const TransformStamped_)’:
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:109:103error: ‘PyString_FromString’ was not declared in this scope
PyObject_SetAttrString(pheader, "frame_id", PyString_FromString((transform->header.frame_id).c_str()));
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp: In function ‘PyObject_ allFramesAsYAML(PyObject_, PyObject_)’:
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:186:59:error: ‘PyString_FromString’ was not declared in this scope
return PyString_FromString(bc->allFramesAsYAML().c_str());
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp: In function ‘PyObject* allFramesAsString(PyObject_, PyObject_)’:
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:192:61:error: ‘PyString_FromString’ was not declared in this scope
return PyString_FromString(bc->allFramesAsString().c_str());
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp: In function ‘PyObject* setTransform(PyObject_, PyObject_)’:
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:388:105error: ‘PyString_AsString’ was not declared in this scope
transform.child_frame_id = PyString_AsString(PyObject_BorrowAttrString(py_transform, "child_frame_id"));
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp: In function ‘PyObject* setTransformStatic(PyObject_, PyObject_)’:
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:419:105error: ‘PyString_AsString’ was not declared in this scope
transform.child_frame_id = PyString_AsString(PyObject_BorrowAttrString(py_transform, "child_frame_id"));
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp: In function ‘void init_tf2()’:
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:512:67:error: ‘PyString_FromString’ was not declared in this scope
pModulerospy = PyImport_Import(item= PyString_FromString("rospy")); Py_DECREF(item);
^
/home/allenh1/catkin_ws/src/geometry_experimental/tf2_py/src/tf2_py.cpp:529:43:error: ‘Py_InitModule’ was not declared in this scope
m = Py_InitModule("_tf2", module_methods);
^
CMakeFiles/tf2_py.dir/build.make:54: recipe for target 'CMakeFiles/tf2_py.dir/src/tf2_py.cpp.o' failed
make[2]: *** [CMakeFiles/tf2_py.dir/src/tf2_py.cpp.o] Error 1
CMakeFiles/Makefile2:711: recipe for target 'CMakeFiles/tf2_py.dir/all' failed
make[1]: *** [CMakeFiles/tf2_py.dir/all] Error 2
Makefile:126: recipe for target 'all' failed
make: *** [all] Error 2
<== Failed to process package 'tf2_py':
Command '['/opt/ros/jade/env.sh', 'make', '-j8', '-l8']' returned non-zero exit status 2

Reproduce this error by running:
==> cd /home/allenh1/catkin_ws/build_isolated/tf2_py && /opt/ros/jade/env.sh make -j8 -l8

Command failed, exiting.


[allenh1]

Update: everything builds if I use Python-2.7.

[spaghetti-]

We still need to fix this though.

[allenh1]

Ya - that would be good. Let me know if I can do anything.

[benureau]

So I checked, and indeed, tf2_py.cpp is using the Python 2 C API, which is incompatible with Python 3. From there, there are two choices:

• Making the code compatible with Python 3, for instance by using py3c.
• Rewriting the extension in cython, which would make it easier to understand and maintain, and immediately compatible with both versions of Python.

I don't know enough about the ins and outs of this extension to know which way is better. Anyone knowledgeable has any input on this?

[tfoote]

Thanks to @de-vri-es I just merged #173 which adds python3 compatibility. From his testing this is fixed. If anyone else has time to test this before I make the next release that would be appreciated.