Undefined Symbol problems at load-time in Linux

[ghost]

Hi,

I am trying to use lunatic python to access python code from within lua.
The problem is, whenever I load the python bridge, I get this error:

lua-python.so: undefined symbol: _Py_ZeroStruct

I am running Ubuntu 13.04 and have tried with lua5.1 and lua5.2, both installed with apt and from source with editing loadlib.c to "RTLD_NOW | RTLD_GLOBAL", as suggested by the original author.
I always recompiled lunatic-python to match the current lua version.

Maybe you can help to resolve this issue.
Thank you!

[greatwolf]

This sounds to be a known problem on lunatic-python. It is likely the same issue as the one posted on SO here and here.

Unfortunately, I doubt I can offer any solution to this problem since I don't really program on linux. I can boot up a VM and investigate this further. From what I recall in lua 5.2 require has an optional second argument that can make all loaded symbols for that module visible to the process. See if py = require("python", "*") helps.

Someone more familiar with linux can chime in.

[greatwolf]

what does ldd lua-python.so show on your system? I'm particularly interested in dependencies to lua_.so and python_.so it shows if any. Also how did you build lunatic-python. Is it through python's distutil setup.py?

[ghost]

I already knew the first post, it covers Python -> Lua and tells to enable RTLD_GLOBAL, which i did in Lua, by changing der loadlib.c file and compiling.
The second talks about linking python to the program, which in my case would be lua itself?

I tried the lua5.2 thing with the second argument, but i tried to apply it to loadlib function, maybe I was a bit confused..

I installed lunatic-python via pip. (yes, the bugfix version)

Please don't invest too much of your time in this, I already found a workaround using a communication library (ZMQ).

Here is the output:
$ ldd lua-python.so
linux-vdso.so.1 => (0x00007fff2eeda000)
liblua5.2.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.2.so.0 (0x00007fc25fae9000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc25f8cc000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc25f503000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc25f1fe000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc25effa000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc25ff3a000)

[greatwolf]

This is just a guess but could it be because when you build lua-python.so, python wasn't included in the library linking? Check and make sure that -lpython* is passed to ld linker (substiute * with your actual python version of course).

I'm thinking none of the Python C functions are defined because libpython*.so was never loaded into the process. If you're not too crazy about rebuilding lunatic again, something else you can try is to explicitly load the python so using package.loadlib in lua 5.2. After that, you can require 'lua-python' and see if the symbols resolve.

package.loadlib("libpython2.7.so", "*")
require "lua-python"

[ghost]

IT WORKED! =)
Thank you very much for you help!

I didn't even recompile lua-python.so, but I had to use my lua5.1 compiled with the RTLD_GLOBAL tag.

could I ask you of your opinion about my use of this?
I have a very time critical application (yes, i know im using python at the moment, and I will reprogram the python part to c as soon I have more time, but now i have to finish this...). Would you suggest using this lua-python bridge or something like ZMQ(http://zeromq.org/)?
I have no idea if such a bridge like this has any overhead...

FOR EVERYONE ELSE ENCOUNTERING THIS PROBLEM:
this is how I resolved it:

I'm using Ubuntu 13.04 with Python 2.7
-> installed lua5.1 dev
-> installed lunatic-python-bugfix with pip
-> download lua5.1 source
-> open file 'loadlib.c' and change 'RTLD_NOW' to 'RTLD_NOW | RTLD_GLOBAL' at around line 70
-> compile lua5.1
-> open your compiled lua
-> execute in lua: print(package.loadlib("libpython2.7.so", "*"))
-> this should return true
-> execute in lua: require "lua-python"
-> execute in lua: python.execute("import string") to import what you need!

[greatwolf]

I have ZMQ somewhere and tried it briefly. From what I recall, that library's carefully designed to be as efficient as possible. Of course, like anything else, there is a learning curved involved unless you're already familiar with the library.

Lunatic python needs to translate from Lua -> C -> Python and back again whenever you access facilities from the other language. This could add up to quite a bit of overhead so I wouldn't recommend making too many lunatic calls in performance critical code. Having said that, the obvious advantage is you can access tools and facilities from the other language that otherwise wouldn't be possible. And if you're already comfortable using those facilities that can be a big win.

[barius]

@eloshoman @greatwolf

Hi there! I've tested your method and found out that one does not to have to add the RTLD_GLOBAL flag in the Lua distribution. All you have to do is

1. execute in lua: print(package.loadlib("libpython2.7.so", "*"))
2. execute in lua: require "lua-python"
   and it simply works :D
   I am going to using a embedded version of Lua on some mobile platform and cannot use a modified version, then I found this post. It's really helped me out. Thanks!
   BTW I'm on Ubuntu 13.10 with Python 2.7.5 and Lua 5.2

[greatwolf]

Glad you found it useful :)

[ghost]

Afaik, Lua5.2 is compiled with the GLOBAL tag by default.
Nice to know it actually helped someone =)