Ubuntu / Debian - undefined Python symbols when loading native Python modules

[JonathanSalwan]

Looks like that on some distro (debian, ubuntu) there is a symbol issue when loading ctypes (and probably others modules) in a pintool.

jonathan@djobian:Triton$ cat ./src/examples/pin/ir.py
#!/usr/bin/env python2
## -*- coding: utf-8 -*-

import sys
import ctypes

from pintool import *
from triton  import *


def mycb(inst):
    print inst
    for expr in inst.getSymbolicExpressions():
        print expr
    print
    return


if __name__ == '__main__':
    # Set arch
    setArchitecture(ARCH.X86)

    # Start JIT at the entry point
    startAnalysisFromEntry()

    # Add callback
    addCallback(mycb, CALLBACK.BEFORE)

    # Run Program
    runProgram()

jonathan@djobian:Triton$ ./triton ./src/examples/pin/ir.py /usr/bin/id 
Traceback (most recent call last):
  File "./src/examples/pin/ir.py", line 5, in <module>
    import ctypes
  File "/usr/lib/python2.7/ctypes/__init__.py", line 10, in <module>
    from _ctypes import Union, Structure, Array
ImportError: /usr/lib/python2.7/lib-dynload/_ctypes.i386-linux-gnu.so: undefined symbol: PyFloat_Type

[JonathanSalwan]

On Debian/Ubuntu modules into /usr/lib/python2.7/lib-dynload/ are not linked with Python which is not a real problem using these modules into the Python interpreter.

On my Gentoo and Archlinux these modules are all linked with python.

$ ldd _ctypes.so
        linux-vdso.so.1 (0x00007ffe0b5fc000)
        libffi.so.6 => /usr/lib64/libffi.so.6 (0x00007ff02b3c1000)
        libpython2.7.so.1.0 => /usr/lib64/libpython2.7.so.1.0 (0x00007ff02aff2000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff02add6000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff02aa3b000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff02a837000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007ff02a634000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff02a331000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff02b7e8000)
$

I'm pretty sure this is the root cause of this behavior.

What I don't understand yet, is that libpintool.so is linked with libpython2.7.so, so, when ctypes.so is loaded it should resolve python's symbols.

If don't know yet, if it's due to Pin or ld.

[JonathanSalwan]

I confirm that the root cause is due to the non linking of modules with Python. I replaced my Gentoo ctypes.so with that of Ubuntu and it worked well.

$ sudo cp /home/jonathan/_ctypes.so /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so

The only difference between these two modules is that _ctypes.so is linked with libpython2.7.so.1.0.