ImportError: dlopen failed: cannot locate symbol "PyBool_Type"

[muhzii]

BPO	35997
Nosy	@ezio-melotti, @matrixise, @moreati, @muhzii
Files	xcompile-py3.sh

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-02-14.20:36:32.866>
labels = ['extension-modules', 'type-bug', '3.7', 'build']
title = 'ImportError: dlopen failed: cannot locate symbol "PyBool_Type"'
updated_at = <Date 2019-12-10.08:07:15.936>
user = 'https://github.com/muhzii'

bugs.python.org fields:

activity = <Date 2019-12-10.08:07:15.936>
actor = 'xdegaye'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules', 'Cross-Build']
creation = <Date 2019-02-14.20:36:32.866>
creator = 'muhzi'
dependencies = []
files = ['48140']
hgrepos = []
issue_num = 35997
keywords = []
message_count = 8.0
messages = ['335560', '335563', '335565', '335567', '335568', '335570', '335625', '335665']
nosy_count = 5.0
nosy_names = ['ezio.melotti', 'mrabarnett', 'matrixise', 'Alex.Willmer', 'muhzi']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35997'
versions = ['Python 3.7']

[muhzii]

I cross compiled python for android x86_64, and the interpreter works fine, no problems. But when I compiled some other extension and try to import it. I get an import error as such the imported shared library fails to locate the symbol "PyBool_Type".

ImportError: dlopen failed: cannot locate symbol "PyBool_Type" referenced by ....

The extension was compiled with -I && -L flags pointing to the Python installation include and lib folders.

[matrixise]

Can you see the symbols when you use nm or readelf?

[matrixise]

if you have compiled python with --enable-shared you will get a lib/libpython3.Xdm.so file and you can use nm -D lib/libpython3.Xdm.so | grep PyBool_Type

nm --debug-syms lib/libpython3.8dm.so | grep PyBool 0000000000077696 T PyBool_FromLong
0000000000328d20 D PyBool_Type

I think it's a problem with your compilation.

Could you explain how do you cross-compile for android?

Thank you

[muhzii]

Yes, the symbols are there.
And here is how I cross compiled Python:

export CC="${ANDROID_TARGET}${ANDROID_API}-clang"
export CXX="$CC++"

# environment variables for binary utils..
export AR="$BIN_UTILS_PREFIX-ar"
export LD="$BIN_UTILS_PREFIX-ld"
export AS="$BIN_UTILS_PREFIX-as"
export STRIP="$BIN_UTILS_PREFIX-strip"
export RANLIB="$BIN_UTILS_PREFIX-ranlib"
export READELF="$BIN_UTILS_PREFIX-readelf"

# compiler and linker flags
export CFLAGS="-fPIE -fPIC"
export LDFLAGS="-fuse-ld=lld -Wl,--rpath=../lib"

export CONFIG_SITE="config.site"

CONFIG_BUILD="$(uname -m)-linux-gnu"
CONFIG_ARGS="--enable-shared --disable-ipv6"
CONFIG_PREFIX="$PYTHON_PATH/output/$ARCH-android"

$PYTHON_PATH/configure --host=$ANDROID_TARGET --build=$CONFIG_BUILD --prefix=$CONFIG_PREFIX $CONFIG_ARGS
make -C $PYTHON_PATH
make -C $PYTHON_PATH install

[matrixise]

Thank you, but I think there is missing info with your script.

Do you have a shell script to share? What are the values of:

ANDROID_TARGET
ANDROID_API
CC++
BIN_UTILS_PREFIX
etc...

For example, if I want to use a docker image on my Fedora, how can I compile Python for Android from my Fedora or Debian x86?

Thank you

[muhzii]

OK, I uploaded it. I give the path for NDK then it determines those values for the target architecture, in my case I use it for x86_64.

[xdegaye]

It is not yet possible to cross-compile python extensions. The reason is that _init_posix() in Lib/sysconfig.py still reads the sysconfigdata module of the native compiler instead of the sysconfigdata of the cross-built one. See the patch in issue bpo-28833.

[muhzii]

Ah! I used the option -Bsymbolic while linking the extension to make it work and it did, but I figure it is not ideal. I will be checking that patch!