Building with static libffi issue (ffi_type_float)

[matejsp]

System Details

Amazon Linux 2023 changed libffi version and broke our package. We want to build cffi with static libffi.

Problems Description

docker run -it docker.bitstamp.net/amazonlinux/amazonlinux:2023.2.20230920.1 bash
dnf install -y wget gcc pkgconfig tar gzip python3-pip python3-devel 
cd /root
wget https://files.pythonhosted.org/packages/2b/a8/050ab4f0c3d4c1b8aaa805f70e26e84d0e27004907c5b8ecc1d31815f92a/cffi-1.15.1.tar.gz
wget https://github.com/libffi/libffi/releases/download/v3.4.4/libffi-3.4.4.tar.gz
rm -rf libffi-3.4.4 && tar -xvzf libffi-3.4.4.tar.gz && cd libffi-3.4.4 && ./configure --prefix=$PWD/build --enable-static --disable-shared && make && make install && cd ..
export PKG_CONFIG_PATH=$PWD/libffi-3.4.4/build/lib/pkgconfig
pip install wheel setuptools
pip wheel cffi-1.15.1.tar.gz

We get:

bash-5.2# pip wheel cffi-1.15.1.tar.gz
Processing ./cffi-1.15.1.tar.gz
  File was already downloaded /root/cffi-1.15.1.tar.gz
  Preparing metadata (setup.py) ... done
Collecting pycparser
  File was already downloaded /root/pycparser-2.21-py2.py3-none-any.whl
Building wheels for collected packages: cffi
  Building wheel for cffi (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-iu1uy_lc/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-iu1uy_lc/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-h8m2o66r
       cwd: /tmp/pip-req-build-iu1uy_lc/
  Complete output (50 lines):
  /usr/local/lib/python3.9/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
  !!
  
          ********************************************************************************
          The license_file parameter is deprecated, use license_files instead.
  
          By 2023-Oct-30, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.
  
          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************
  
  !!
    parsed = self.parsers.get(option_name, lambda x: x)(value)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-39
  creating build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/cparser.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/commontypes.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/error.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/lock.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/vengine_gen.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/verifier.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/recompiler.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/__init__.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/api.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/pkgconfig.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/ffiplatform.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/model.py -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/_cffi_include.h -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/parse_c_type.h -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/_embedding.h -> build/lib.linux-x86_64-cpython-39/cffi
  copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-cpython-39/cffi
  running build_ext
  building '_cffi_backend' extension
  creating build/temp.linux-x86_64-cpython-39
  creating build/temp.linux-x86_64-cpython-39/c
  gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -ftree-vectorize -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -ftree-vectorize -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -ftree-vectorize -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/root/libffi-3.4.4/build/include -I/usr/include/python3.9 -c c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-39/c/_cffi_backend.o
  gcc -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g build/temp.linux-x86_64-cpython-39/c/_cffi_backend.o -L/root/libffi-3.4.4/build/lib/../lib64 -L/usr/lib64 -lffi -o build/lib.linux-x86_64-cpython-39/_cffi_backend.cpython-39-x86_64-linux-gnu.so
  /usr/bin/ld: /root/libffi-3.4.4/build/lib/../lib64/libffi.a(prep_cif.o): relocation R_X86_64_32S against symbol `ffi_type_float' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: failed to set dynamic section sizes: bad value
  collect2: error: ld returned 1 exit status
  error: command '/usr/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for cffi
  Running setup.py clean for cffi
Failed to build cffi
ERROR: Failed to build one or more wheels
bash-5.2# 

I saw other issue and they were fixed for other symbols (but not for ffi_type_float).
#393

[matejsp]

It does not work:

./configure --prefix=$PWD/build --enable-static 

But it does work:

LDFLAGS=-fPIC CFLAGS=-fPIC ./configure --prefix=$PWD/build --enable-static --disable-shared

Just not sure that this is correct way.

[matejsp]

I seems that I need to run as ./configure --without-gcc-arch --disable-docs --with-pic --enable-shared=no && \
Now question is why is this not enabled by default for static lib.

[atgreen]

This is because you are linking the static libffi code to a shared library -- the libffi code still needs to be position-independent. If you were just linking it to a static executable it wouldn't require -fPIC. I don't think I want to change this default behaviour, but if you want to suggest a change to the README via PR, that would be welcome.

See also https://stackoverflow.com/questions/3961446/why-does-gcc-not-implicitly-supply-the-fpic-flag-when-compiling-static-librarie