libffi assembler relocation check is not robust, fails with clang

[donmez]

BPO	11729
Nosy	@doko42, @pitrou, @benjaminp, @merwok, @skrah, @koobs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/skrah'
closed_at = <Date 2013-01-16.22:35:47.493>
created_at = <Date 2011-03-31.12:38:37.839>
labels = ['type-bug', 'build']
title = 'libffi assembler relocation check is not robust, fails with clang'
updated_at = <Date 2013-04-30.08:05:39.172>
user = 'https://bugs.python.org/donmez'

bugs.python.org fields:

activity = <Date 2013-04-30.08:05:39.172>
actor = 'python-dev'
assignee = 'skrah'
closed = True
closed_date = <Date 2013-01-16.22:35:47.493>
closer = 'skrah'
components = ['Build']
creation = <Date 2011-03-31.12:38:37.839>
creator = 'donmez'
dependencies = []
files = []
hgrepos = []
issue_num = 11729
keywords = []
message_count = 21.0
messages = ['132663', '132675', '132676', '132677', '132679', '132680', '135233', '135234', '135236', '136639', '180103', '180105', '180114', '180115', '180182', '180184', '180185', '180243', '180771', '180776', '188141']
nosy_count = 9.0
nosy_names = ['doko', 'pitrou', 'donmez', 'benjamin.peterson', 'eric.araujo', 'Arfrever', 'skrah', 'python-dev', 'koobs']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue11729'
versions = ['Python 3.3', 'Python 3.4']

[donmez]

The check under Modules/_ctypes/libffi/configure.ac does;

echo '.text; foo: nop; .data; .long foo-.; .text' > conftest.s
if $CC $CFLAGS -c conftest.s 2>&1 | grep -i warning > /dev/null; then
    libffi_cv_as_x86_pcrel=no
fi  

With clang; we get:

clang: warning: argument unused during compilation: '-g'

So it fails to detect pc relocation support due to an unrelated warning.

[benjaminp]

I suggest you complain to the libffi maintainers.

[donmez]

This is already fixed in upstream, check has been changed into:

libffi_cv_as_x86_pcrel=no
echo '.text; foo: nop; .data; .long foo-.; .text' > conftest.s
if $CC $CFLAGS -c conftest.s > /dev/null; then
libffi_cv_as_x86_pcrel=yes
fi
])

[Arfrever]

You can install the newest libffi and pass --with-system-ffi option to main configure of Python.

[donmez]

Yes I can workaround it but I'd like to get it fixed inside Python too ;)

[Arfrever]

Internal copies of third-party libraries are rather always outdated, so it's better to always use system libraries. IMHO Python shouldn't bundle any third-party libraries and should force users to use system libraries. At least, system libraries could be used by default and main configure could accept --with-internal-expat and --with-internal-libffi options.

[donmez]

Ping? Shall I submit a fix for Modules/_ctypes/libffi/configure.ac or better yet will you sync in-tree libffi?

[pitrou]

Well, ctypes is kindof unmaintained right now, and the private libffi copy belongs to that module.
(the reason we have a libffi copy is that, IIUC, it's patched)

[donmez]

Patch for configure.ac then?

[merwok]

See also bpo-12081.

[skrah]

In 3.3 libffi has been updated to 3.0.11. Our clang buildbot does not
show this particular warning, but still fails to compile libffi:

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%203.3/builds/320/steps/compile/logs/stdio

It would help if you provided a link to the upstream patch.

[donmez]

See http://sourceware.org/ml/libffi-discuss/2011/msg00024.html for the libffi patch.

[python-dev]

New changeset 190a115b7748 by Stefan Krah in branch '3.3':
Issue bpo-11729: Backport commit bff052d from libffi upstream in order to fix
http://hg.python.org/cpython/rev/190a115b7748

[skrah]

Thanks for the link. The diff was committed last week to the upstream
libffi development tree, so I backported it.

[koobs]

Stefan, can we merge this to the 2.7 branch as well please?

http://buildbot.python.org/all/builders/AMD64%20FreeBSD%209.0%20dtrace%2Bclang%202.7/builds/308/steps/compile/logs/stdio

[skrah]

koobs <report@bugs.python.org> wrote:

Stefan, can we merge this to the 2.7 branch as well please?

If Benjamin is okay with it, yes. The problem with these configure fixes
is that they might break other systems in unexpected ways. I already hesitated
to put it into 3.3, but the buildbots are fine.

[koobs]

Thanks for the quick response. I'd be happy for it to be FreeBSD conditional/specific if that's more suitable, safer?

Having said that, our buildbot OS coverage is pretty good, no?

[skrah]

The buildbot coverage is good, but the number of (OS, shell, compiler)
combinations is much higher.

[benjaminp]

Okay with me to backport.

[python-dev]

New changeset b9792b27d410 by Stefan Krah in branch '2.7':
Issue bpo-11729: Backport commit bff052d from libffi upstream in order to fix
http://hg.python.org/cpython/rev/b9792b27d410

[python-dev]

New changeset 7a2e0c1bb1a9 by Gregory P. Smith in branch '3.2':

• Fix bpo-17192 for 3.2 - reapply the bpo-11729 patch that was undone
  http://hg.python.org/cpython/rev/7a2e0c1bb1a9

New changeset aa3371fa9773 by Gregory P. Smith in branch '3.3':

• Fix PART of bpo-17192 for 3.3 - reapply the bpo-11729 patch
  http://hg.python.org/cpython/rev/aa3371fa9773

New changeset e0fdc21c7d13 by Gregory P. Smith in branch 'default':

• Fix PART of bpo-17192 for 3.4 - reapply the bpo-11729 patch
  http://hg.python.org/cpython/rev/e0fdc21c7d13