BASECFLAGS are not passed to module build line

[vaxhacker]

BPO	969718
Nosy	@malemburg, @loewis, @birkenfeld, @ronaldoussoren, @mdickinson, @marienz, @pitrou, @vstinner, @tarekziade, @jwilk, @ned-deily, @merwok, @ambv, @koobs
Superseder	bpo-36235: distutils.sysconfig.customize_compiler() overrides CFLAGS var with OPT var if CFLAGS env var is set
Files	distutils_opt_flag.patch: Preliminary Patch debian-sysconfig-flags.patch sysconfig-flags-minimal.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tarekziade'
closed_at = <Date 2019-03-18.17:34:44.770>
created_at = <Date 2004-06-09.15:56:47.000>
labels = ['type-bug', 'library']
title = 'BASECFLAGS are not passed to module build line'
updated_at = <Date 2019-03-18.17:34:44.769>
user = 'https://bugs.python.org/vaxhacker'

bugs.python.org fields:

activity = <Date 2019-03-18.17:34:44.769>
actor = 'vstinner'
assignee = 'tarek'
closed = True
closed_date = <Date 2019-03-18.17:34:44.770>
closer = 'vstinner'
components = ['Distutils']
creation = <Date 2004-06-09.15:56:47.000>
creator = 'vaxhacker'
dependencies = []
files = ['19750', '20689', '20698']
hgrepos = []
issue_num = 969718
keywords = ['patch']
message_count = 21.0
messages = ['21099', '21100', '21101', '21102', '93701', '121944', '127850', '127851', '128031', '128032', '128034', '128051', '128060', '128065', '128155', '128158', '128159', '200379', '200383', '200385', '338266']
nosy_count = 20.0
nosy_names = ['lemburg', 'loewis', 'georg.brandl', 'ronaldoussoren', 'mark.dickinson', 'vaxhacker', 'nyogtha', 'marienz', 'pitrou', 'vstinner', 'jyasskin', 'tarek', 'jwilk', 'ned.deily', 'eric.araujo', 'rpetrov', 'Arfrever', 'lambacck', 'lukasz.langa', 'koobs']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '36235'
type = 'behavior'
url = 'https://bugs.python.org/issue969718'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']

[vaxhacker]

The value of BASECFLAGS from
/prefix/lib/pythonver/config/Makefile is not present on
the compile command for modules being built by
distutils ("python setup.py build"). It seems that
only the value of OPT is passed along.

This is insufficient when BASECFLAGS contains
"-fno-static-aliasing", since recent versions of gcc
will emit incorrect (crashing) code if this flag is not
provided, when compiling certain modules (the mx
products from egenix, for example).

I did try to set CFLAGS in my environment, as directed
by documentation, but this also had zero effect on the
final build command.

[nyogtha]

Logged In: YES
user_id=1426882

This is still a bug in Python 2.4.2.

[loewis]

Logged In: YES
user_id=21627

I don't think I will do anything about this anytime soon, so
unassigning myself.

[marienz]

I'm seeing a variation of this bug in python 2.5.

As far as I can tell in python 2.4.3 on linux it passes BASECFLAGS and OPT, appending CFLAGS from the environment to that if set. In python 2.5 it passes CFLAGS from the Makefile (which is defined as $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)), or OPT and the CFLAGS from the environment if CFLAGS is set there (this change was made in revision 45232). That means that if you run setup.py with CFLAGS set they must include -fno-strict-aliasing if using python 2.5.

I think it would be preferable to prepend BASECFLAGS instead of OPT if CFLAGS is set in the environment. On my linux machine after building python 2.5 with CFLAGS set to "-O2 -march=athlon-xp" the Makefile has:

OPT= -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
BASECFLAGS= -fno-strict-aliasing
CFLAGS= $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)

If I run a setup.py with CFLAGS unset it runs:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC ...

Which is reasonable. If I run it with CFLAGS="-O2 -march=athlon-xp":

gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -O2 -march=athlon-xp -fPIC ...

Which misses -fno-strict-aliasing and still includes all the general flags that I'm trying to set through CFLAGS.

If it used BASECFLAGS from the Makefile instead of OPT it would be:

gcc -pthread -fno-strict-aliasing -O2 -march=athlon-xp -fPIC ...

Which is what I think is the desired result here.

[lambacck]

I am running into a problem related to this. I am attempting to cross
compile extensions but Fedora includes -march in the OPT variable. Since
there is no way to exclude the OPT values the build fails.

It seems that forcing OPT to stay the same is a disservice generally
speaking since as Marien said you potentially get extra conflicting
flags (-O seems like a stand-out since that could mess with gdb).

2.6 and 3.1 say:
if 'CFLAGS' in os.environ:
cflags = opt + ' ' + os.environ['CFLAGS']
ldshared = ldshared + ' ' + os.environ['CFLAGS']

There is no ability to override opt. I'd be willing to put together a
patch to allow opt to be overridden by an environment variable if I
could get some support for this from a core maintainer.

I am also open to other suggestions about how to get around this. I
potentially have time to put into fixing this (or the more general cross
compile issue).

[lambacck]

I am attaching a preliminary patch to allow override of $(OPT). I am not sure this is sufficient, but am wary of breaking packages that depend on the existing behaviour.

The logic indeed seems wrong, but maybe this is something that has to go in distutils2 rather than distutils.

[merwok]

Pythons in Debian seem to be immune to this problem, thanks to this patch: http://deb.li/3ku1g (via http://lists.debian.org/debian-python/2010/12/msg00005.html).

I haven’t had time to learn the intricacies of make variables yet, so I can’t approve a patch. I’m adding people from the nosy lists of other make variables bugs, hopefully someone will be able to review.

[merwok]

Duplicate report from Stephan Krah:

When CFLAGS are set, distutils drops -fno-strict-aliasing (among other
flags):

$ python2.7 setup.py build
...
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./src -I/usr/local/include -I/usr/local/include/python2.7 -c src/gmpy.c -o build/temp.linux-x86_64-2.7/src/gmpy.o
...

$ CFLAGS="-fomit-frame-pointer" python2.7 setup.py build
...
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fomit-frame-pointer -fPIC -I./src -I/usr/local/include -I/usr/local/include/python2.7 -c src/gmpy.c -o build/temp.linux-x86_64-2.7/src/gmpy.o
src/gmpy.c: In function ‘_cmp_to_object’:
src/gmpy.c:4692: warning: dereferencing type-punned pointer will break strict-aliasing rules
...

I'm not sure if this is intentional. The documentation says:

"Compiler flags can also be supplied through setting the CFLAGS
environment variable. If set, the contents of CFLAGS will be added
to the compiler flags specified in the Setup file."

To me, this sounds as if they should be appended to the regular flags.

[skrah]

Éric, the Debian patch looks good to me and it solves my build problem.

The only question I have is why EXTRA_CFLAGS still go behind CFLAGS
and cannot be overridden via the environment.

But as it is, the patch is an improvement. I'm attaching the version
for 2.7.