configure script cannot detect float word ordering on linux

[sourishbasu]

BPO	45477
Nosy	@tiran
Files	config.log conftest.o conftest.c conftest_unoptimized.o

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-10-14.21:19:53.235>
labels = ['build', 'expert-installation', '3.10']
title = 'configure script cannot detect float word ordering on linux'
updated_at = <Date 2021-10-22.22:49:31.555>
user = 'https://bugs.python.org/sourishbasu'

bugs.python.org fields:

activity = <Date 2021-10-22.22:49:31.555>
actor = 'sourish.basu'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Installation']
creation = <Date 2021-10-14.21:19:53.235>
creator = 'sourish.basu'
dependencies = []
files = ['50364', '50374', '50375', '50376']
hgrepos = []
issue_num = 45477
keywords = []
message_count = 12.0
messages = ['403951', '403978', '404096', '404135', '404138', '404447', '404448', '404449', '404450', '404459', '404463', '404836']
nosy_count = 2.0
nosy_names = ['christian.heimes', 'sourish.basu']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue45477'
versions = ['Python 3.10']

[sourishbasu]

I am trying to compile Python 3.10.0 on a Red Hat linux system with intel C/C++ compilers. This is my configure command:

./configure --with-computed-gotos --without-gcc --with-ensurepip=install --enable-optimizations --with-libm=-limf --with-cxx-main=icpc --with-threads --enable-ipv6 --with-signal-module CC=icc CXX=icpc LD=xild AR=xiar LIBS='-lpthread -limf -lirc' CFLAGS='-O3 -fPIC -fp-model strict -fp-model source -axCORE-AVX512,CORE-AVX2 -xAVX -ipo -prec-div -prec-sqrt' LDFLAGS='-ipo' CPP='icc -E'

The failure comes as follows:

checking for --with-libm=STRING... set LIBM="-limf"
checking for --with-libc=STRING... default LIBC=""
checking for x64 gcc inline assembler... yes
checking whether float word ordering is bigendian... unknown
configure: error:

Unknown float word ordering. You need to manually preset
ax_cv_c_float_words_bigendian=no (or yes) according to your system.

I saw bpo-38527 was fixed in 3.10. Perhaps fixing it on Solaris broke it on some other platforms?

[tiran]

Python uses the AX_C_FLOAT_WORDS_BIGENDIAN macro from autoconf-archive to detect byte order of floats. You can find the code at https://www.gnu.org/software/autoconf-archive/ax_c_float_words_bigendian.html .

Please attach your config.log file and the conftest.o output file of the follow command:

$ cat > conftest.c << EOF
double d = 90904234967036810337470478905505011476211692735615632014797120844053488865816695273723469097858056257517020191247487429516932130503560650002327564517570778480236724525140520121371739201496540132640109977779420565776568942592.0;
EOF
$ icc -c conftest.c

[sourishbasu]

See attached my config log.

When I tried to compile conftest.c with 'icc -c conftest.c', it produced 'conftest.o' without any errors.

[tiran]

Please build conftest.o with compiler flags

icc -c -O3 -fPIC -fp-model strict -fp-model source -axCORE-AVX512,CORE-AVX2 -xAVX -ipo -prec-div -prec-sqrt 

and attach the file.

[sourishbasu]

I executed:

$ icc -c -O3 -fPIC -fp-model strict -fp-model source -axCORE-AVX512,CORE-AVX2 -xAVX -ipo -prec-div -prec-sqrt conftest.c

and conftest.o was built without errors. It has one symbol, as expected:

$ nm conftest.o
                 U __must_be_linked_with_icc_or_xild
0000000000000000 D d

[tiran]

Please upload and attach the .o file to this bug. I need to see the content of the file.

[sourishbasu]

conftest.o attached

[sourishbasu]

conftest.c attached

[tiran]

Your conftest.o does not contain the magic string "seesnoon". Either ICC uses a different serialization format or your CFLAGS optimize the value away.

Please run

icc -c -o conftest_unoptimized.o conftest.c

and upload "conftest_unoptimized.o".

[sourishbasu]

conftest_unoptimized.o uploaded

[tiran]

One of your optimization flags breaks the AX_C_FLOAT_WORDS_BIGENDIAN macro. The optimized object file does not have seesnoon in its data section.

$ objdump -x conftest.o conftest_unoptimized.o 

conftest.o: file format elf64-x86-64
conftest.o
architecture: i386:x86-64, flags 0x00000011:
HAS_RELOC, HAS_SYMS
start address 0x0000000000000000

Sections:
Idx Name Size VMA LMA File off Algn
0 __ildata 000004eb 0000000000000000 0000000000000000 00000118 2**0
CONTENTS, READONLY, EXCLUDE
1 .data 00000008 0000000000000000 0000000000000000 00000608 2**3
CONTENTS, ALLOC, LOAD, RELOC, DATA
2 .note.GNU-stack 0000000 0000000000000000 0000000000000000 00000610 2**0
CONTENTS, READONLY
SYMBOL TABLE:
0000000000000000 l df *ABS* 0000000000000000 conftest.c
0000000000000000 l d __ildata 0000000000000000 __ildata
0000000000000000 l d .data 0000000000000000 .data
0000000000000000 l d .note.GNU-stack 0000000000000000 .note.GNU-stack
0000000000000000 g O .data 0000000000000008 d
0000000000000000 *UND* 0000000000000000 __must_be_linked_with_icc_or_xild

RELOCATION RECORDS FOR [.data]:
OFFSET TYPE VALUE
0000000000000000 R_X86_64_64 __must_be_linked_with_icc_or_xild

conftest_unoptimized.o: file format elf64-x86-64
conftest_unoptimized.o
architecture: i386:x86-64, flags 0x00000010:
HAS_SYMS
start address 0x0000000000000000

Sections:
Idx Name Size VMA LMA File off Algn
0 .data 00000008 0000000000000000 0000000000000000 000000f8 2**3
CONTENTS, ALLOC, LOAD, DATA
1 .bss 0000000 0000000000000000 0000000000000000 00000100 2**2
ALLOC
2 .text 0000000 0000000000000000 0000000000000000 00000100 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
3 .note.GNU-stack 0000000 0000000000000000 0000000000000000 00000100 2**0
CONTENTS, READONLY
SYMBOL TABLE:
0000000000000000 l df *ABS* 0000000000000000 conftest.c
0000000000000000 l d .data 0000000000000000 .data
0000000000000000 l d .bss 0000000000000000 .bss
0000000000000000 l d .text 0000000000000000 .text
0000000000000000 l d .note.GNU-stack 0000000000000000 .note.GNU-stack
0000000000000000 g O .data 0000000000000008 d

$ readelf -x .data conftest.o conftest_unoptimized.o 

File: conftest.o

Hex dump of section '.data':
NOTE: This section has relocations against it, but these have NOT been applied to this dump.
0x00000000 0000000 0000000 ........

File: conftest_unoptimized.o

Hex dump of section '.data':
0x00000000 73656573 6e6f6f6e seesnoon

[sourishbasu]

By enabling and disabling optimization flags one by one, I figured out it was '-ipo' (inter-procedural optimization) that was the problem. Without that flag, 'configure' is successful. However, now I have a 'make' error:

./Programs/_testembed.c(1774): error: "__builtin_types_compatible_p" is only allowed in C
config_set_argv(&config, Py_ARRAY_LENGTH(argv), argv);
^

compilation aborted for ./Programs/_testembed.c (code 2)
make[3]: *** [Programs/_testembed.o] Error 2

[kumaraditya303]

Closing as this is not a CPython issue but rather a compiler bug and icc is not supported as per PEP 11.

[damelang]

Author of the AX_C_FLOAT_WORDS_BIGENDIAN macro here. I committed a permanent fix to this issue in
Autoconf Archive (see above, commit 23be7cc). Update to latest and you should be able to close this issue once and for all.

[mikoxyz]

Pinging @tiran; without the updated macro, float word ordering won't be detected if e.g. -flto is present in one's CFLAGS.

[yan12125]

There is a CI job that verifies generated files (autoconf-related and some others) match those found on ubuntu 22.04:

cpython/.github/workflows/build.yml

Line 121 in 681778c

check_generated_files:

As a result, CPython may need to wait for a new autoconf-archive Ubuntu package that includes the fix (autoconf-archive/autoconf-archive#279)

[erlend-aasland]

Author of the AX_C_FLOAT_WORDS_BIGENDIAN macro here. I committed a permanent fix to this issue in Autoconf Archive (see above, commit 23be7cc). Update to latest and you should be able to close this issue once and for all.

Was this included in Autoconf 2.71, or must we upgrade to Autoconf 2.72?

[thesamesam]

It's in autoconf-archive, which is a separate collection of autoconf macros, namespaced AX_*.

[erlend-aasland]

It's in autoconf-archive, which is a separate collection of autoconf macros, namespaced AX_*.

Ah, yes of course. Unless Ubuntu 22.04 upgrades autoconf-archive, we'd have to explicitly force Tools/build/regen-configure.sh to use the new version. Our CI check is currently (and surprisingly) a lot more forgiving than it used to be, so I don't think we'd have to do anything with that.

[yan12125]

Unless Ubuntu 22.04 upgrades autoconf-archive, we'd have to explicitly force Tools/build/regen-configure.sh to use the new version

It may take months or even years to get fixed autoconf-archive into a Ubuntu LTS version. How about vendoring the latest ax_c_float_words_bigendian.m4 for now? For example: main...yan12125:cpython:issue89640

[damelang]

Unless Ubuntu 22.04 upgrades autoconf-archive, we'd have to explicitly force Tools/build/regen-configure.sh to use the new version

How about vendoring the latest ax_c_float_words_bigendian.m4 for now?
(followed by copy, paste, and renaming of code)

Although the license on my code permits this, I would really prefer you all take the force-to-newer-version approach. (If those are the only two options.)

The copy-paste-rename approach severs the link between the codebases. People have done copy-paste-rename with this very same code before, and it ends up causing headaches for maintainers and myself. I still get emails from people about some out-of-date / hacked up version of this code (often they don't even know where the code came from).

So, please, not the copy-paste-rename route?

[erlend-aasland]

So, please, not the copy-paste-rename route?

I agree, let's not take that route. Forcing the regen script to use the an updated version of autoconf-archive sounds like the best option so far. I'll see if I can find time for this the coming week.

[erlend-aasland]

FTR, Ubuntu 24.04 ships with autoconf-archive 20220903.

[erlend-aasland]

I committed a permanent fix to this issue in Autoconf Archive (see above, commit 23be7cc). Update to latest and you should be able to close this issue once and for all.

@damelang, ISTM there is no official autoconf-archive release containing this commit. Do you know when the next release is scheduled?

[damelang]

@damelang, ISTM there is no official autoconf-archive release containing this commit. Do you know when the next release is scheduled?

I don't. I think it's an "as needed" kind of thing. I'll ping the owner to let him know that there is need. Will report back.

[yan12125]

So, please, not the copy-paste-rename route?

Sorry for the bad example, I removed it

FTR, Ubuntu 24.04 ships with autoconf-archive 20220903.

That seems inherited from Debian, which is also on 20220903 https://tracker.debian.org/pkg/autoconf-archive

EDIT: found #115765 - maybe updated Debian/Ubuntu package will not be needed

[erlend-aasland]

EDIT: found #115765 - maybe updated Debian/Ubuntu package will not be needed

Yes, Donghee and I are working on a container (like we had until recently). This way, we can explicitly control the autoconf and autoconf-archive version.

[damelang]

@damelang, ISTM there is no official autoconf-archive release containing this commit. Do you know when the next release is scheduled?

I don't. I think it's an "as needed" kind of thing. I'll ping the owner to let him know that there is need. Will report back.

The owner (@peti) hasn't responded to my email from last week. Maybe he'll see this?