OverflowError: signed integer is greater than maximum on mips64

[jasper]

BPO	7296
Nosy	@mdickinson
Files	config.log: configure log file. Python-2.6.3.log: Build log Debug_build.log: Build log with -O0

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-02-08.20:59:34.014>
created_at = <Date 2009-11-09.23:02:13.146>
labels = ['build']
title = 'OverflowError: signed integer is greater than maximum on mips64'
updated_at = <Date 2010-09-22.13:41:42.254>
user = 'https://bugs.python.org/jasper'

bugs.python.org fields:

activity = <Date 2010-09-22.13:41:42.254>
actor = 'jasper'
assignee = 'none'
closed = True
closed_date = <Date 2010-02-08.20:59:34.014>
closer = 'mark.dickinson'
components = ['Build']
creation = <Date 2009-11-09.23:02:13.146>
creator = 'jasper'
dependencies = []
files = ['15300', '15301', '15307']
hgrepos = []
issue_num = 7296
keywords = []
message_count = 11.0
messages = ['95098', '95099', '95103', '95118', '95120', '95121', '95123', '95231', '99063', '99068', '117140']
nosy_count = 3.0
nosy_names = ['mark.dickinson', 'jasper', 'omoerbeek']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue7296'
versions = ['Python 2.6']

[jasper]

While trying to get Python 2.6 working on OpenBSD/sgi (64-bit port)
I ran into the following during build:

OverflowError: signed integer is greater than maximum

I ran the command that triggered this by hand with -v added:

(sgi Python-2.6.3 40)$ export PATH; PATH="`pwd`:$PATH";  export
PYTHONPATH; PYTHONPATH="`pwd`/Lib";  export DYLD_FRAMEWORK_PATH;
DYLD_FRAMEWORK_PATH="`pwd`";  export EXE; EXE="";  cd
./Lib/plat-openbsd4;  ./regen
python$EXE -v ../../Tools/scripts/h2py.py -i '(u_long)'
/usr/include/netinet/in.h
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc matches
/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py
import site # precompiled from
/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.pyc
'import site' failed; traceback:
Traceback (most recent call last):
  File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/site.py", line 61,
in <module>
    import sys
OverflowError: signed integer is greater than maximum
import encodings # directory
/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings
# /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc
matches /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.py
import encodings # precompiled from
/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/encodings/__init__.pyc
Python 2.6.3 (r263:75183, Nov  6 2009, 09:50:33) 
[GCC 3.3.5 (propolice)] on openbsd4
Type "help", "copyright", "credits" or "license" for more information.
# /usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc matches
/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py
import re # precompiled from
/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.pyc
Traceback (most recent call last):
  File "../../Tools/scripts/h2py.py", line 24, in <module>
    import sys, re, getopt, os
  File "/usr/obj/ports/Python-2.6.3/Python-2.6.3/Lib/re.py", line 104,
in <module>
    import sys
OverflowError: signed integer is greater than maximum
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] zipimport
# cleanup[1] signal
# cleanup[1] exceptions
# cleanup[1] _warnings
# cleanup sys
# cleanup __builtin__
# cleanup ints: 3 unfreed ints
# cleanup floats
(sgi plat-openbsd4 41)$

There have been several patches applied:
http://www.openbsd.org/cgi-bin/cvsweb/ports/lang/python/2.6/patches/
Although none seem to be relevant as far as I can see.

Please find attached the build log and the configure log.

[jasper]

And the build log on OpenBSD/sgi.

[mdickinson]

Thanks for filing the report! Some questions:

If you configure with the --with-pydebug option, and also do whatever
else (if anything) is necessary to remove the -O2 flag from the
compilation steps, does the build failure still occur?

What's the minimal Python code required to cause the failure. Is it
enough to launch the interpreter and then just do 'import sys'?

Judging by the error message, it looks as though the OverflowError is
being set in the 'convertsimple' function in Python/getargs.c: the
relevant code looks something like:

	case 'i': {/* signed int */
		int *p = va_arg(*p_va, int *);
		long ival;
		if (float_argument_error(arg))
			return converterr("integer<i>", arg, msgbuf, bufsize);
		ival = PyInt_AsLong(arg);
		if (ival == -1 && PyErr_Occurred())
			return converterr("integer<i>", arg, msgbuf, bufsize);
		else if (ival > INT_MAX) {
			PyErr_SetString(PyExc_OverflowError,
				"signed integer is greater than maximum");
			return converterr("integer<i>", arg, msgbuf, bufsize);
		}

But this code is part of Python's general argument parsing mechanism, so
is called from many many places; we really need some way of figuring out
where it's getting called from when the build fails. Still with a
--with-pydebug build, could you try using gdb (or an equivalent) to set
a breakpoint on the PyErr_SetString line in the (ival > INT_MAX) branch,
then do whatever is required to trigger the failure and report the
backtrace at that breakpoint?

[jasper]

After properly compiling with -O0, it actually gets a lot further in the
build. It crashes elsewhere though:

PYTHONPATH=/usr/obj/ports/Python-2.6.3/fake-sgi/usr/local/lib/python2.6
./python -Wi -tt
/usr/obj/ports/Python-2.6.3/fake-sgi/usr/local/lib/python2.6/compileall.py
-d /usr/local/lib/python2.6 -f -x 'bad_coding|badsyntax|site-packages'
/usr/obj/ports/Python-2.6.3/fake-sgi/usr/local/lib/python2.6
Floating point exception (core dumped)

Attached is the full build log with the backtrace of that core file.

[mdickinson]

Hmm. I don't understand that backtrace at all. It seems to say that
the conversion of this particular double value (2.34e17) to long is
causing some kind of arithmetic exception. I'd assume overflow, except
that the configure script says sizeof(long) == 8, and a 64-bit long
should be plenty large enough to hold the result of the conversion.

Is it possible that the configure script is somehow ending up with the
wrong value for SIZEOF_LONG? Or do C longs definitely have width 64 on
this platform?

[jasper]

this little test program:
#include <unistd.h>
int main(int argc, char*argv[])
{
  printf("short = %d\n", sizeof(short));
  printf("int = %d\n", sizeof(int));
  printf("float = %d\n", sizeof(float));
  printf("long = %d\n", sizeof(long));
  printf("double = %d\n", sizeof(double));
  printf("long long = %d\n", sizeof(long long));
  printf("double long = %d\n", sizeof(double long));
  return 0;
}
gives the following values on mips64:
short = 2
int = 4
float = 4
long = 8
double = 8
long long = 8
double long = 16

is there any other thing I should check?

[mdickinson]

Sorry, I'm running out of ideas. The traceback is truly baffling.

I'm not sure why you're configuring with --with-fpectl. Does removing
this make any difference?

Maybe you could also try copying _PyHash_Double into a small test program,
calling it with an input of 2.34e17 (get rid of the Python-specific calls
in the if-branch, which I believe isn't taken in this case anyway) and see
if you can reproduce the FP signal there.

[jasper]

Removing --with-fpectl makes no difference.

I'll try the _PyHash_Double-thing later this weekend.

[omoerbeek]

This problem has been diagnosed as a problem in the mips64 port of OpenBSD. mips64 systems need to emulate some floating point instructions in software, depending on the cpu type. In this case we hit an instruction for which the emulation was incomplete. The floating point exception actually signals this.

Work is being done to fix this.

[mdickinson]

Many thanks for the update. I'll close this as not a Python bug, then.
(Unless there's an easy and nonintrusive workaround...)

[jasper]

FYI, the issue has been fixed now in the mips64 port of OpenBSD by "replacing the previous/old floating point completion code with a C interface to the MI softfloat code, implementing all MIPS IV specified floating point
operations."