test__xxsubinterpreters.ShareableTypeTests._assert_values fails on AIX - 32-bit mode

[aixtools]

BPO	34569
Nosy	@ericsnowcurrently, @aixtools
PRs	bpo-34569: Fix subinterpreter 32-bit ABI, pystate.c/_new_long_object() #9127

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-01-11.18:18:13.036>
created_at = <Date 2018-09-03.13:47:59.656>
labels = ['3.8', 'type-bug', 'tests']
title = 'test__xxsubinterpreters.ShareableTypeTests._assert_values fails on AIX - 32-bit mode'
updated_at = <Date 2019-01-11.18:18:13.035>
user = 'https://github.com/aixtools'

bugs.python.org fields:

activity = <Date 2019-01-11.18:18:13.035>
actor = 'eric.snow'
assignee = 'none'
closed = True
closed_date = <Date 2019-01-11.18:18:13.036>
closer = 'eric.snow'
components = ['Tests']
creation = <Date 2018-09-03.13:47:59.656>
creator = 'Michael.Felt'
dependencies = []
files = []
hgrepos = []
issue_num = 34569
keywords = ['patch']
message_count = 5.0
messages = ['324519', '324560', '325013', '333496', '333498']
nosy_count = 2.0
nosy_names = ['eric.snow', 'Michael.Felt']
pr_nums = ['9127']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34569'
versions = ['Python 3.8']

[aixtools]

+364 def _assert_values(self, values):
+365 for obj in values:
+366 with self.subTest(obj):
+367 interpreters.channel_send(self.cid, obj)
+368 got = interpreters.channel_recv(self.cid)
+369
+370 self.assertEqual(got, obj)
+371 self.assertIs(type(got), type(obj))
+372 # XXX Check the following in the channel tests?
+373 #self.assertIsNot(got, obj)
+374

+395 def test_int(self):
+396 self._assert_values(range(-1, 258))
+397

The assert fails on -1 with:

======================================================================
FAIL: test_int (test.test__xxsubinterpreters.ShareableTypeTests) [-1]
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/data/prj/python/python3-3.8.0/Lib/test/test__xxsubinterpreters.py", line 371, in _assert_values
    self.assertEqual(got, obj)
AssertionError: 4294967295 != -1

Note that this value is the unsigned value for 32-bit int as -1

root@x066:[/data/prj/python/python3-3.8.0]grep 4294967295 /usr/include/sys/*.h
/usr/include/sys/limits.h:#define ULONG_MAX (4294967295UL)
/usr/include/sys/limits.h:#define UINT_MAX (4294967295U)
/usr/include/sys/stdint.h:#define UINT32_MAX (4294967295U)

After quite a lot of "learning", I narrow the issue to:

+1432 static int
+1433 _long_shared(PyObject *obj, _PyCrossInterpreterData *data)
+1434 {
+1435 int64_t value = PyLong_AsLongLong(obj);
+1436 if (value == -1 && PyErr_Occurred()) {
+1437 if (PyErr_ExceptionMatches(PyExc_OverflowError)) {
+1438 PyErr_SetString(PyExc_OverflowError, "try sending as bytes");
+1439 }
+1440 return -1;
+1441 }
+1442 data->data = (void *)value;
+1443 data->obj = NULL;
+1444 data->new_object = _new_long_object;
+1445 data->free = NULL;
+1446 return 0;
+1447 }
+1448

+1426 static PyObject *
+1427 _new_long_object(_PyCrossInterpreterData *data)
+1428 {
+1429 return PyLong_FromLongLong((int64_t)(data->data));
+1430 }

The "value" is stored as a void data type, and the high-order 64-bits are zero. When it gets returned as a PyLong... it goes positive.

I do not dare touch anything here without some "mentoring".

Or, we change the test so that it knows it is in 32-bit mode, and compares with something else.

In short, "mentoring" requested.

p.s. not had time to test in 64-bit mode. Will post on that later.

[aixtools]

64-bit mode, no error.

root@x066:[/data/prj/python/python3-3.8.0]./python -m test -v test__xxsubinterpreters
== CPython 3.8.0a0 (heads/master-dirty:d500e5307a, Sep 3 2018, 13:55:44) [C]
== AIX-1-00C291F54C00-powerpc-64bit-COFF big-endian
== cwd: /data/prj/python/python3-3.8.0/build/test_python_16908532
== CPU count: 8
== encodings: locale=ISO8859-1, FS=iso8859-1
Run tests sequentially
0:00:00 [1/1] test__xxsubinterpreters
test_bad_id (test.test__xxsubinterpreters.ChannelIDTests) ... ok
...
test_int (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_singletons (test.test__xxsubinterpreters.ShareableTypeTests) ... ok
test_types (test.test__xxsubinterpreters.ShareableTypeTests) ... ok

----------------------------------------------------------------------
Ran 111 tests in 4.572s

OK (skipped=5)

== Tests result: SUCCESS ==

[ericsnowcurrently]

Thanks for bringing this up, Michael. I'll give you a review on the PR sometime this week (while at the core sprint).

[ericsnowcurrently]

New changeset a909460 by Eric Snow (Michael Felt) in branch 'master':
bpo-34569: Fix subinterpreter 32-bit ABI, pystate.c/_new_long_object() (gh-9127)
a909460

[ericsnowcurrently]

Thanks, Michael.