AIX build fails with failure to get random numbers #81024

datalogics-robb · 2019-05-07T20:59:52Z

BPO	36843
Nosy	@vstinner, @tiran, @aixtools, @jamadden, @datalogics-robb
Files	config.log.gz urandom.c

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-05-27.23:51:27.509>
created_at = <Date 2019-05-07.20:59:51.712>
labels = ['invalid', 'build', '3.7']
title = 'AIX build fails with failure to get random numbers'
updated_at = <Date 2020-02-07.09:20:00.909>
user = 'https://github.com/datalogics-robb'

bugs.python.org fields:

activity = <Date 2020-02-07.09:20:00.909>
actor = 'Michael.Felt'
assignee = 'none'
closed = True
closed_date = <Date 2019-05-27.23:51:27.509>
closer = 'vstinner'
components = ['Build']
creation = <Date 2019-05-07.20:59:51.712>
creator = 'Robert Boehne'
dependencies = []
files = ['48317', '48322']
hgrepos = []
issue_num = 36843
keywords = []
message_count = 15.0
messages = ['341821', '341822', '341828', '341830', '341892', '341910', '341929', '341930', '341936', '342009', '342076', '342077', '342136', '343714', '361550']
nosy_count = 5.0
nosy_names = ['vstinner', 'christian.heimes', 'Michael.Felt', 'jmadden', 'Robert Boehne']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue36843'
versions = ['Python 3.7']

datalogics-robb · 2019-05-07T20:59:52Z

build fails with:
./python -E ../../Python-3.7.3/setup.py build
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python

datalogics-robb · 2019-05-07T21:00:30Z

robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ gmake
xlc_r -c -DNDEBUG -O -q64 -qlanglvl=extc99 -IObjects -IInclude -IPython -I. -I../../Python-3.7.3/Include -I/raid/checkouts-raid/robb/Python-2.7.15/Modules/zlib -DPy_BUILD_CORE -o Modules/_math.o ../../Python-3.7.3/Modules/_math.c
LIBPATH=/raid/checkouts-raid/robb/nepal/build-py37 CC='xlc_r' LDSHARED='Modules/ld_so_aix xlc_r -bI:Modules/python.exp -L/raid/checkouts-raid/robb/zlib-dl/Release/rs6000aix_64/lib -Wl,-blibpath:/opt/IBM/xlmass/8.1.3/lib/aix61:/opt/IBM/xlc/13.1.3/lib:/usr/lib:/lib:/opt/Python-3.7/lib -q64 ' OPT='-DNDEBUG -O' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ../../Python-3.7.3/setup.py build
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python

Makefile:626: recipe for target 'sharedmods' failed
gmake: *** [sharedmods] Error 1

tiran · 2019-05-07T21:20:04Z

Could you please use a debugger and step through _Py_HashRandomization_Init and pyurandom to see, where the initialization of the RNG is failing?

vstinner · 2019-05-07T21:26:57Z

Try to compress config.log to attach it. Or at least attach the output of "./configure" as a file.

I'm looking for HAVE_GETRANDOM, HAVE_GETRANDOM_SYSCALL, HAVE_GETENTROPY defines that you can find in pyconfig.h.

About /dev/urandom: does this device exist? Is your user allowed to read from it? For example, run "dd if=/dev/urandom of=random bs=1 count=1" command: does it fail?

datalogics-robb · 2019-05-08T15:41:39Z

from pyconfig.h:

/* Define to 1 if the getrandom() function is available */
/* #undef HAVE_GETRANDOM */

/* Define to 1 if the Linux getrandom() syscall is available */
/* #undef HAVE_GETRANDOM_SYSCALL */

/* Define to 1 if you have the <linux/random.h> header file. */
/* #undef HAVE_LINUX_RANDOM_H */

/* Define to 1 if you have the `getentropy' function. */
/* #undef HAVE_GETENTROPY */

vstinner · 2019-05-08T16:50:48Z

Ok, so Python uses /dev/urandom. Can you try to read a few bytes from it? Like 256 bytes. You can try my dd command.

datalogics-robb · 2019-05-08T18:55:32Z

Opening /dev/urandom seems to return -1

(dbx) print buffer
0x09001000a5f49380
(dbx) print size
24
(dbx) print raise
0
(dbx) step
stopped in dev_urandom at line 311 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
311 if (raise) {
(dbx) step
stopped in dev_urandom at line 378 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
378 fd = _Py_open_noraise("/dev/urandom", O_RDONLY);
(dbx) next
stopped in dev_urandom at line 379 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
379 if (fd < 0) {
(dbx) print fd
-1
(dbx) step
stopped in dev_urandom at line 380 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
380 return -1;
(dbx) step
stopped in dev_urandom at line 401 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
401 }
(dbx) step
stopped in pyurandom at line 519 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
519 }
(dbx) step
stopped in unnamed block in _Py_HashRandomization_Init at line 611 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
611 if (res < 0) {
(dbx) step
stopped in unnamed block in _Py_HashRandomization_Init at line 612 in file "/raid/checkouts-raid/robb/nepal/build-py37/../../Python-3.7.3/Python/bootstrap_hash.c" ($t1)
612 return _Py_INIT_USER_ERR("failed to get random numbers "
(dbx) continue
continue
^ unrecognized command
(dbx) cont
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python

datalogics-robb · 2019-05-08T19:09:56Z

The call to open("/dev/urandom", flags) is returning -1, and errno is set to 22, EINVAL - Invalid argument. could the flags be set incorrectly?

datalogics-robb · 2019-05-08T19:59:10Z

Reading a few bytes from /dev/urandom via dd:

robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ dd if=/dev/urandom bs=256 count=1
??S?(#L???????????]?B?^??8?f&?_|Vi??@??[joG>St??;?$?1?*??24???RD?"4??aҼ#???f??? ?} ??^z????????C?@?"?.^?gn??p?t?R*=Ӫ?8*?L?? 8??bu?߂???8<d?/yd٠u0$9L_?g?e?P2s?Թ??4<?qF??0'????! ?c????%Z?;Pɟo=CIb?1+0 records in. 1+0 records out. robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ dd if=/dev/urandom bs=256 count=1 )|???Cnc9UR???r?(?@w???}?1??I~b?b??CmN?1(??6N?U?(??n???:I@;??@/ N??3w?˫u??r>?Ŵ???y??٫K?.?~陵#5?t??h?e!?G@)B?R???(9kL????X??BUH5=?||??El???a????+????n@??V?c?N?^ͯ?y?n??]??$݇?Mp?,#???K?i8??????v??????`L?1+0 records in.
1+0 records out.
robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$ dd if=/dev/urandom bs=256 count=1
_d?(?n???1bRԺ{??I??"Ѱ"|???E
p??^??q#??
~yZ$u??e}?3?????????qw#?????{*T1+0 records in.nR)?-?B???x??{~hϼ????+?7۪شV????W??"?d?n??V??t??R(\YC?3=?p?-ӽ?m??P???A?M??68??????9Y?](??xn+?a?ܯ?????????W
1+0 records out.
robb@nepal:/raid/checkouts-raid/robb/nepal/build-py37$

vstinner · 2019-05-09T23:21:25Z

Robert Boehne: pyurandom() uses _Py_open_noraise("/dev/urandom", O_RDONLY) which uses O_CLOEXEC if available. If this flag available? Does it work?

Please try to build attached urandom.c.

Example on my Fedora 29:

open O_RDONLY succeeded
read(16) -> 16
open O_RDONLY | O_CLOEXEC succeeded
read(16) -> 16

datalogics-robb · 2019-05-10T16:06:40Z

It doesn't look good:

robb@nepal:/raid/checkouts-raid/robb/nepal$ xlc_r -q64 -O0 -g
-qlanglvl=extc1x -o urandom urandom.c

robb@nepal:/raid/checkouts-raid/robb/nepal$ ./urandom

open O_RDONLY failed

open O_RDONLY | O_CLOEXEC failed

robb@nepal:/raid/checkouts-raid/robb/nepal$ uname -a

AIX nepal 1 7 00FA7FB84C00

robb@nepal:/raid/checkouts-raid/robb/nepal$

On Thu, May 9, 2019 at 6:21 PM STINNER Victor <report@bugs.python.org>
wrote:

STINNER Victor <vstinner@redhat.com> added the comment:

Robert Boehne: pyurandom() uses _Py_open_noraise("/dev/urandom", O_RDONLY)
which uses O_CLOEXEC if available. If this flag available? Does it work?

Please try to build attached urandom.c.

Example on my Fedora 29:

open O_RDONLY succeeded
read(16) -> 16
open O_RDONLY | O_CLOEXEC succeeded
read(16) -> 16

----------
Added file: https://bugs.python.org/file48322/urandom.c

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue36843\>

datalogics-robb · 2019-05-10T16:15:01Z

I wonder if there's anyone with AIX 7 who can attempt to reproduce this. We have another AIX machine, but it is down for the moment. I would like to eliminate a problem on this machine as the cause.

vstinner · 2019-05-10T21:28:45Z

open O_RDONLY failed

Ah. That sounds like an issue on your machine or specific to AIX. I don't see what Python can do to support a platform where /dev/urandom doesn't work. Python really needs /dev/urandom at startup to initialize its "hash secret" to reduce the risk of DoS attack attack against dict.

https://python-security.readthedocs.io/vuln/cve-2012-1150_hash_dos.html

Maybe it's a permission issue. Maybe a libc issue. I don't know. But I suggest to close the issue and try to find help from AIX instead.

vstinner · 2019-05-27T23:51:27Z

I close the issue. Maybe contact Michael Felt to get help to debug your issue.

aixtools · 2020-02-07T09:20:01Z

FYI: I was contacted this week by someone with this problem.

The problem was resolved after they updated AIX (was 7100-04-00-0000).

Please note: any oslevel -s reporting six zeros at the end needs the SP that is released in parallel with the base.

datalogics-robb mannequin added 3.7 (EOL) end of life build The build process and cross-build labels May 7, 2019

vstinner closed this as completed May 27, 2019

vstinner added the invalid label May 27, 2019

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AIX build fails with failure to get random numbers #81024

AIX build fails with failure to get random numbers #81024

datalogics-robb mannequin commented May 7, 2019

datalogics-robb mannequin commented May 7, 2019

datalogics-robb mannequin commented May 7, 2019

tiran commented May 7, 2019

vstinner commented May 7, 2019

datalogics-robb mannequin commented May 8, 2019

vstinner commented May 8, 2019

datalogics-robb mannequin commented May 8, 2019

datalogics-robb mannequin commented May 8, 2019

datalogics-robb mannequin commented May 8, 2019

vstinner commented May 9, 2019

datalogics-robb mannequin commented May 10, 2019

datalogics-robb mannequin commented May 10, 2019

vstinner commented May 10, 2019

vstinner commented May 27, 2019

aixtools commented Feb 7, 2020

AIX build fails with failure to get random numbers #81024

AIX build fails with failure to get random numbers #81024

Comments

datalogics-robb mannequin commented May 7, 2019

datalogics-robb mannequin commented May 7, 2019

datalogics-robb mannequin commented May 7, 2019

tiran commented May 7, 2019

vstinner commented May 7, 2019

datalogics-robb mannequin commented May 8, 2019

vstinner commented May 8, 2019

datalogics-robb mannequin commented May 8, 2019

datalogics-robb mannequin commented May 8, 2019

datalogics-robb mannequin commented May 8, 2019

vstinner commented May 9, 2019

datalogics-robb mannequin commented May 10, 2019

datalogics-robb mannequin commented May 10, 2019

vstinner commented May 10, 2019

vstinner commented May 27, 2019

aixtools commented Feb 7, 2020