os.urandom() should use Linux 3.17 getrandom() syscall

[vstinner]

BPO	22181
Nosy	@pitrou, @vstinner, @tiran, @jwilk, @alex, @MojoVampire
Files	random.patch random-2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-03-30.11:42:45.852>
created_at = <Date 2014-08-10.23:32:43.376>
labels = ['type-security']
title = 'os.urandom() should use Linux 3.17 getrandom() syscall'
updated_at = <Date 2015-03-30.11:42:45.851>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2015-03-30.11:42:45.851>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-03-30.11:42:45.852>
closer = 'vstinner'
components = []
creation = <Date 2014-08-10.23:32:43.376>
creator = 'vstinner'
dependencies = []
files = ['38310', '38330']
hgrepos = []
issue_num = 22181
keywords = ['patch']
message_count = 23.0
messages = ['225171', '225363', '228250', '228639', '228645', '228655', '228785', '228790', '228791', '228792', '228794', '228795', '228847', '228850', '237102', '237190', '238440', '238504', '238559', '238568', '238676', '238688', '239585']
nosy_count = 11.0
nosy_names = ['pitrou', 'vstinner', 'christian.heimes', 'jwilk', 'Arfrever', 'alex', 'neologix', 'python-dev', 'anand.jeyahar', 'josh.r', '700eb415']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue22181'
versions = ['Python 3.5']

[vstinner]

The future Linux kernel 3.17 will have a new getrandom() syscall which avoids the need of a file descriptor:
http://lwn.net/Articles/606141/

The file descriptor of os.urandom() causes perfomance issues and surprising bugs: bpo-18756, bpo-21207.

I don't know when the function will land in the libc.

OpenBSD 5.6 (not released yet) will also have a new getentropy() syscall.

For Python 2.7, see also the PEP-466 and the issue bpo-21305.

[vstinner]

Manual page of the OpenBSD getentropy() function:
http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2

LibreSSL didn't wait for the libc, search for getentropy_getrandom():
http://openbsd.cs.toronto.edu/cgi-bin/cvsweb/src/lib/libcrypto/crypto/getentropy_linux.c?rev=1.32&content-type=text/x-cvsweb-markup

The code is currently disabled with "#if 0". The syscall is directly used, the function doesn't handle the ENOSYS error.

See also this issue of the cryptography project, "Use getentropy(2) and getrandom(2) syscalls when available 1299":
pyca/cryptography#1299

[700eb415]

It's worth noting that LibreSSL has now enabled the blocked code. If anyone is interested, I would be willing to help port it.

[anandjeyahar]

Hi,
This will need latest kernel to develop, fix and test. I (on Debian 7) couldn't find the latest kernel, but picked up ubuntu kernel from here http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.17-rc7-utopic/. I picked up the latest i.e: linux-image-3.17.0-031700rc7-generic_3.17.0-031700rc7.201409281835_amd64.deb and installed manually, but couldn't find the getrandom() function call either in stdlib.h or linux/random.h.

Can anyone confirm it's availability in a kernel image (from some other distribution?).

[neologix]

Note that I'm not fussed about it: far from simplifying the code, it
will make it more complex, thus more error-prone.

[vstinner]

The Linux kernel 3.17 has been released with the new getrandom() syscall.

glibc request to implement the function in the C library:
https://sourceware.org/bugzilla/show_bug.cgi?id=17252
"Bug 17252 - getrandom and getentropy syscall"

It looks like nobody asks for it on the libc-alpha mailing list yet.

[tiran]

Let's not be early adopters here. I suggest we wait until glibc has a proper interface.

[700eb415]

OpenBSD already provides high quality pseudorandom numbers from arc4random(). I don't think this would make us "early adopters" since it has been around for some time on this platform.

It's also worth mentioning that getentropy() is not recommended in use for normal code as per stated in the man page. arc4random() is recommended, but there may be a reason the first poster has recommended getentropy()

[pitrou]

This issue is about Linux support. Does the glibc have arc4random? I can't find it on my Ubuntu 13.10 system.

[alex]

As I said on the other ticket, using arc4random() indiscriminately would be a very poor idea, on some platforms (such as OS X) arc4random() really does use ARC4, which means there are serious security concerns with it.

[700eb415]

While I agree it may not be wise to use arc4random() globally, OpenBSD is unlikely to create a duplicate interface since it's already available.

Python is currently unusable in chroots on that platform without reducing the security of the host partition by removing the nodev mount flag.

I feel like there must be a good solution to this.

[pitrou]

Since this is a Linux-specific issue (see the title), you should create a separate issue for OpenBSD support. Bonus points if you want to submit a patch as well :-)