pyurandom() fails if user does not have an entropy device

[pehdrah]

BPO	35127
Nosy	@tiran, @koobs, @pehdrah

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-10-31.19:06:46.752>
labels = ['type-bug', '3.7']
title = 'pyurandom() fails if user does not have an entropy device'
updated_at = <Date 2018-10-31.19:51:22.938>
user = 'https://github.com/pehdrah'

bugs.python.org fields:

activity = <Date 2018-10-31.19:51:22.938>
actor = 'pablogsal'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['FreeBSD']
creation = <Date 2018-10-31.19:06:46.752>
creator = 'pehdrah'
dependencies = []
files = []
hgrepos = []
issue_num = 35127
keywords = []
message_count = 1.0
messages = ['329005']
nosy_count = 3.0
nosy_names = ['christian.heimes', 'koobs', 'pehdrah']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35127'
versions = ['Python 3.7']

[pehdrah]

Because of [reasons], I installed a FreeBSD and recompiled it to provide a minimalist kernel. Thus, it does not have the entropy devices /dev/urandom and /dev/random

It is a FreeBSD-10-0 running in a Hyper-V virutalization platform.

I kinda recompiled python-3.7.1 from the source code to get my own version of python running, but I got stuck in this part. So looking at the Modules/bootstrap_hash.c file I do not see the code of pyurandom() falling back to a something silly.

Plus, there is a problem that happens only in Unix at least in this version of python. Looking at the code of the same file. we can see:

static int pyurandom(...)
{
...
int res
...
if (res < 0){
  return -1
}
if (res == 1){
  return 0
}
...
}

I am sorry for the laziness but I believe my point is clear.

The thing is. If the random function returns 0 it will return 0 but if it returns 1 it will also return 0.

In other words, the number 1 is out of the scope of the random numbers. It is a small prejudice for the random function, but it is not mathematically right. This issue with the number 1, does not happen if the user does not have /dev/random (But it still needs /dev/urandom)

[iritkatriel]

My reading of the code is that res is a success/failure indicator, and the random number is returned in the buffer (first arg).

[vstinner]

The code works as expected. Thanks for closing the issue @iritkatriel.

I modified Python locally to make sure that opening /dev/urandom fails: I replaced /dev/urandom with /xdev/urandom. I also removed the code calling py_getrandom() or py_getentropy().

So looking at the Modules/bootstrap_hash.c file I do not see the code of pyurandom() falling back to a something silly.

With my changes, Python fails at startup with a fatal error as expected:

Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python
Python runtime state: preinitialized

The thing is. If the random function returns 0 it will return 0 but if it returns 1 it will also return 0.

Your comment is about py_getrandom() / py_getentropy() API which has a different API than pyurandom().

pyurandom() returns 0 on success:

/* Read random bytes:

   - Return 0 on success
   - Raise an exception (if raise is non-zero) and return -1 on error

   Used sources of entropy ordered by preference, preferred source first:

   - BCryptGenRandom() on Windows
   - getrandom() function (ex: Linux and Solaris): call py_getrandom()
   - getentropy() function (ex: OpenBSD): call py_getentropy()
   - /dev/urandom device

   Read from the /dev/urandom device if getrandom() or getentropy() function
   is not available or does not work.

   Prefer getrandom() over getentropy() because getrandom() supports blocking
   and non-blocking mode: see the PEP 524. Python requires non-blocking RNG at
   startup to initialize its hash secret, but os.urandom() must block until the
   system urandom is initialized (at least on Linux 3.17 and newer).

   Prefer getrandom() and getentropy() over reading directly /dev/urandom
   because these functions don't need file descriptors and so avoid ENFILE or
   EMFILE errors (too many open files): see the issue #18756.

   Only the getrandom() function supports non-blocking mode.

   Only use RNG running in the kernel. They are more secure because it is
   harder to get the internal state of a RNG running in the kernel land than a
   RNG running in the user land. The kernel has a direct access to the hardware
   and has access to hardware RNG, they are used as entropy sources.

   Note: the OpenSSL RAND_pseudo_bytes() function does not automatically reseed
   its RNG on fork(), two child processes (with the same pid) generate the same
   random numbers: see issue #18747. Kernel RNGs don't have this issue,
   they have access to good quality entropy sources.

   If raise is zero:

   - Don't raise an exception on error
   - Don't call the Python signal handler (don't call PyErr_CheckSignals()) if
     a function fails with EINTR: retry directly the interrupted function
   - Don't release the GIL to call functions.
*/
static int
pyurandom(void *buffer, Py_ssize_t size, int blocking, int raise)

py_getrandom() and py_getentropy() returns 1 on success:

/* Call getrandom() to get random bytes:

   - Return 1 on success
   - Return 0 if getrandom() is not available (failed with ENOSYS or EPERM),
     or if getrandom(GRND_NONBLOCK) failed with EAGAIN (system urandom not
     initialized yet) and raise=0.
   - Raise an exception (if raise is non-zero) and return -1 on error:
     if getrandom() failed with EINTR, raise is non-zero and the Python signal
     handler raised an exception, or if getrandom() failed with a different
     error.

   getrandom() is retried if it failed with EINTR: interrupted by a signal. */
static int
py_getrandom(void *buffer, Py_ssize_t size, int blocking, int raise)


/* Fill buffer with size pseudo-random bytes generated by getentropy():

   - Return 1 on success
   - Return 0 if getentropy() syscall is not available (failed with ENOSYS or
     EPERM).
   - Raise an exception (if raise is non-zero) and return -1 on error:
     if getentropy() failed with EINTR, raise is non-zero and the Python signal
     handler raised an exception, or if getentropy() failed with a different
     error.

   getentropy() is retried if it failed with EINTR: interrupted by a signal. */
static int
py_getentropy(char *buffer, Py_ssize_t size, int raise)