Use gnu_get_libc_version() in platform.libc_ver()?

[vstinner]

BPO	35389
Nosy	@malemburg, @vstinner, @jwilk, @serhiy-storchaka
PRs	bpo-35389: platform.libc_ver() uses os.confstr() #10891 bpo-35389: test.pythoninfo logs platform.libc_ver #10951 bpo-35389: platform.platform() calls libc_ver() without executable #14418

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-12-05.13:53:18.228>
created_at = <Date 2018-12-03.16:02:13.435>
labels = ['3.8', 'library']
title = 'Use gnu_get_libc_version() in platform.libc_ver()?'
updated_at = <Date 2019-06-27.07:04:39.140>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2019-06-27.07:04:39.140>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-12-05.13:53:18.228>
closer = 'vstinner'
components = ['Library (Lib)']
creation = <Date 2018-12-03.16:02:13.435>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 35389
keywords = ['patch']
message_count = 11.0
messages = ['330952', '330953', '330956', '331036', '331041', '331109', '331117', '331124', '331126', '331185', '346713']
nosy_count = 4.0
nosy_names = ['lemburg', 'vstinner', 'jwilk', 'serhiy.storchaka']
pr_nums = ['10891', '10951', '14418']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35389'
versions = ['Python 3.8']

[vstinner]

Currently, platform.libc_ver() opens Python binary file (ex: /usr/bin/python3) and looks for a string like "GLIBC-2.28".

Maybe gnu_get_libc_version() should be exposed in Python to get the version of the running glibc version? And use it if available, or fall back on parsing the binary file (as done currenetly) otherwise.

Example:

$ cat x.c 
#include <gnu/libc-version.h>
#include <stdlib.h>
#include <stdio.h>

int
main(int argc, char *argv[])
{
    printf("GNU libc version: %s\n", gnu_get_libc_version());
    printf("GNU libc release: %s\n", gnu_get_libc_release());
    exit(EXIT_SUCCESS);
}

$ ./x
GNU libc version: 2.28
GNU libc release: stable

I'm not sure if it's possible that Python is compiled with glibc but run with a different libc implementation?

--

Alternative: run a program to get the libc version which *might* be different than the libc version of Python if the libc is upgraded in the meanwhile (unlikely, but is technically possible on a server running for days):

$ ldd --version
ldd (GNU libc) 2.28
...

$ /lib64/libc.so.6 
GNU C Library (GNU libc) stable release version 2.28.
...

$ rpm -q glibc
glibc-2.28-17.fc29.x86_64
...

etc.

--

See also discussions on platform.libc_ver() performance:
#10868

[jwilk]

You can use confstr to get (running) glibc version:

>>> os.confstr('CS_GNU_LIBC_VERSION')
'glibc 2.28'

[vstinner]

>>> os.confstr('CS_GNU_LIBC_VERSION')
'glibc 2.28'

That's cool because it doesn't require to write new C code ;-)

[serhiy-storchaka]

Currently libc_ver() can be used for other executable. See bpo-26544 for discussion about libc_ver().

[vstinner]

Currently libc_ver() can be used for other executable. See bpo-26544 for discussion about libc_ver().

Oh, my PR had a bug: it ignores executable. Fixed: it now only uses os.confstr() if the executable argument is not set.

[vstinner]

New changeset 476b113 by Victor Stinner in branch 'master':
bpo-35389: platform.libc_ver() uses os.confstr() (GH-10891)
476b113

[vstinner]

Quick benchmark on Fedora 29:
python3 -m perf command ./python -S -c 'import platform; platform.libc_ver()'
94.9 ms +- 4.3 ms -> 33.2 ms +- 1.4 ms: 2.86x faster (-65%)

Oops, my benchmark in the commit message was wrong, it includes the startup time...

Correct benchmark says 44,538x faster, it's *WAY* better!

[regex] 56.1 ms +- 1.9 ms -> [confstr] 1.26 us +- 0.04 us: 44537.88x faster (-100%)

[malemburg]

Nice. I never liked the "parse the executable approach", but there wasn't anything better available at the time.

[vstinner]

Nice. I never liked the "parse the executable approach", but there wasn't anything better available at the time.

Aha. Well, it's not perfect but it works and was fast enough (since libc_ver() is never used in performance critical code) :-)

I'm now curious and looked at the history of this feature.

"man confstr" says:

_CS_GNU_LIBC_VERSION (GNU C library only; since glibc 2.3.2)

glibc 2.3.2 has been released in March 2003, so it's fine, we should get this constant in most "modern" Linux (using glibc) in 2018 :-)

man gnu_get_libc_version says:

These functions first appeared in glibc in version 2.1.

glibc 2.1 has been released in Feb 1999. Using this function might provide even better compatibility but I'm not sure that it's worth it to use it. As I wrote, I prefer to not write a new function C, if os.confstr() can already be used in pure Python!

Sadly, these functions (confstr(_CS_GNU_LIBC_VERSION) / gnu_get_libc_version()) are specific to glibc. Sorry, I'm not interested to support other libc, I mostly care about Fedora, sorry :-)

[vstinner]

New changeset 848acf7 by Victor Stinner in branch 'master':
bpo-35389: test.pythoninfo logs platform.libc_ver (GH-10951)
848acf7

[vstinner]

New changeset a719c8f by Victor Stinner in branch 'master':
bpo-35389: platform.platform() calls libc_ver() without executable (GH-14418)
a719c8f