Add platform.freedesktop_os_release() #72654
Comments
|
bpo-1322 has deprecated platform.linux_distribution(). The feature is going to be removed without replacement functions in the stdlib. It's no longer possible to detect the Linux distribution from stdlib. Let's add a function to read /etc/os-release [1] instead. It's a very simple format and pretty standard for many years. |
tiran
added
3.7 (EOL)
|
and for the older distributions without Systemd ? What do you suggest ? |
|
Also there is an external project now aiming to provide this functionality: |
|
I'm not suggesting a generic platform detection function but rather a limited function just for os-release. On platforms without /etc/os-release the function should obviously fail and raise an exception. A large majority of Linux distributions has the file. It's available on Debian, Fedora, RHEL 6+, Ubuntu 14.04+ and many more. |
|
-1 This is not available everywhere, not even on Linux distributions. Why would you implement something which only works on 50% of existing Linux distributions? Who would even use that? And for what would you use it? You have a distro module available now from PyPi, you can use that one instead, and it gives 100% correct results for all currently known distros, and can be updated independent of Python releases. If it's that a simple function, then please implement it for your own needs. I think it's bad style too, to rely on specific distro and release information instead of checking release names and versions. Such a style should not be encouraged in the standard lib from my point of view. |
|
Can you elaborate the expected API? /etc/os-release is not a single line, but a list of variables. Example on my Fedora 24: $ cat /etc/os-release
NAME=Fedora
VERSION="24 (Workstation Edition)"
ID=fedora
VERSION_ID=24
PRETTY_NAME="Fedora 24 (Workstation Edition)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:24"
HOME_URL="https://fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=24
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=24
PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy
VARIANT="Workstation Edition"
VARIANT_ID=workstationI think that you should return a dictionary key=>value. Maybe we can simply return an empty dictionary if the file doesn't exist or cannot be read (but raise an error on parsing error). FYI On Fedora 24, /etc/os-release is in fact a symbolic link to a symbolic link: $ ls -l /etc/os-release
lrwxrwxrwx. 1 root root 21 24 juin 03:25 /etc/os-release -> ../usr/lib/os-release
$ ls -l /usr/lib/os-release
lrwxrwxrwx. 1 root root 37 13 sept. 09:51 /usr/lib/os-release -> ./os.release.d/os-release-workstation
$ ls -l /usr/lib/os.release.d/os-release-workstation
-rw-r--r--. 1 root root 518 24 juin 03:25 /usr/lib/os.release.d/os-release-workstationSee also systemd manual page: |
|
"This is not available everywhere, not even on Linux distributions. Why would you implement something which only works on 50% of existing Linux distributions?" According to http://0pointer.de/blog/projects/os-release systemd requires /etc/os-release. Since all major Linux distribution now use systemd, it makes sense to me to add such function. We can document that the feature is specific to Linux (or don't declare the function on Linux?), document that it only works on recent versions of Linux distributions, and document that it doesn't work on all Linux distributions. |
|
Please read the title of this issue. It is "Add platform.linux_os_release()". The name of the function clearly indicats that the function only works on Linux with a os-release file. |
|
"Since all major Linux distribution now use systemd, it makes sense to me to add such function." I'm not aware of any embedded Linux distro using systemd (no, I don't consider Raspian an embedded Linux distro). |
|
"The name of the function clearly indicats that the function only works on Linux with a os-release file." No, it does not. That would be linux_distribution_with_os_release_file(). |
|
I hate systemd. So does the company I'm currently doing most of my work for. We're using centos6 because it doesn't have systemd. I personally use gentoo without systemd. That said, I don't use the platform module, so this doesn't actually affect me. |
|
ETOOMUCHBIKESHEDDING and another uncalled systemd flame wars. |
|
We need a fallback in the case where /etc/os-release does not exists, or just use hasattr on if hasattr(platform, 'linux_os_release'):
print(platform.linux_os_release())
else:
...or just raise an exception if platform.linux_os_release is not available. |
|
Sorry, it wasn't my intent to (re)start a flame war, just to point out that there are linux platforms in wide use that do not support systemd. But I don't have an opinion on whether or not adding this function is a good idea or not. |
|
Since we need distribution information in tests and code, I decided to reopen the bug. I named the new function freedesktop_osrelease because it's technically not restricted to Linux. It's freedesktop.org standard that may be used by non-Linux platforms, too. Note: os-release is **not** tight to systemd. It just happens that the standard was defined in the systemd space of freedesktop.org. I'm not aware of any major Linux platform that does not ship the file in its base os. |
tiran
changed the title
|
it's easy to say "we need it". It took a while to remove that, so I don't think it should be easily re-introduced. so why do you need it, and why should it be exposed as part of the standard library? |
|
What do you mean by "that"? Python never had any code to parse and handle freekdesktop.org's os-release file. Are you referring to linux_distribution() function? Petr removed platform.linux_distribution() in bpo-28167 because the function was problematic on so many levels and relied on ill defined behavior. The os-release file has been a well-defined standard for over 8 years. These days it's available in the base image of all major Linux distributions and countless other distributions. os-release became *the* standard to identify the version and vendor of a Linux distribution. In my opinion that's reason enough to include the feature. |
|
Matthias Klose:
As soon as the function is documented to return an error if the file doesn't exist, I don't see how this is a blocker issue. Matthias: I don't understand your arguments against adding a Python function to read a standardize configuration file. The platform module is the right place to add such function. The distro module exists on PyPI because there is a need for such a function. The function even existed previously in platform. It only had to be removed because it was a pain in the *** to maintain it. The stdlib is slow to be updated, whereas Linux vendors were creative for the filename and the format of the filename. Since the filename is now fixed and the format is well specified, IMO it's perfectly fine to add PR 23492. |
I disagree with that. The distro module is the preferred way to do this (which cannot be used in the Python core). The rationale given is an not yet investigated issue with a timeout in tk/ttk, which from my point of view is not good enough to add a public API. |
|
The Chef Software seems to maintain a copy of the os-release of major Linux distribution: Copy of the project README: # os_release This repo contains the /etc/os-release file from Linux distros. ## About os-release /etc/os-release is a standard Linux file used to identify the OS, os release, and similar distros. It's now a standard file in systemd distros, but it can be found in just about every Linux distro released over the last 3-4 years. ## Why collect them The fields in /etc/os-release are standardized, but the values are not. The only way to know that Redhat Enterprise Linux is 'rhel' or that openSUSE 15 is 'opensuse-leap' is to install the distros and check the file. This repo is a large collection of os-release files so you don't need to install each and every distro. |
|
"The os-release file has been a well-defined standard for over 8 years." ... doesn't implicate that it's a good style to base your checks on that information. |
|
the Chef repo seems to be a little bit out-of-date, but anyway. I don't see the relevance for this issue. |
|
From PR discussion on GH: I made ID_LIKE a special case because it's likely a users will use the field in a wrong way. The issue won't be detected in common CI because the field is either not present or contains a single item for majority of Linux distros. Common values are debian or fedora. For example Ubuntu has ID_LIKE=debian. Only very few distros have an ID_LIKE with multiple entries. Victor asked me to remove the special case because splitting is trivial. In that case users would have to split the field and always check for "name in info["ID_LIKE"].split()". The special case makes it less likely that users get the field wrong and reduces their burden. |
|
I'm -1 on adding any kind of replacement for platform.linux_distribution() tp Python's stdlib. The experiment has failed and we should acknowledge this. The main reason why it failed was the Linux distros keep inventing new ways to identify themselves, sometimes misuse existing mechanisms to maintain compatibility (e.g. as a Ubuntu or RedHat based OS) or ship with two different release files. At the time we removed the API, we said that people should use PyPI package to get this information, since those can much more easily be maintained with the ever changing patterns distributions use or invent. The distro package is one package and if it doesn't satisfy the needs, simply create a new one which does. Note that the original implementation had a mechanism to read such release files. Even the names of those files changed every few years. People appear to have settled on "os-release" nowadays, since using "<distro>-release" was found to create too much confusion, but this is not fixed and when it comes to parsing the contents, I'm pretty sure that there a subtle differences in how to interpret them between distros. Then the LSB standardized this and wanted everyone to use lsb_release. Guess what... The tool is not even installed anymore by default on recent OpenSUSE releases. Experience shows that these things change too often to make the stdlib a good place to maintain support for this. |
|
It's not a replacement for platform.linux_distribution(). |
tiran
changed the title
|
On 25.11.2020 10:06, Christian Heimes wrote:
Right, not even that :-) It would be an interface to a file /etc/os-release To get a sense of the complexity involved in all this, have a look It's not just about the file name, the content also needs a lot -- Professional Python Services directly from the Experts (#1, Nov 25 2020)
>>> Python Projects, Coaching and Support ... https://www.egenix.com/
>>> Python Product Development ... https://consulting.egenix.com/
________________________________________________________________________::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 |
malemburg
changed the title
That's pointless speculation -- unless you found a fool-proof way to see into the future. Where do you draw the line? Should the stdlib ignore any standard and API that is less than 10 years old? 20 years? |
there's a module available outside cpython. Your use case is to add some work around for some tests (like bpo-42142). If you need it for the tests, add it to the test framework. |
|
On 25.11.2020 10:39, Christian Heimes wrote:
It's not pointless. The rate of change in the field is why this Note that I'm not saying it's a useless API. The main point is Additionally, such a package could provide normalization features, The distro package is doing a pretty good job at this. If you need this for tests in the stdlib, I'd suggest to add a helper For tests outside the stdlib, the distro package will do the -- Professional Python Services directly from the Experts (#1, Nov 25 2020)
>>> Python Projects, Coaching and Support ... https://www.egenix.com/
>>> Python Product Development ... https://consulting.egenix.com/
________________________________________________________________________::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 |
|
Marc-Andre Lemburg:
Please read the discussion. os-release is a standardize file: fixed filename, only 2 possible locations, standardize format. It's a freedesktop specification. In my experience, freedesktop standards are well adopted, whereas LSB standards look like a failed attempt. This is no need to bet on the future. os-release file is already available on all major Linux distribution, as, again, explained in depth in previous comments. There is no need to install a lsb_release program or anything. It's part of the *base* image on these systems. As you wrote, lsb_release isn't installed on many operating systems. For example, I just checked on my Fedora 33, it's not installed... Moreover, I'm not comfortable with running an external program which causes its own set of issue. Reading a plain text file is safer and faster.
Ansible has a high-level API similar to what the Python distro module does: it calls uname, parse /etc/*release files, try to normalize values, has special cases, etc. Not everybody needs like kind of advanced API. For my own use cases (see previous comments), I'm perfectly fine with basic and incomplete information.
I strongly suggest to keep platform.freedesktop_os_release() as simple as possible: just parse the file, no special case at all. For example, I'm against treating ID_LIKE variable differently in PR 23492. Users would expect if another space separated variable is added tomorrow, the function would also specialize it, and so we would come back to the linux_distribution() maintenance issue. IHMO a dummy text parser remains very useful. The file format is not as trivial as it looks, there are single quote and double quotes, and escaped characters. If someone wants a more complete API, I suggest to develop it on PyPI on top of the new function. It's fine to provide any API where some variables can miss if the there are not provided by the operating system. For me, it's like the os and select modules which are thin wrappers to operating system functions (syscalls). And then there are more high-level module like shutil and selectors. But we don't need to ship a high-level API for os-release. Platform was always a thin wrapper to OS functions. For example, there is no unified API to retrieve OS name and version on Windows, macOS or Linux. You need to pick the proper function. For me, freedesktop_os_release() just follows this trend. |
|
Marc-Andre Lemburg:
Please look at the history of the os-release file. The os-release file format exists for 8 years and has not changed in 8 years. It seems like you are referring to /etc/*release files. I agree that these files are causing a lot of maintenance troubles, and... that's exactly why os-release file has been standardized and created. So I don't get your point. |
vstinner
changed the title
|
On 25.11.2020 11:13, STINNER Victor wrote:
Not really. We have functions per OS, but not functions which only work The patch also has other issues: A text file parse could be a private function in the module, platform module APIs should return meaningful information and Regardless, I don't see the point of opening up this can of -- Professional Python Services directly from the Experts (#1, Nov 25 2020)
>>> Python Projects, Coaching and Support ... https://www.egenix.com/
>>> Python Product Development ... https://consulting.egenix.com/
________________________________________________________________________::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 |
Yes, it's pointless. You are arguing that os-release is bad because lsb-release was bad. It's like arguing against TOML file format because YAML and JSON have issues. os-release was purposely designed to address concerns with lsb-release. os-release has been around for more than a decade and wildly adopted (except on Android platforms). The format hasn't changed since its first appearance in November 2010. Some optional fields were standardized and the meaning of "no shell features are supported" was elaborated on. |
|
PS: MAL, would it be possible to suppress your email footer? BPO is not an advertisement channel. |
|
On 25.11.2020 11:45, Christian Heimes wrote:
That's an ancient bug on bpo, which apparently still haven't been |
|
Pablo, I would like to get mediation from the release manager. I want to add an interface to the os-release file. It's a decade-old standard from freedesktop.org. The file is available in the minimal base installation of virtually every Linux distributions except for Android. It's in Alpine, Arch Linux, Debian family (Mint, Raspberry Pi OS, Ubuntu, ...), Fedora family (CentOS, RHEL, ...), Gentoo, SUSE, and many more. The os-release file contains human readable and machine readable operating system information. The file format is simple but not trivial. Matthias and Marc-Andre oppose my patch. Victor and I are in favor of the new feature. |
|
I just needed such functionality for PTY tests: #23514 |
another use case for a test. so please add it to the test framework, also suggested by Marc. |
|
Don't get me wrong. So I'm +0 for adding this function -- but can live without it. |
|
We have reached an impasse. Therefore I have contacted the steering council and requested mediation. Victor will abstain from the decision process. |
|
This issue was brought to the Python Steering Council, and we deliberated it at today's SC meeting. With a vote of 4 approvals and one abstention, we have approved the addition of this API. |
|
I don't agree with this decision, but ok. In this case, please also provide a way to provide the value of the VERSION_ID field. It doesn't make sense to blacklist whole distributions in tests. |
|
Matthias Klose:
Adding freedesktop_os_release() and using it are two different topics. I understand the the SC decision is on adding the function. Using it to skip tests should be decided on a case by case basic. In general, I really dislike to skip tests on a specific Linux distribution. In bpo-41818, the os-release file was used as a temporary fix just to unblock buildbots, but it was quickly removed. It was not used to skip a test on Gentoo, but to run the test on Gentoo ;-) Also, it was quickly removed, and it seems like even the skip on BSD was removed. Great! It was there for a very long time. |
I'm not sure what you mean. PR 23492 exposes the whole content of the os-release file as a dict. There is no special case for any field. VERSION_ID can be retrieved as any field with: platform.freedesktop_os_release().get('VERSION_ID', ''). Example on Fedora 33. $ cat /etc/os-release
NAME=Fedora
VERSION="33 (Workstation Edition)"
ID=fedora
VERSION_ID=33
VERSION_CODENAME=""
PLATFORM_ID="platform:f33"
PRETTY_NAME="Fedora 33 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:33"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/"
SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=33
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=33
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation$ ./python
Python 3.10.0a2+ (heads/pr/23492:6a4c92bcbe, Nov 30 2020, 22:29:23)
[GCC 10.2.1 20201016 (Red Hat 10.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform, pprint
>>> os_release=platform.freedesktop_os_release()
>>> pprint.pprint(os_release)
{'ANSI_COLOR': '0;38;2;60;110;180',
'BUG_REPORT_URL': 'https://bugzilla.redhat.com/',
'CPE_NAME': 'cpe:/o:fedoraproject:fedora:33',
'DOCUMENTATION_URL': 'https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/',
'HOME_URL': 'https://fedoraproject.org/',
'ID': 'fedora',
'LOGO': 'fedora-logo-icon',
'NAME': 'Fedora',
'PLATFORM_ID': 'platform:f33',
'PRETTY_NAME': 'Fedora 33 (Workstation Edition)',
'PRIVACY_POLICY_URL': 'https://fedoraproject.org/wiki/Legal:PrivacyPolicy',
'REDHAT_BUGZILLA_PRODUCT': 'Fedora',
'REDHAT_BUGZILLA_PRODUCT_VERSION': '33',
'REDHAT_SUPPORT_PRODUCT': 'Fedora',
'REDHAT_SUPPORT_PRODUCT_VERSION': '33',
'SUPPORT_URL': 'https://fedoraproject.org/wiki/Communicating_and_getting_help',
'VARIANT': 'Workstation Edition',
'VARIANT_ID': 'workstation',
'VERSION': '33 (Workstation Edition)',
'VERSION_CODENAME': '',
'VERSION_ID': '33'}
>>> os_release['VERSION_ID']
'33' |
|
I merged Christian's PR 23492. The initial issue, add a function to parse the os-release file, is now solved, so I close the issue. If you want to use it in a test, please open a new issue. Thanks everyone for this constructive discussion, IMO the final merged function is now better thanks to that. And thanks Christian for pushing the feature and implementing it ;-) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: