Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add platform.freedesktop_os_release() #72654

Closed
tiran opened this issue Oct 18, 2016 · 62 comments
Closed

Add platform.freedesktop_os_release() #72654

tiran opened this issue Oct 18, 2016 · 62 comments
Labels
3.10 stdlib type-feature

Comments

@tiran
Copy link
Member

@tiran tiran commented Oct 18, 2016

BPO 28468
Nosy @malemburg, @warsaw, @doko42, @vstinner, @tiran, @bitdancer, @asvetlov, @pmp-p, @matrixise, @stratakis, @pablogsal, @miss-islington
PRs
  • #23492
  • #23913
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-11-30.21:37:48.382>
    created_at = <Date 2016-10-18.08:47:38.710>
    labels = ['type-feature', 'library', '3.10']
    title = 'Add platform.freedesktop_os_release()'
    updated_at = <Date 2020-12-23.16:36:03.116>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2020-12-23.16:36:03.116>
    actor = 'miss-islington'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-11-30.21:37:48.382>
    closer = 'vstinner'
    components = ['Library (Lib)']
    creation = <Date 2016-10-18.08:47:38.710>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 28468
    keywords = []
    message_count = 62.0
    messages = ['278847', '278849', '278850', '278851', '278860', '278861', '278862', '278863', '278864', '278865', '278868', '278869', '278871', '278874', '381719', '381721', '381722', '381724', '381726', '381727', '381728', '381729', '381730', '381733', '381734', '381735', '381736', '381737', '381741', '381742', '381743', '381744', '381745', '381747', '381750', '381757', '381758', '381768', '381798', '381799', '381804', '381805', '381808', '381812', '381813', '381814', '381816', '381817', '381819', '381820', '381825', '381849', '381851', '381855', '381856', '382181', '382184', '382194', '382195', '382196', '382197', '383649']
    nosy_count = 12.0
    nosy_names = ['lemburg', 'barry', 'doko', 'vstinner', 'christian.heimes', 'r.david.murray', 'asvetlov', 'pmpp', 'matrixise', 'cstratak', 'pablogsal', 'miss-islington']
    pr_nums = ['23492', '23913']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue28468'
    versions = ['Python 3.10']

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 18, 2016

    bpo-1322 has deprecated platform.linux_distribution(). The feature is going to be removed without replacement functions in the stdlib. It's no longer possible to detect the Linux distribution from stdlib. Let's add a function to read /etc/os-release [1] instead. It's a very simple format and pretty standard for many years.

    [1] http://0pointer.de/blog/projects/os-release

    @tiran tiran added 3.7 stdlib type-feature labels Oct 18, 2016
    @matrixise
    Copy link
    Member

    @matrixise matrixise commented Oct 18, 2016

    and for the older distributions without Systemd ? What do you suggest ?

    @stratakis
    Copy link
    Mannequin

    @stratakis stratakis mannequin commented Oct 18, 2016

    Also there is an external project now aiming to provide this functionality:

    https://github.com/nir0s/distro

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 18, 2016

    I'm not suggesting a generic platform detection function but rather a limited function just for os-release. On platforms without /etc/os-release the function should obviously fail and raise an exception.
    /etc/os-release is available on distributions without systemd. Even my oldest Debian installation on a Raspberry Pi has it.

    A large majority of Linux distributions has the file. It's available on Debian, Fedora, RHEL 6+, Ubuntu 14.04+ and many more.

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Oct 18, 2016

    -1

    This is not available everywhere, not even on Linux distributions. Why would you implement something which only works on 50% of existing Linux distributions? Who would even use that? And for what would you use it? You have a distro module available now from PyPi, you can use that one instead, and it gives 100% correct results for all currently known distros, and can be updated independent of Python releases.

    If it's that a simple function, then please implement it for your own needs. I think it's bad style too, to rely on specific distro and release information instead of checking release names and versions. Such a style should not be encouraged in the standard lib from my point of view.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Oct 18, 2016

    Can you elaborate the expected API? /etc/os-release is not a single line, but a list of variables. Example on my Fedora 24:
    ---

    $ cat /etc/os-release 
    NAME=Fedora
    VERSION="24 (Workstation Edition)"
    ID=fedora
    VERSION_ID=24
    PRETTY_NAME="Fedora 24 (Workstation Edition)"
    ANSI_COLOR="0;34"
    CPE_NAME="cpe:/o:fedoraproject:fedora:24"
    HOME_URL="https://fedoraproject.org/"
    BUG_REPORT_URL="https://bugzilla.redhat.com/"
    REDHAT_BUGZILLA_PRODUCT="Fedora"
    REDHAT_BUGZILLA_PRODUCT_VERSION=24
    REDHAT_SUPPORT_PRODUCT="Fedora"
    REDHAT_SUPPORT_PRODUCT_VERSION=24
    PRIVACY_POLICY_URL=https://fedoraproject.org/wiki/Legal:PrivacyPolicy
    VARIANT="Workstation Edition"
    VARIANT_ID=workstation

    I think that you should return a dictionary key=>value. Maybe we can simply return an empty dictionary if the file doesn't exist or cannot be read (but raise an error on parsing error).

    FYI On Fedora 24, /etc/os-release is in fact a symbolic link to a symbolic link:
    ---

    $ ls -l /etc/os-release 
    lrwxrwxrwx. 1 root root 21 24 juin  03:25 /etc/os-release -> ../usr/lib/os-release
    
    $ ls -l /usr/lib/os-release
    lrwxrwxrwx. 1 root root 37 13 sept. 09:51 /usr/lib/os-release -> ./os.release.d/os-release-workstation
    
    $ ls -l /usr/lib/os.release.d/os-release-workstation
    -rw-r--r--. 1 root root 518 24 juin  03:25 /usr/lib/os.release.d/os-release-workstation

    See also systemd manual page:
    https://www.freedesktop.org/software/systemd/man/os-release.html

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Oct 18, 2016

    "This is not available everywhere, not even on Linux distributions. Why would you implement something which only works on 50% of existing Linux distributions?"

    According to http://0pointer.de/blog/projects/os-release systemd requires /etc/os-release. Since all major Linux distribution now use systemd, it makes sense to me to add such function.

    We can document that the feature is specific to Linux (or don't declare the function on Linux?), document that it only works on recent versions of Linux distributions, and document that it doesn't work on all Linux distributions.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 18, 2016

    Please read the title of this issue. It is "Add platform.linux_os_release()". The name of the function clearly indicats that the function only works on Linux with a os-release file.

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Oct 18, 2016

    "Since all major Linux distribution now use systemd, it makes sense to me to add such function."

    I'm not aware of any embedded Linux distro using systemd (no, I don't consider Raspian an embedded Linux distro).

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Oct 18, 2016

    "The name of the function clearly indicats that the function only works on Linux with a os-release file."

    No, it does not. That would be linux_distribution_with_os_release_file().

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented Oct 18, 2016

    I hate systemd. So does the company I'm currently doing most of my work for. We're using centos6 because it doesn't have systemd. I personally use gentoo without systemd.

    That said, I don't use the platform module, so this doesn't actually affect me.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 18, 2016

    ETOOMUCHBIKESHEDDING and another uncalled systemd flame wars.

    @tiran tiran closed this as completed Oct 18, 2016
    @matrixise
    Copy link
    Member

    @matrixise matrixise commented Oct 18, 2016

    We need a fallback in the case where /etc/os-release does not exists, or just use hasattr on platform

    if hasattr(platform, 'linux_os_release'):
       print(platform.linux_os_release())
    else:
       ...

    or just raise an exception if platform.linux_os_release is not available.

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented Oct 18, 2016

    Sorry, it wasn't my intent to (re)start a flame war, just to point out that there are linux platforms in wide use that do not support systemd. But I don't have an opinion on whether or not adding this function is a good idea or not.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 24, 2020

    Since we need distribution information in tests and code, I decided to reopen the bug.

    I named the new function freedesktop_osrelease because it's technically not restricted to Linux. It's freedesktop.org standard that may be used by non-Linux platforms, too.

    Note: os-release is **not** tight to systemd. It just happens that the standard was defined in the systemd space of freedesktop.org. I'm not aware of any major Linux platform that does not ship the file in its base os.

    @tiran tiran added 3.10 and removed 3.7 labels Nov 24, 2020
    @tiran tiran changed the title Add platform.linux_os_release() Add platform.freedesktop_osrelease() Nov 24, 2020
    @tiran tiran reopened this Nov 24, 2020
    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 24, 2020

    it's easy to say "we need it". It took a while to remove that, so I don't think it should be easily re-introduced.

    so why do you need it, and why should it be exposed as part of the standard library?

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 24, 2020

    What do you mean by "that"? Python never had any code to parse and handle freekdesktop.org's os-release file. Are you referring to linux_distribution() function? Petr removed platform.linux_distribution() in bpo-28167 because the function was problematic on so many levels and relied on ill defined behavior.

    The os-release file has been a well-defined standard for over 8 years. These days it's available in the base image of all major Linux distributions and countless other distributions. os-release became *the* standard to identify the version and vendor of a Linux distribution. In my opinion that's reason enough to include the feature.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 24, 2020

    Matthias Klose:

    I'm not aware of any embedded Linux distro using systemd (no, I don't consider Raspian an embedded Linux distro).

    As soon as the function is documented to return an error if the file doesn't exist, I don't see how this is a blocker issue.

    Matthias: I don't understand your arguments against adding a Python function to read a standardize configuration file.

    The platform module is the right place to add such function.

    The distro module exists on PyPI because there is a need for such a function. The function even existed previously in platform. It only had to be removed because it was a pain in the *** to maintain it. The stdlib is slow to be updated, whereas Linux vendors were creative for the filename and the format of the filename.

    Since the filename is now fixed and the format is well specified, IMO it's perfectly fine to add PR 23492.

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 24, 2020

    Since the filename is now fixed and the format is well specified,
    IMO it's perfectly fine to add PR 23492.

    I disagree with that. The distro module is the preferred way to do this (which cannot be used in the Python core). The rationale given is an not yet investigated issue with a timeout in tk/ttk, which from my point of view is not good enough to add a public API.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 24, 2020

    The Chef Software seems to maintain a copy of the os-release of major Linux distribution:
    https://github.com/chef/os_release

    Copy of the project README:

    # os_release

    This repo contains the /etc/os-release file from Linux distros.

    ## About os-release

    /etc/os-release is a standard Linux file used to identify the OS, os release, and similar distros. It's now a standard file in systemd distros, but it can be found in just about every Linux distro released over the last 3-4 years.

    ## Why collect them

    The fields in /etc/os-release are standardized, but the values are not. The only way to know that Redhat Enterprise Linux is 'rhel' or that openSUSE 15 is 'opensuse-leap' is to install the distros and check the file. This repo is a large collection of os-release files so you don't need to install each and every distro.

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 24, 2020

    "The os-release file has been a well-defined standard for over 8 years."

    ... doesn't implicate that it's a good style to base your checks on that information.

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 24, 2020

    the Chef repo seems to be a little bit out-of-date, but anyway. I don't see the relevance for this issue.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 24, 2020

    From PR discussion on GH:

    I made ID_LIKE a special case because it's likely a users will use the field in a wrong way. The issue won't be detected in common CI because the field is either not present or contains a single item for majority of Linux distros. Common values are debian or fedora. For example Ubuntu has ID_LIKE=debian. Only very few distros have an ID_LIKE with multiple entries.

    Victor asked me to remove the special case because splitting is trivial. In that case users would have to split the field and always check for "name in info["ID_LIKE"].split()".

    The special case makes it less likely that users get the field wrong and reduces their burden.

    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Nov 25, 2020

    I'm -1 on adding any kind of replacement for platform.linux_distribution() tp Python's stdlib. The experiment has failed and we should acknowledge this.

    The main reason why it failed was the Linux distros keep inventing new ways to identify themselves, sometimes misuse existing mechanisms to maintain compatibility (e.g. as a Ubuntu or RedHat based OS) or ship with two different release files.

    At the time we removed the API, we said that people should use PyPI package to get this information, since those can much more easily be maintained with the ever changing patterns distributions use or invent.

    The distro package is one package and if it doesn't satisfy the needs, simply create a new one which does.

    Note that the original implementation had a mechanism to read such release files. Even the names of those files changed every few years. People appear to have settled on "os-release" nowadays, since using "<distro>-release" was found to create too much confusion, but this is not fixed and when it comes to parsing the contents, I'm pretty sure that there a subtle differences in how to interpret them between distros.

    Then the LSB standardized this and wanted everyone to use lsb_release. Guess what... The tool is not even installed anymore by default on recent OpenSUSE releases. Experience shows that these things change too often to make the stdlib a good place to maintain support for this.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 25, 2020

    It's not a replacement for platform.linux_distribution().

    @tiran tiran changed the title Add platform.freedesktop_osrelease() Add platform.freedesktop_os_release() Nov 25, 2020
    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Nov 25, 2020

    On 25.11.2020 10:06, Christian Heimes wrote:

    It's not a replacement for platform.linux_distribution().

    Right, not even that :-)

    It would be an interface to a file /etc/os-release
    that's common nowadays, just like /etc/lsb-release was some years
    ago. These things change too often to make the stdlib a good fit.
    I'm pretty sure distros will invent something new in 5 years which
    would then render the API mostly useless again.

    To get a sense of the complexity involved in all this, have a look
    at how Ansible does this:

    https://github.com/ansible/ansible/blob/ea119d30894478b84b5fbe271f580cb2b0401b86/lib/ansible/module_utils/facts/system/distribution.py

    It's not just about the file name, the content also needs a lot
    of massaging to make it useful for a general purpose API. And
    the content quirks are not set in stone either. Here's a ticket
    where Debian, for example, drops the version info from the file:

    ansible/ansible#19874

    --
    Marc-Andre Lemburg
    eGenix.com

    Professional Python Services directly from the Experts (#1, Nov 25 2020)
    >>> Python Projects, Coaching and Support ...    https://www.egenix.com/
    >>> Python Product Development ...        https://consulting.egenix.com/
    ________________________________________________________________________

    ::: We implement business ideas - efficiently in both time and costs :::

    eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
    Registered at Amtsgericht Duesseldorf: HRB 46611
    https://www.egenix.com/company/contact/
    https://www.malemburg.com/

    @malemburg malemburg changed the title Add platform.freedesktop_os_release() Add platform.freedesktop_osrelease() Nov 25, 2020
    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 25, 2020

    It would be an interface to a file /etc/os-release
    that's common nowadays, just like /etc/lsb-release was some years
    ago. These things change too often to make the stdlib a good fit.
    I'm pretty sure distros will invent something new in 5 years which
    would then render the API mostly useless again.

    That's pointless speculation -- unless you found a fool-proof way to see into the future.

    Where do you draw the line? Should the stdlib ignore any standard and API that is less than 10 years old? 20 years?

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 25, 2020

    Where do you draw the line?

    there's a module available outside cpython. Your use case is to add some work around for some tests (like bpo-42142). If you need it for the tests, add it to the test framework.

    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Nov 25, 2020

    On 25.11.2020 10:39, Christian Heimes wrote:

    > It would be an interface to a file /etc/os-release
    > that's common nowadays, just like /etc/lsb-release was some years
    > ago. These things change too often to make the stdlib a good fit.
    > I'm pretty sure distros will invent something new in 5 years which
    > would then render the API mostly useless again.

    That's pointless speculation -- unless you found a fool-proof way to see into the future.

    Where do you draw the line? Should the stdlib ignore any standard and API that is less than 10 years old? 20 years?

    It's not pointless. The rate of change in the field is why this
    particular API did not work out in practice. It was working fine
    at the time I added it, but then quickly became unmaintainable.

    Note that I'm not saying it's a useless API. The main point is
    that such code is better placed into a PyPI package, where it
    can be updated more frequently and also in a safe way across
    Python versions.

    Additionally, such a package could provide normalization features,
    to make matching distros easier, even when the distros decide
    to change the way they are named, e.g. for marketing purposes.

    The distro package is doing a pretty good job at this.

    If you need this for tests in the stdlib, I'd suggest to add a helper
    to the test package to find the needed information for the platforms
    where you want to run platform dependent tests.

    For tests outside the stdlib, the distro package will do the
    job just fine.

    --
    Marc-Andre Lemburg
    eGenix.com

    Professional Python Services directly from the Experts (#1, Nov 25 2020)
    >>> Python Projects, Coaching and Support ...    https://www.egenix.com/
    >>> Python Product Development ...        https://consulting.egenix.com/
    ________________________________________________________________________

    ::: We implement business ideas - efficiently in both time and costs :::

    eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
    Registered at Amtsgericht Duesseldorf: HRB 46611
    https://www.egenix.com/company/contact/
    https://www.malemburg.com/

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 25, 2020

    Marc-Andre Lemburg:

    The main reason why it failed was the Linux distros keep inventing new ways to identify themselves, sometimes misuse existing mechanisms to maintain compatibility (e.g. as a Ubuntu or RedHat based OS) or ship with two different release files.
    (...)
    Then the LSB standardized this and wanted everyone to use lsb_release. Guess what... The tool is not even installed anymore by default on recent OpenSUSE releases. Experience shows that these things change too often to make the stdlib a good place to maintain support for this.

    Please read the discussion. os-release is a standardize file: fixed filename, only 2 possible locations, standardize format. It's a freedesktop specification. In my experience, freedesktop standards are well adopted, whereas LSB standards look like a failed attempt.

    This is no need to bet on the future. os-release file is already available on all major Linux distribution, as, again, explained in depth in previous comments. There is no need to install a lsb_release program or anything. It's part of the *base* image on these systems.

    As you wrote, lsb_release isn't installed on many operating systems. For example, I just checked on my Fedora 33, it's not installed... Moreover, I'm not comfortable with running an external program which causes its own set of issue. Reading a plain text file is safer and faster.

    To get a sense of the complexity involved in all this, have a look
    at how Ansible does this:
    https://github.com/ansible/ansible/blob/ea119d30894478b84b5fbe271f580cb2b0401b86/lib/ansible/module_utils/facts/system/distribution.py

    Ansible has a high-level API similar to what the Python distro module does: it calls uname, parse /etc/*release files, try to normalize values, has special cases, etc.

    Not everybody needs like kind of advanced API. For my own use cases (see previous comments), I'm perfectly fine with basic and incomplete information.

    It's not just about the file name, the content also needs a lot
    of massaging to make it useful for a general purpose API. (...)

    I strongly suggest to keep platform.freedesktop_os_release() as simple as possible: just parse the file, no special case at all.

    For example, I'm against treating ID_LIKE variable differently in PR 23492. Users would expect if another space separated variable is added tomorrow, the function would also specialize it, and so we would come back to the linux_distribution() maintenance issue.

    IHMO a dummy text parser remains very useful. The file format is not as trivial as it looks, there are single quote and double quotes, and escaped characters.

    If someone wants a more complete API, I suggest to develop it on PyPI on top of the new function.

    It's fine to provide any API where some variables can miss if the there are not provided by the operating system. For me, it's like the os and select modules which are thin wrappers to operating system functions (syscalls). And then there are more high-level module like shutil and selectors. But we don't need to ship a high-level API for os-release.

    Platform was always a thin wrapper to OS functions. For example, there is no unified API to retrieve OS name and version on Windows, macOS or Linux. You need to pick the proper function. For me, freedesktop_os_release() just follows this trend.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 25, 2020

    Marc-Andre Lemburg:

    It's not pointless. The rate of change in the field is why this
    particular API did not work out in practice. It was working fine
    at the time I added it, but then quickly became unmaintainable.

    Please look at the history of the os-release file. The os-release file format exists for 8 years and has not changed in 8 years.

    It seems like you are referring to /etc/*release files. I agree that these files are causing a lot of maintenance troubles, and... that's exactly why os-release file has been standardized and created.

    So I don't get your point.

    @vstinner vstinner changed the title Add platform.freedesktop_osrelease() Add platform.freedesktop_os_release() Nov 25, 2020
    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Nov 25, 2020

    On 25.11.2020 11:13, STINNER Victor wrote:

    Platform was always a thin wrapper to OS functions. For example, there is no unified API to retrieve OS name and version on Windows, macOS or Linux. You need to pick the proper function. For me, freedesktop_os_release() just follows this trend.

    Not really. We have functions per OS, but not functions which only work
    on a subset of distros of an OS.

    The patch also has other issues:

    A text file parse could be a private function in the module,
    but it doesn't fit the platform module API spirit.

    platform module APIs should return meaningful information and
    provide defaults where these cannot be determined. Accordingly,
    an API would have to return a tuple (distname, version, id), just
    like linux_distribution() did.

    Regardless, I don't see the point of opening up this can of
    worms again. We settled on moving Linux distribution version detection
    out of the stdlib and that was a good decision.

    --
    Marc-Andre Lemburg
    eGenix.com

    Professional Python Services directly from the Experts (#1, Nov 25 2020)
    >>> Python Projects, Coaching and Support ...    https://www.egenix.com/
    >>> Python Product Development ...        https://consulting.egenix.com/
    ________________________________________________________________________

    ::: We implement business ideas - efficiently in both time and costs :::

    eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
    Registered at Amtsgericht Duesseldorf: HRB 46611
    https://www.egenix.com/company/contact/
    https://www.malemburg.com/

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 25, 2020

    It's not pointless. The rate of change in the field is why this
    particular API did not work out in practice. It was working fine
    at the time I added it, but then quickly became unmaintainable.

    Yes, it's pointless. You are arguing that os-release is bad because lsb-release was bad. It's like arguing against TOML file format because YAML and JSON have issues.

    os-release was purposely designed to address concerns with lsb-release. os-release has been around for more than a decade and wildly adopted (except on Android platforms). The format hasn't changed since its first appearance in November 2010. Some optional fields were standardized and the meaning of "no shell features are supported" was elaborated on.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 25, 2020

    PS: MAL, would it be possible to suppress your email footer? BPO is not an advertisement channel.

    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Nov 25, 2020

    On 25.11.2020 11:45, Christian Heimes wrote:

    Christian Heimes <lists@cheimes.de> added the comment:

    PS: MAL, would it be possible to suppress your email footer? BPO is not an advertisement channel.

    That's an ancient bug on bpo, which apparently still haven't been
    resolved - it doesn't handle longer sigs correctly. Sorry for that.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 25, 2020

    Pablo, I would like to get mediation from the release manager.

    I want to add an interface to the os-release file. It's a decade-old standard from freedesktop.org. The file is available in the minimal base installation of virtually every Linux distributions except for Android. It's in Alpine, Arch Linux, Debian family (Mint, Raspberry Pi OS, Ubuntu, ...), Fedora family (CentOS, RHEL, ...), Gentoo, SUSE, and many more. The os-release file contains human readable and machine readable operating system information. The file format is simple but not trivial.

    Matthias and Marc-Andre oppose my patch. Victor and I are in favor of the new feature.

    @asvetlov
    Copy link
    Contributor

    @asvetlov asvetlov commented Nov 25, 2020

    I just needed such functionality for PTY tests: #23514

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 25, 2020

    I just needed such functionality for PTY tests

    another use case for a test. so please add it to the test framework, also suggested by Marc.

    @asvetlov
    Copy link
    Contributor

    @asvetlov asvetlov commented Nov 25, 2020

    Don't get me wrong.
    I don't want to put my vote in this hot discussion but mention my use case.
    test.support can solve my need.
    On another hand, very many Python batteries exist to solve third-party needs and are not required by CPython itself.

    So I'm +0 for adding this function -- but can live without it.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 25, 2020

    We have reached an impasse. Therefore I have contacted the steering council and requested mediation. Victor will abstain from the decision process.

    @warsaw
    Copy link
    Member

    @warsaw warsaw commented Nov 30, 2020

    This issue was brought to the Python Steering Council, and we deliberated it at today's SC meeting. With a vote of 4 approvals and one abstention, we have approved the addition of this API.

    @doko42
    Copy link
    Member

    @doko42 doko42 commented Nov 30, 2020

    I don't agree with this decision, but ok.

    In this case, please also provide a way to provide the value of the VERSION_ID field. It doesn't make sense to blacklist whole distributions in tests.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 30, 2020

    Matthias Klose:

    It doesn't make sense to blacklist whole distributions in tests.

    Adding freedesktop_os_release() and using it are two different topics. I understand the the SC decision is on adding the function.

    Using it to skip tests should be decided on a case by case basic. In general, I really dislike to skip tests on a specific Linux distribution.

    In bpo-41818, the os-release file was used as a temporary fix just to unblock buildbots, but it was quickly removed. It was not used to skip a test on Gentoo, but to run the test on Gentoo ;-) Also, it was quickly removed, and it seems like even the skip on BSD was removed. Great! It was there for a very long time.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 30, 2020

    In this case, please also provide a way to provide the value of the VERSION_ID field.

    I'm not sure what you mean. PR 23492 exposes the whole content of the os-release file as a dict. There is no special case for any field.

    VERSION_ID can be retrieved as any field with: platform.freedesktop_os_release().get('VERSION_ID', '').

    Example on Fedora 33.

    $ cat /etc/os-release 
    NAME=Fedora
    VERSION="33 (Workstation Edition)"
    ID=fedora
    VERSION_ID=33
    VERSION_CODENAME=""
    PLATFORM_ID="platform:f33"
    PRETTY_NAME="Fedora 33 (Workstation Edition)"
    ANSI_COLOR="0;38;2;60;110;180"
    LOGO=fedora-logo-icon
    CPE_NAME="cpe:/o:fedoraproject:fedora:33"
    HOME_URL="https://fedoraproject.org/"
    DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/"
    SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
    BUG_REPORT_URL="https://bugzilla.redhat.com/"
    REDHAT_BUGZILLA_PRODUCT="Fedora"
    REDHAT_BUGZILLA_PRODUCT_VERSION=33
    REDHAT_SUPPORT_PRODUCT="Fedora"
    REDHAT_SUPPORT_PRODUCT_VERSION=33
    PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
    VARIANT="Workstation Edition"
    VARIANT_ID=workstation
    $ ./python 
    Python 3.10.0a2+ (heads/pr/23492:6a4c92bcbe, Nov 30 2020, 22:29:23) 
    [GCC 10.2.1 20201016 (Red Hat 10.2.1-6)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import platform, pprint
    >>> os_release=platform.freedesktop_os_release()
    
    >>> pprint.pprint(os_release)
    {'ANSI_COLOR': '0;38;2;60;110;180',
     'BUG_REPORT_URL': 'https://bugzilla.redhat.com/',
     'CPE_NAME': 'cpe:/o:fedoraproject:fedora:33',
     'DOCUMENTATION_URL': 'https://docs.fedoraproject.org/en-US/fedora/f33/system-administrators-guide/',
     'HOME_URL': 'https://fedoraproject.org/',
     'ID': 'fedora',
     'LOGO': 'fedora-logo-icon',
     'NAME': 'Fedora',
     'PLATFORM_ID': 'platform:f33',
     'PRETTY_NAME': 'Fedora 33 (Workstation Edition)',
     'PRIVACY_POLICY_URL': 'https://fedoraproject.org/wiki/Legal:PrivacyPolicy',
     'REDHAT_BUGZILLA_PRODUCT': 'Fedora',
     'REDHAT_BUGZILLA_PRODUCT_VERSION': '33',
     'REDHAT_SUPPORT_PRODUCT': 'Fedora',
     'REDHAT_SUPPORT_PRODUCT_VERSION': '33',
     'SUPPORT_URL': 'https://fedoraproject.org/wiki/Communicating_and_getting_help',
     'VARIANT': 'Workstation Edition',
     'VARIANT_ID': 'workstation',
     'VERSION': '33 (Workstation Edition)',
     'VERSION_CODENAME': '',
     'VERSION_ID': '33'}
    
    >>> os_release['VERSION_ID']
    '33'

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 30, 2020

    New changeset 5c73afc by Christian Heimes in branch 'master':
    bpo-28468: Add platform.freedesktop_os_release() (GH-23492)
    5c73afc

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 30, 2020

    I merged Christian's PR 23492. The initial issue, add a function to parse the os-release file, is now solved, so I close the issue.

    If you want to use it in a test, please open a new issue.

    Thanks everyone for this constructive discussion, IMO the final merged function is now better thanks to that. And thanks Christian for pushing the feature and implementing it ;-)

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Dec 23, 2020

    New changeset bfda4f5 by Victor Stinner in branch 'master':
    bpo-28468: Fix typo in _os_release_candidates (GH-23913)
    bfda4f5

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 stdlib type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    9 participants