Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the major version from sys.platform #57004

Closed
vstinner opened this issue Aug 20, 2011 · 14 comments
Closed

Remove the major version from sys.platform #57004

vstinner opened this issue Aug 20, 2011 · 14 comments

Comments

@vstinner
Copy link
Member

vstinner commented Aug 20, 2011

BPO 12795
Nosy @malemburg, @loewis, @vstinner, @ezio-melotti
Files
  • sys_platform_without_major.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2011-10-14.01:09:42.580>
    created_at = <Date 2011-08-20.16:23:15.361>
    labels = []
    title = 'Remove the major version from sys.platform'
    updated_at = <Date 2011-10-14.01:09:42.579>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2011-10-14.01:09:42.579>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2011-10-14.01:09:42.580>
    closer = 'vstinner'
    components = []
    creation = <Date 2011-08-20.16:23:15.361>
    creator = 'vstinner'
    dependencies = []
    files = ['22964']
    hgrepos = []
    issue_num = 12795
    keywords = ['patch']
    message_count = 14.0
    messages = ['142540', '142542', '142552', '142564', '142704', '142706', '142709', '142712', '142723', '142726', '142730', '142737', '142738', '145495']
    nosy_count = 6.0
    nosy_names = ['lemburg', 'loewis', 'vstinner', 'nadeem.vawda', 'ezio.melotti', 'Arfrever']
    pr_nums = []
    priority = 'normal'
    resolution = 'wont fix'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue12795'
    versions = ['Python 3.3']

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 20, 2011

    sys.platform contains the major system version. If you test the sys.platform value (e.g. sys.platform == 'linux2'), your program doesn't work anymore with the new system major version (e.g. Linux 3). This problem is common with NetBSD, OpenBSD and FreeBSD. You don't have the problem if you test the prefix of sys.platform (e.g. sys.platform.startswith('linux')), but this code pattern is rare.

    Because of the release of Linux 3, it was proposed in bpo-12326 to remove the major version from sys.platform. It is already done for Cygwin and Darwin. Example of new sys.platform values: 'linux', 'freebsd', 'hp-ux', ... (instead of 'linux2', 'freebsd8', hp-ux11', ...).

    I don't know if "osf1" becomes "osf". I don't know if "irix646" becomes "irix6" or just "irix"?

    What about sys.platform=="win32"? Should it be truncated to "win"? Many tests use already sys.platform.startswith("win"). And what about sys.platform=="java"? It would be nice to be consistent (e.g. never have digits in sys.platform).

    Without the major version, it's much easier when you only care of the system name: you can use a dictionary with the name for the key (it's used in regrtest.py with my patch).

    Another example :
    ------------------------
    if sys.platform in ('netbsd1', 'netbsd2', 'netbsd3',
    'Darwin1.2', 'darwin',
    'freebsd2', 'freebsd3', 'freebsd4', 'freebsd5',
    'freebsd6', 'freebsd7', 'freebsd8',
    'bsdos2', 'bsdos3', 'bsdos4',
    'openbsd', 'openbsd2', 'openbsd3', 'openbsd4'):
    ------------------------
    becomes:
    ------------------------
    if sys.platform in ('netbsd', 'freebsd', 'openbsd', 'bsdos',
    'Darwin1.2', 'darwin'):
    ------------------------
    ('Darwin1.2'? WTF?)

    This issue follows my previous commit 50f1922bc1d5:

    Issue bpo-12326: don't test the major version of sys.platform
    Use startswith, instead of ==, when testing sys.platform to support
    new platforms like Linux 3 or OpenBSD 5.

    I chose to keep sys.platform in distutils and packaging because these modules are supposed to work on older Python versions (e.g. packaging will be backported to Python 2.4-3.2).

    Attached patch implements this issue. It requires platform.major(): see issue bpo-12794. The patch require to rerun "autoconf".

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 20, 2011

    To have an idea of how much code has to be changed for the sys.platform change, there is the diffstat :

    $ diffstat sys_platform_without_major.patch 
     Lib/ctypes/util.py                      |    6 --
     Lib/distutils/command/build_ext.py      |    3 -
     Lib/distutils/unixccompiler.py          |    4 -
     Lib/distutils/util.py                   |   12 ++

    Lib/packaging/command/build_ext.py | 3 -
    Lib/packaging/compiler/unixccompiler.py | 6 +-
    Lib/packaging/tests/test_config.py | 4 -
    Lib/sysconfig.py | 12 ++---
    Lib/test/fork_wait.py | 2
    Lib/test/regrtest.py | 71 ++++++++++++++------------------
    Lib/test/support.py | 2
    Lib/test/test_asyncore.py | 2
    Lib/test/test_fcntl.py | 9 +---
    Lib/test/test_locale.py | 2
    Lib/test/test_logging.py | 2
    Lib/test/test_multiprocessing.py | 2
    Lib/test/test_os.py | 6 +-
    Lib/test/test_posix.py | 2
    Lib/test/test_signal.py | 11 ++--
    Lib/test/test_socket.py | 7 ---
    Lib/test/test_tarfile.py | 2
    Lib/test/test_tempfile.py | 2
    Lib/test/test_threading.py | 14 +++---
    Lib/unittest/test/test_break.py | 4 -
    configure.in | 17 ++-----
    setup.py | 19 ++++----
    26 files changed, 105 insertions(+), 121 deletions(-)

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented Aug 20, 2011

    I think this change should be much much smaller in scope. It was (nearly) agreed to drop the major version if the system is Linux. There is no consensus (that I'm aware of) to drop the major OS version for all systems.

    So I would propose to do this *only* selectively, and *only* if experts of the platform request such a change. For example, hpux9, hpux10, and hpux11 are *very* different operating systems; it's not clear that users are helped if we drop the major version for them. Also, if HP-UX 12 ever gets released, it is probably again different.

    I'm not sure sure about the BSDs: it is my understanding that they follow the tradition of only adding major changes to major OS releases, which would be in favor of keeping the status quo for them. OTOH, it may also be that the impact of the major version on Python and Python applications is too small so that applications would be better helped with dropping the major version.

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 20, 2011

    At the beginning, I thaught that it would be better to remove the major version from sys.platform on all platforms. When I started to write the patch, I expected to change only 2 or 3 files. But the patch is now huge ("26 files changed, 105 insertions(+), 121 deletions(-)"), which means that it will break most third party projects.

    I would propose to do this *only* selectively,
    and *only* if experts of the platform request such a change.

    Ok, I (now) agree. So let's close this issue.

    @ezio-melotti
    Copy link
    Member

    ezio-melotti commented Aug 22, 2011

    Having the major version in sys.platform only for a few selected platforms doesn't seem too consistent for me.

    If linux 4 will introduce major changes, will we have to look for the major version of freebsd in sys.platform and for the linux major version elsewhere?

    That said, removing the major version for the other platforms might introduce unnecessary breakage, but if it's ever going to happen, I think it's better to do it now for all the platforms and not waiting for the experts to speak and change it in later versions, adding even more inconsistency.

    @malemburg
    Copy link
    Member

    malemburg commented Aug 22, 2011

    Moved the discussion here from bpo-12326:

    > [Larry Hastings]
    > >> If we're changing "linux2" / "linux3" to just "linux", we should be
    > >> consistent and do it for everybody. I propose sys.platform under 3.3
    > >> should contain things like "linux", "freebsd", "openbsd", "darwin",
    > >> and "windows".

    [Martin v. Löwis]
    > Definitely not. The reasoning that applies to Linux doesn't necessarily
    > apply to the other systems. My understanding that it definitely does not
    > apply to HP-UX, where major version number changes really indicate major
    > changes (unlike in Linux).

    Actually, with that reasoning we would need to reintroduce the
    version for Mac OS, and even go a step further and add the minor
    version number as well, since since major changes have happened on Mac OS
    with every single minor release for the last couple of years.

    IMO, a better approach is to split the information in two parts:

    • sys.platform, which only specifies the platform name on which
      Python was built (uname -s)

    • sys.platform_build_version, which provides the full platform
      version (uname -r; either as string or as tuple or both -
      that would have to be hashed out)

    It is obvious that the version number in sys.platform often
    doesn't really get used (see Victor's example). In such cases,
    having access to just the build platform name is better.
    In other cases, where you do need to know the version (e.g.
    on Mac OS X), you can then easily get if from the new attribute,
    and including the minor and even patch level release version
    details (e.g. to determine whether Python was compiled with
    a Windows specific service pack in place or not).

    @malemburg malemburg reopened this Aug 22, 2011
    @malemburg malemburg removed the invalid label Aug 22, 2011
    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 22, 2011

    FreeBSD or OpenBSD release major version frequently, something like one
    per year, or one per two years. FreeBSD and OpenBSD developers knows
    that for years, and Python programs use sys.platform.startswith() for
    these OSes.

    For Linux, it's different. Linux 2.0 was released in 1997 and 3.0 in
    2011: it took 14 years to change the major version. It don't think that
    any program working on Linux was prepared for this change: see bpo-12326
    history to have an idea on the problem. It looks like
    sys.platform=='linux3' breaks most programs testing sys.platform
    (including Python itself because of Lib/plat-linux2/ directory).

    If you want the OS name, use platform.system() or os.uname()[0].

    If you want the OS version, use platform.release(). If you want the OS
    version as a tuple, hum... see the issue bpo-12794.

    @malemburg
    Copy link
    Member

    malemburg commented Aug 22, 2011

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    FreeBSD or OpenBSD release major version frequently, something like one
    per year, or one per two years. FreeBSD and OpenBSD developers knows
    that for years, and Python programs use sys.platform.startswith() for
    these OSes.

    For Linux, it's different. Linux 2.0 was released in 1997 and 3.0 in
    2011: it took 14 years to change the major version. It don't think that
    any program working on Linux was prepared for this change: see bpo-12326
    history to have an idea on the problem. It looks like
    sys.platform=='linux3' breaks most programs testing sys.platform
    (including Python itself because of Lib/plat-linux2/ directory).

    If you want the OS name, use platform.system() or os.uname()[0].

    If you want the OS version, use platform.release(). If you want the OS
    version as a tuple, hum... see the issue bpo-12794.

    Victor, you are constantly mixing build time information with
    runtime information. Those are two different types of information
    and we should regard them as such.

    sys.platform refers to build time information, so the platform
    module won't help.

    The version of the build platform is important to know, but
    adding it to sys.platform is not necessarily the right
    thing to do, since adding just the major version is often
    not enough (see Mac OS X) and can sometimes lead to breakage
    due to frequent new releases of OSes (see FreeBSD).

    Given that the various OSes use different schemes for versioning
    and backwards compatibility, a single string cannot possibly
    cover all aspects.

    Separating the information into OS name and version is a much
    more future proof approach.

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 22, 2011

    sys.platform refers to build time information,
    so the platform module won't help.

    When I wrote my patch, I realized that sys.platform is used to get the runtime information, which is wrong. See the tests in Lib/test/test_*.py: they use sys.platform to check features of the running OS (kernel). It is mostly to test FreeBSD major version, and it looks like Python is build on the same FreeBSD major version that it is running on. Maybe because on FreeBSD, most programs are compiled instead of using prebuild packages (pkgadd).

    @malemburg
    Copy link
    Member

    malemburg commented Aug 22, 2011

    STINNER Victor wrote:

    STINNER Victor <victor.stinner@haypocalc.com> added the comment:

    > sys.platform refers to build time information,
    > so the platform module won't help.

    When I wrote my patch, I realized that sys.platform is used to get the runtime information, which is wrong. See the tests in Lib/test/test_*.py: they use sys.platform to check features of the running OS (kernel). It is mostly to test FreeBSD major version, and it looks like Python is build on the same FreeBSD major version that it is running on. Maybe because on FreeBSD, most programs are compiled instead of using prebuild packages (pkgadd).

    Right. FreeBSD doesn't ship prebuilt binaries for packages -
    it uses a very elegant "ports" collection, where everything gets
    compiled on the target machine:

    http://www.freebsd.org/ports/
    

    So you don't really run into the issue of using a older Python build
    on the system, unless you have an application which ships Python
    along with the application package.

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 22, 2011

    > It is mostly to test FreeBSD major version, and it looks like Python is build on the same FreeBSD major version that it is running on. Maybe because on FreeBSD, most programs are compiled instead of using prebuild packages (pkgadd).

    Right. FreeBSD doesn't ship prebuilt binaries for packages

    FreeBSD does ship prebuild Python packages.

    @malemburg
    Copy link
    Member

    malemburg commented Aug 22, 2011

    STINNER Victor wrote:
    > 
    > STINNER Victor <victor.stinner@haypocalc.com> added the comment:
    > 
    >>> It is mostly to test FreeBSD major version, and it looks like Python is build on the same FreeBSD major version that it is running on. Maybe because on FreeBSD, most programs are compiled instead of using prebuild packages (pkgadd).
    >>
    >> Right. FreeBSD doesn't ship prebuilt binaries for packages
    > 
    > FreeBSD does ship prebuild Python packages.

    The FreeBSD base distribution (which has the binaries
    and, AFAIK, the only binaries that ship with FreeBSD) doesn't
    include Python. It's in the ports collection, though,
    and for those you have the option of installing prebuilt
    binaries of the packages via pkg_add.

    Still, that's not exactly the FreeBSD way of doing things :)
    See the big warning in the pkg_add man page, various discussions
    on the net and the pros/cons in the handbook:
    http://www.freebsd.org/doc/handbook/ports-overview.html

    But in general you are right: if you use system provided Python
    packages, chances are that your runtime OS version will be
    identical or very close to the build time OS version.

    @vstinner
    Copy link
    Member Author

    vstinner commented Aug 22, 2011

    But in general you are right: if you use system provided Python
    packages, chances are that your runtime OS version will be
    identical or very close to the build time OS version.

    I asked on #freebsd-fr: even with pkg_add, the installed Python is compiled on the same major version than the running major version. So the problem is "hidden" (it doesn't affect) on FreeBSD.

    @vstinner
    Copy link
    Member Author

    vstinner commented Oct 14, 2011

    While working on bpo-12326, it's realized that many modules and applications rely on sys.platform. Change this value should be decided for each platform because it breaks backward compatibility. Linux was a special case and now I don't think that we should drop the major version from sys.platform on all platforms. Let's close this issue as "wont fix".

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    None yet
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants