Linux 3: code should avoid using sys.platform == 'linux2'

[neologix]

BPO	12326
Nosy	@malemburg, @loewis, @warsaw, @birkenfeld, @doko42, @amauryfa, @pitrou, @vstinner, @larryhastings, @benjaminp, @jwilk, @djc, @ezio-melotti, @merwok, @bitdancer, @davidmalcolm, @sandrotosi, @akheron
Files	linux3-v2.patch configure_linux2.python3.2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2011-09-05.20:39:30.835>
created_at = <Date 2011-06-13.14:34:41.657>
labels = ['type-bug', 'release-blocker']
title = "Linux 3: code should avoid using sys.platform == 'linux2'"
updated_at = <Date 2011-10-04.14:24:53.642>
user = 'https://bugs.python.org/neologix'

bugs.python.org fields:

activity = <Date 2011-10-04.14:24:53.642>
actor = 'barry'
assignee = 'none'
closed = True
closed_date = <Date 2011-09-05.20:39:30.835>
closer = 'vstinner'
components = []
creation = <Date 2011-06-13.14:34:41.657>
creator = 'neologix'
dependencies = []
files = ['22917', '22946']
hgrepos = []
issue_num = 12326
keywords = ['patch']
message_count = 145.0
messages = ['138251', '138252', '138253', '138254', '138255', '138271', '138294', '138295', '138298', '138299', '138300', '138306', '138308', '138527', '138582', '138597', '138711', '138730', '138757', '138759', '138816', '138817', '138823', '138824', '139166', '139208', '139224', '139243', '139248', '140012', '140017', '140061', '140062', '140063', '140064', '140065', '140394', '140397', '140684', '140703', '140708', '140710', '140801', '140802', '140830', '140831', '140939', '140944', '140949', '140955', '140985', '141088', '141651', '141659', '142195', '142196', '142197', '142219', '142226', '142272', '142290', '142291', '142297', '142298', '142313', '142344', '142345', '142346', '142350', '142353', '142354', '142356', '142358', '142379', '142380', '142381', '142382', '142384', '142385', '142387', '142389', '142394', '142396', '142398', '142401', '142406', '142407', '142408', '142409', '142410', '142417', '142418', '142419', '142420', '142425', '142440', '142441', '142444', '142446', '142450', '142451', '142453', '142457', '142459', '142460', '142461', '142463', '142482', '142490', '142493', '142498', '142499', '142500', '142524', '142531', '142539', '142541', '142570', '142571', '142574', '142579', '142593', '142605', '142614', '142615', '142628', '142635', '142678', '142694', '142697', '142698', '142700', '142707', '142711', '142762', '142766', '142769', '143455', '143470', '143479', '143557', '143561', '143562', '144889', '144890']
nosy_count = 25.0
nosy_names = ['lemburg', 'loewis', 'barry', 'georg.brandl', 'doko', 'amaury.forgeotdarc', 'gagern', 'foom', 'pitrou', 'vstinner', 'larry', 'benjamin.peterson', 'jwilk', 'djc', 'ezio.melotti', 'eric.araujo', 'Arfrever', 'Retro', 'r.david.murray', 'dmalcolm', 'sandro.tosi', 'rosslagerwall', 'python-dev', 'petri.lehtinen', 'Ramchandra Apte']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12326'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[neologix]

Linus recently decided that the next Linux kernel version would 3.0 [1].
As a consequence, sys.platform (which is actually MACHDEP) will be 'linux3' on machines running Linux 3 kernels, and tests checking explicitely against 'linux2' will either break and won't run.
A quick grep through the code base returns only a couple problematic places, but this should definitely be fixed.
For information, here's a - probably incomplete - list of such occurrences:

"""
./Lib/test/test_logging.py: if sys.platform in ('linux2', 'darwin'):
./Lib/test/test_sysconfig.py: sys.platform = 'linux2'
./Lib/test/regrtest.py: 'linux2':
./Lib/test/test_socket.py: if sys.platform == 'linux2':
./Lib/test/test_tarfile.py: if sys.platform == "linux2":
./Lib/distutils/tests/test_bdist_rpm.py: if sys.platform != 'linux2':
./Lib/distutils/tests/test_bdist_rpm.py: if sys.platform != 'linux2':
./Lib/distutils/tests/test_util.py: sys.platform = 'linux2'
./Lib/packaging/tests/test_config.py: inotify (0.0.1); sys.platform == 'linux2'
./Lib/packaging/tests/test_config.py: "inotify (0.0.1); sys.platform == 'linux2'"]
./setup.py: if (platform in ('linux2', 'freebsd4', 'freebsd5', 'freebsd6',
"""

[1] http://thread.gmane.org/gmane.linux.kernel/1147415

[vstinner]

Why should be used instead?

• sys.platform.startswith('linux')
• os.uname()[0] == 'Linux'
• platform.system() == 'Linux'
• other?

Tests like sys.platform in ('linux2', 'darwin') can be replaced by sys.platform in ('linux2', 'linux3', 'darwin'). We will have to patch this test again for Linux 4.

[pitrou]

I would expect changing sys.platform will also break a lot of third-party code. Perhaps sys.platform can still be 'linux2' under Linux 3.x? After all, there's no significant change that deserves changing sys.platform.

[vstinner]

sys.platform comes from Py_GetPlatform() which comes from PLATFORM define. On Linux, this define comes from Makefile: MACHDEP variable which comes from configure. Finally, MACHDEP is defined by:

ac_sys_system=`uname -s`
if test "$ac_sys_system" = "AIX" \
-o "$ac_sys_system" = "UnixWare" -o "$ac_sys_system" = "OpenUNIX"; then
	ac_sys_release=`uname -v`
else
	ac_sys_release=`uname -r`
fi
ac_md_system=`echo $ac_sys_system |
		   tr -d '/ ' | tr '[A-Z]' '[a-z]'`
ac_md_release=`echo $ac_sys_release |
		   tr -d '/ ' | sed 's/^[A-Z]\.//' | sed 's/\..*//'`
MACHDEP="$ac_md_system$ac_md_release"

[neologix]

I would expect changing sys.platform will also break a lot of third-
party code.

Maybe, but this would be an application bug.

Here's sys.platform current implementation:

const char *
Py_GetPlatform(void)
{
    return PLATFORM;
}

And here's the documentation, from http://docs.python.org/c-api/init.html

"""
Return the platform identifier for the current platform. On Unix, this is formed from the “official” name of the operating system, converted to lower case, followed by the major revision number"""

So it's actually documented.

Perhaps sys.platform can still be 'linux2' under Linux 3.x? After
all, there's no significant change that deserves changing
sys.platform.

Sounds like a recipe for confusion.

[loewis]

I suggest to change sys.platform to 'linux' in future releases (3.3). For bugfix releases (2.7, 3.2), I'd keep it as 'linux2' even on Linux 3. For security-only releases (2.6, 3.1), no change should be made.

[amauryfa]

This change is reasonable for the long term. But it *will* break a lot of code.

[loewis]

This change is reasonable for the long term. But it *will* break a lot of code.

[If you favor a specific change, please indicate what that is. I'm
assuming you support my proposal for the moment :-]

I agree it will break a lot of code, but it's also somewhat urgent
because we will get 'linux3' if we don't act, which will also break
a lot of code (but more subtly, since people testing their code may
do so on Linux 2, only to get bug reports that it breaks on "some"
Linux systems). I'm sure Linus Torvalds is fully aware of the possible
consequences of the version change, and just accepted the breakage
that this would cause.

It's important that we set a policy before the Linux distributions
do (which may end up choosing different policies). We don't actually
have to *release* this change quickly, since Linux distributions
who release 3.x kernels will fix their Python packages themselves.

[amauryfa]

The change to sys.platform=='linux' would break code even on current platforms.
OTOH, we have sys.platform=='win32' even on Windows 64bit; would this favor keeping 'linux2' on all versions of Linux as well?

[loewis]

The change to sys.platform=='linux' would break code even on current platforms.

Correct. Compared to introducing 'linux3', I consider this the better
change - it likely breaks earlier (i.e. when porting to Python 3.3).

OTOH, we have sys.platform=='win32' even on Windows 64bit; would this
favor keeping 'linux2' on all versions of Linux as well?

While this has better compatibility, it's also a constant source of
irritation. Introducing 'win64' would have been a worse choice (just
as introducing 'linux3' would: incompatibility for no gain, since
the distinction between win32 and win64, from a Python POV, is
irrelevant). Plus, Microsoft dislikes the term Win64 somewhat, and
rather wants people to refer to the "Windows API".

I personally disliked 'linux2' when it was introduced, for its
incompatibilities. Anticipating that, some day, we may have 'Linux 4',
and so on, I still claim it is better to fix this now. We could even
come up with a 2to3 fixer for people who dual-source their code.

[neologix]

I'm sure Linus Torvalds is fully aware of the possible
consequences of the version change, and just accepted the breakage
that this would cause.

Any application relying on sys.platform == 'linux2' is already broken.
It's exactly the same if an application checks for 'freebsd6' or
'openbsd4' : if you want to check for a specific operating system,
there's already POSIX' struct utsname sysname field, i.e. uname()[0].

[malemburg]

Martin v. Löwis wrote:

Martin v. Löwis <martin@v.loewis.de> added the comment:

> The change to sys.platform=='linux' would break code even on current platforms.

Correct. Compared to introducing 'linux3', I consider this the better
change - it likely breaks earlier (i.e. when porting to Python 3.3).

> OTOH, we have sys.platform=='win32' even on Windows 64bit; would this
> favor keeping 'linux2' on all versions of Linux as well?

While this has better compatibility, it's also a constant source of
irritation. Introducing 'win64' would have been a worse choice (just
as introducing 'linux3' would: incompatibility for no gain, since
the distinction between win32 and win64, from a Python POV, is
irrelevant). Plus, Microsoft dislikes the term Win64 somewhat, and
rather wants people to refer to the "Windows API".

I personally disliked 'linux2' when it was introduced, for its
incompatibilities. Anticipating that, some day, we may have 'Linux 4',
and so on, I still claim it is better to fix this now. We could even
come up with a 2to3 fixer for people who dual-source their code.

I think we should consider adding a better mechanism and just
keep the old mechanism for determining sys.platform in place
(letting it break on Linux to raise awareness) and add a new better
attribute along the lines of what Martin suggested:

sys.system == 'linux', 'freebsd', 'openbsd', 'windows', etc.
(without version)

and a new

sys.system_info == system release info (named) tuple similar to
sys.version_info

to query a specific system version.

As already noted, direct sys.platform testing already breaks for
OpenBSD, FreeBSD and probably a few other OSes as well with
every major OS release, so the Linux breakage is not really new
in any way.

[pitrou]

> The change to sys.platform=='linux' would break code even on current platforms.

Correct. Compared to introducing 'linux3', I consider this the better
change - it likely breaks earlier (i.e. when porting to Python 3.3).

FWIW, I also agree that sys.platform == 'linux' would be the better
choice. It seems there's little point having the kernel's major version
there. It's both disruptive and useless (because features are introduced
at any point, not just at new major versions).

[merwok]

Martin’s sys.platform = 'linux' sounds good to me too.