linux kernel 3.0 causes headaches with some python modules

[kiwifb]

This is quite fun. Python setup looks at linux major version number and will build some modules in a folder normally called plat-linux2. With linux 3.0 it looks for the non-existent plat-linux3.
Upstream report is here:
http://bugs.python.org/issue12326
Not completely sure if there are implication for sage or any of its components apart from matplotlib where they updated some code to avoid the problem:
matplotlib/matplotlib@aaef944

One of main problem is that a lot of packages use

sys.platform == 'linux2'

when they should use something like

sys.platform.startswith('linux')

So we will have to fix python (I will do something in #9958 once I figured out what upstream is doing) and potentially some other packages which are looking for linux2.

I think it is too late to do something for 4.7.1 but we should look into fixing 4.7.2.

Upstream: Completely fixed; Fix reported upstream

Component: packages: standard

Reviewer: Jeroen Demeyer

Issue created by migration from https://trac.sagemath.org/ticket/11619

[nexttime]

comment:1

Haha. First step is perhaps to create a symbolic link from plat-linux3 to plat-linux2. And then overwrite sys.platform for the moment for other packages... ;-)

[kiwifb]

comment:2

That's the kind of solution debated (well copying not linking. More fun is the checking for linux2:

grep -r linux2 /usr/lib64/python2.7/site-packages/*
/usr/lib64/python2.7/site-packages/IPython/Extensions/InterpreterExec.py:        sys.platform is: linux2
Binary file /usr/lib64/python2.7/site-packages/IPython/Extensions/InterpreterExec.pyc matches
Binary file /usr/lib64/python2.7/site-packages/IPython/Extensions/InterpreterExec.pyo matches
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/decorators_trial.pyo matches
/usr/lib64/python2.7/site-packages/IPython/testing/decorators_trial.py:skip_linux = skipif(sys.platform == 'linux2',
/usr/lib64/python2.7/site-packages/IPython/testing/decorators_trial.py:skip_if_not_linux = skipif(sys.platform != 'linux2',
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/decorators_trial.pyc matches
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/decorators.pyc matches
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/decorators.pyo matches
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/tests/test_decorators_trial.pyc matches
/usr/lib64/python2.7/site-packages/IPython/testing/tests/test_decorators.py:    nt.assert_not_equals(sys.platform,'linux2',"This test can't run under linux")
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/tests/test_decorators_trial.pyo matches
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/tests/test_decorators.pyo matches
Binary file /usr/lib64/python2.7/site-packages/IPython/testing/tests/test_decorators.pyc matches
/usr/lib64/python2.7/site-packages/IPython/testing/tests/test_decorators_trial.py:        self.assertNotEquals(sys.platform,'linux2',"This test can't run under linux")
/usr/lib64/python2.7/site-packages/IPython/testing/decorators.py:skip_linux = skipif(sys.platform == 'linux2',
/usr/lib64/python2.7/site-packages/IPython/testing/decorators.py:skip_if_not_linux = skipif(sys.platform != 'linux2',
Binary file /usr/lib64/python2.7/site-packages/OpenGL/__init__.pyo matches
/usr/lib64/python2.7/site-packages/OpenGL/__init__.py:PlatformPlugin( 'linux2', 'OpenGL.platform.glx.GLXPlatform' )
Binary file /usr/lib64/python2.7/site-packages/OpenGL/__init__.pyc matches
Binary file /usr/lib64/python2.7/site-packages/networkx/drawing/nx_agraph.pyc matches
/usr/lib64/python2.7/site-packages/networkx/drawing/nx_agraph.py:    cmds = {'darwin': 'open', 'linux2': 'xdg-open', 'win32': 'start'}
Binary file /usr/lib64/python2.7/site-packages/networkx/drawing/nx_agraph.pyo matches
/usr/lib64/python2.7/site-packages/numpy/distutils/exec_command.py:  posix   | linux2       | Debian (sid) Linux, Python 2.1.3+, 2.2.3+, 2.3.3
/usr/lib64/python2.7/site-packages/numpy/distutils/exec_command.py:  posix   | linux2       | Red Hat 9 Linux, Python 2.1.3, 2.2.2, 2.3.2
Binary file /usr/lib64/python2.7/site-packages/numpy/distutils/exec_command.pyo matches
/usr/lib64/python2.7/site-packages/numpy/distutils/cpuinfo.py:if sys.platform.startswith('linux'): # variations: linux2,linux-i386 (any others?)
Binary file /usr/lib64/python2.7/site-packages/numpy/distutils/exec_command.pyc matches
/usr/lib64/python2.7/site-packages/scipy/signal/tests/test_signaltools.py:if sys.platform == 'linux2' and platform.architecture()[0] == '64bit':
Binary file /usr/lib64/python2.7/site-packages/scipy/signal/tests/test_signaltools.pyc matches
Binary file /usr/lib64/python2.7/site-packages/scipy/signal/tests/test_signaltools.pyo matches
/usr/lib64/python2.7/site-packages/sqlalchemy/connectors/mxodbc.py:        # this can be the string "linux2", and possibly others
Binary file /usr/lib64/python2.7/site-packages/sympy/thirdparty/pyglet/pyglet/lib.pyc matches
Binary file /usr/lib64/python2.7/site-packages/sympy/thirdparty/pyglet/pyglet/lib.pyo matches
/usr/lib64/python2.7/site-packages/sympy/thirdparty/pyglet/pyglet/lib.py:        if self.platform == 'linux2':
/usr/lib64/python2.7/site-packages/sympy/thirdparty/pyglet/pyglet/lib.py:elif sys.platform == 'linux2':
/usr/lib64/python2.7/site-packages/sympy/galgebra/latex_ex.py:            if sys.platform == 'linux2':
Binary file /usr/lib64/python2.7/site-packages/sympy/galgebra/latex_ex.pyo matches
Binary file /usr/lib64/python2.7/site-packages/sympy/galgebra/latex_ex.pyc matches
Binary file /usr/lib64/python2.7/site-packages/twisted/internet/test/test_unix.pyo matches
Binary file /usr/lib64/python2.7/site-packages/twisted/internet/test/test_unix.pyc matches
/usr/lib64/python2.7/site-packages/twisted/internet/test/test_unix.py:    if platform != 'linux2':
/usr/lib64/python2.7/site-packages/twisted/internet/test/test_unix.py:    if platform != 'linux2':
/usr/lib64/python2.7/site-packages/twisted/internet/test/test_unix.py:    if platform != 'linux2':

ipython, distutils, numpy, scipy, sympy, networkx, sqlalchemy and twisted.

And there are stuff that don't install in site-packages like scons.

grep -r linux2 /usr/lib64/scons-2.0.1/*
/usr/lib64/scons-2.0.1/SCons/Tool/intelc.py:is_linux = sys.platform == 'linux2'

I think that's a check for the intel compiler so we shouldn't worry about that one....

[nexttime]

comment:3

Well, the impact seems to be less hard, as sys.platform appears to be a constant of the Python installation determined at compile time.

So any Python installation / binary package built on a 2.x kernel should work seamlessly on a 3.x one.

Still funny to read the discussion at the Python BTS.

[nexttime]

comment:4

Replying to @kiwifb:

ipython, distutils, numpy, scipy, sympy, networkx, sqlalchemy and twisted.

And there are stuff that don't install in site-packages like scons.

There's also (in the Sage installation):

local/lib/python2.6/hgext/inotify/server.py-                raise
local/lib/python2.6/hgext/inotify/server.py-
local/lib/python2.6/hgext/inotify/server.py:if sys.platform == 'linux2':
local/lib/python2.6/hgext/inotify/server.py-    import linuxserver as _server
local/lib/python2.6/hgext/inotify/server.py-else:
--
local/lib/python2.6/posixfile.py-                l_type, l_whence, l_start, l_len,  l_sysid, l_pid, l_vfs = \
local/lib/python2.6/posixfile.py-                    struct.unpack('hhlllii', flock)
local/lib/python2.6/posixfile.py:            elif sys.platform == "linux2":
local/lib/python2.6/posixfile.py-                l_type, l_whence, l_start, l_len, l_pid, l_sysid = \
local/lib/python2.6/posixfile.py-                    struct.unpack('hhllhh', flock)

[jdemeyer]

comment:5

This is now "fixed" by upstream by only using linux2, even on Linux 3.

[jdemeyer]

Changed upstream from Reported upstream. Developers acknowledge bug. to Completely fixed; Fix reported upstream

[jdemeyer]

Reviewer: Jeroen Demeyer