Make the meataxe wrapper work with py3

[simon-king-jena]

In a py3-build of Sage, installation of the meataxe spkg works, but the wrapper doesn't:

sage: type(random_matrix(GF(9,'x'), 1000))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-01e73e06d744> in <module>()
----> 1 type(random_matrix(GF(Integer(9),'x'), Integer(1000)))

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/matrix/special.py in random_matrix(ring, nrows, ncols, algorithm, implementation, *args, **kwds)
    597     sparse = kwds.pop('sparse', False)
    598     # Construct the parent of the desired matrix
--> 599     parent = matrix_space.MatrixSpace(ring, nrows, ncols, sparse=sparse, implementation=implementation)
    600     if algorithm == 'randomize':
    601         density = kwds.pop('density', None)

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/misc/classcall_metaclass.pyx in sage.misc.classcall_metaclass.ClasscallMetaclass.__call__ (build/cythonized/sage/misc/classcall_metaclass.c:1701)()
    328         """
    329         if cls.classcall is not None:
--> 330             return cls.classcall(cls, *args, **kwds)
    331         else:
    332             # Fast version of type.__call__(cls, *args, **kwds)

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/matrix/matrix_space.py in __classcall__(cls, base_ring, nrows, ncols, sparse, implementation)
    474             raise OverflowError("number of rows and columns may be at most %s" % sys.maxsize)
    475 
--> 476         matrix_cls = get_matrix_class(base_ring, nrows, ncols, sparse, implementation)
    477         return super(MatrixSpace, cls).__classcall__(
    478                 cls, base_ring, nrows, ncols, sparse, matrix_cls)

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/matrix/matrix_space.py in get_matrix_class(R, nrows, ncols, sparse, implementation)
    253                 elif R.order() <= 255:
    254                     try:
--> 255                         from . import matrix_gfpn_dense
    256                     except ImportError:
    257                         implementation = 'generic'

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/matrix/matrix_gfpn_dense.pyx in init sage.matrix.matrix_gfpn_dense (build/cythonized/sage/matrix/matrix_gfpn_dense.c:20584)()
     60 
     61 # The following import is just to ensure that meataxe_init() is called.
---> 62 import sage.libs.meataxe
     63 
     64 ####################

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/libs/meataxe.pyx in init sage.libs.meataxe (build/cythonized/sage/libs/meataxe.c:2493)()
     85     MtxSetErrorHandler(sage_meataxe_error_handler)
     86 
---> 87 meataxe_init()

/home/king/Sage/git/py3/local/lib/python3.6/site-packages/sage/libs/meataxe.pyx in sage.libs.meataxe.meataxe_init (build/cythonized/sage/libs/meataxe.c:1740)()
     81     from sage.env import DOT_SAGE
     82     global MtxLibDir
---> 83     MtxLibDir = PyBytes_AsString(os.path.join(DOT_SAGE,'meataxe'))
     84     ## Error handling for MeatAxe, to prevent immediate exit of the program
     85     MtxSetErrorHandler(sage_meataxe_error_handler)

TypeError: expected bytes, str found

Component: packages: optional

Keywords: meataxe py3

Author: Simon King

Branch/Commit: b39ff35

Reviewer: Jeroen Demeyer

Issue created by migration from https://trac.sagemath.org/ticket/27152

[simon-king-jena]

comment:1

Indeed,

sage: type(os.path.join(DOT_SAGE,'meataxe'))
<class 'str'>

So, question from a py3 newbie: How to assign os.path.join(DOT_SAGE,'meataxe') to MtxLibDir (which is char*?

[dimpase]

comment:2

I'd try os.path.join(DOT_SAGE,b'meataxe')

[simon-king-jena]

comment:3

See comment 16 at #21437, where I was told to use PyBytes_... instead of PyString_... for Python 3 compatibility.

So, the question from my perspective is: What is more py3tonic? Using b'meataxe', as in Dima's comment? Or return to PyString?

[simon-king-jena]

comment:4

Apparently I should use PyString_:

sage: type(os.path.join(DOT_SAGE,b'meataxe'))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-7c4dcce91a7c> in <module>()
----> 1 type(os.path.join(DOT_SAGE,b'meataxe'))

/home/king/Sage/git/py3/local/lib/python3.6/posixpath.py in join(a, *p)
     92                 path += sep + b
     93     except (TypeError, AttributeError, BytesWarning):
---> 94         genericpath._check_arg_types('join', a, *p)
     95         raise
     96     return path

/home/king/Sage/git/py3/local/lib/python3.6/genericpath.py in _check_arg_types(funcname, *args)
    149                             (funcname, s.__class__.__name__)) from None
    150     if hasstr and hasbytes:
--> 151         raise TypeError("Can't mix strings and bytes in path components") from None

TypeError: Can't mix strings and bytes in path components

[dimpase]

comment:5

sorry, I misread your error message. Of course 'meataxe' is fine there.
Could it be b'x', not 'x' in GF(9,'x') ?

[simon-king-jena]

comment:6

Replying to @dimpase:

sorry, I misread your error message. Of course 'meataxe' is fine there.
Could it be b'x', not 'x' in GF(9,'x') ?

Why should it? The traceback says that the error occurs in the function meataxe_init, in the line that translates os.path.join(...) to some global constant of type char*. Hence, has nothing to do with GF(9,'x'), which does work, by the way:

sage: GF(9,'x')
Finite Field in x of size 3^2

[simon-king-jena]

comment:7

A duckduckgo search seems to indicate that I should use PyUnicode_AsString, but I don't know where to cimport it from.

[dimpase]

comment:8

https://bugs.python.org/issue2799 tells you that PyUnicode_AsString is gone.

But yes, of course, you are right, os.path.join(DOT_SAGE,'meataxe') is not bytes, but a string, so you cannot call PyBytes_... on it, as its docs say.

Sorry, long day, already at 6am I was writing an email to a student explaining that a class sum lies in the centraliser, and debugging his GAP code along the way :-)

[simon-king-jena]

Branch: u/SimonKing/make_the_meataxe_wrapper_work_with_py3

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

6978c5b

Py3: Use str in error messages