Exception on startup: TypeError: GLib.filename_to_utf8() takes exactly 2 arguments (4 given)

[jtojnar]

Hello,

I installed mypaint from git and it is crashing on startup. My glib2 version is 2.48.0.5. The same problem also occurs with glib 2.45.6.42 (the earliest version I can downgrade to).

INFO: mypaint: Installation layout: conventional POSIX-like structure with prefix u'/usr'
INFO: lib.i18n: POSIX: LANG='en_GB.UTF-8'
INFO: lib.i18n: POSIX: LANGUAGE=None
Traceback (most recent call last):
  File "/usr/sbin/mypaint", line 464, in <module>
    main.main(datapath, iconspath, old_confpath, version=version)
  File "/usr/share/mypaint/gui/main.py", line 92, in main
    lib.glib.init_user_dir_caches()
  File "/usr/share/mypaint/lib/glib.py", line 109, in init_user_dir_caches
    logger.debug("Init g_get_user_config_dir(): %r", get_user_config_dir())
  File "/usr/share/mypaint/lib/glib.py", line 71, in get_user_config_dir
    return filename_to_unicode(d_fs)
  File "/usr/share/mypaint/lib/glib.py", line 57, in filename_to_unicode
    ustring = GLib.filename_to_utf8(opsysstring, -1, 0, 0)
TypeError: GLib.filename_to_utf8() takes exactly 2 arguments (4 given)

[odysseywestra]

Thanks for bug report! Could you help us out by telling us which Linux Distro you installed this on? Plus what were your steps to install it? Did you manually build, or did you use something like the AUR?

[odysseywestra]

Here's reference info for filename_to_utf8 in Glib:

g_filename_to_utf8 ()

gchar *
g_filename_to_utf8 (const gchar *opsysstring,
                    gssize len,
                    gsize *bytes_read,
                    gsize *bytes_written,
                    GError **error);

Converts a string which is in the encoding used by GLib for filenames into a UTF-8 string. Note that on Windows GLib uses UTF-8 for filenames; on other platforms, this function indirectly depends on the current locale.
Parameters

opsysstring: a string in the encoding for filenames

len: the length of the string, or -1 if the string is nul-terminated (Note that some encodings may allow nul bytes to occur inside strings. In that case, using -1 for the len parameter is unsafe)

bytes_read: location to store the number of bytes in the input string that were successfully converted, or NULL. Even if the conversion was successful, this may be less than len if there were partial characters at the end of the input. If the error G_CONVERT_ERROR_ILLEGAL_SEQUENCE occurs, the value stored will the byte offset after the last valid input sequence.
    [out][optional]

bytes_written: the number of bytes stored in the output buffer (not including the terminating nul).
    [out][optional]

error: location to store the error occurring, or NULL to ignore errors. Any of the errors in GConvertError may occur.

Taken from Glib's Reference Site

[jtojnar]

@odysseywestra I use Manjaro (Arch derivative) and I installed it from AUR mypaint-git package.

[jtojnar]

Turns out I had 3.18 version of pygobject so I upgraded. It did not help though.

Name            : python2-gobject
Version         : 3.20.0-1
Name            : pygobject-devel
Version         : 3.20.0-1
Name            : gobject-introspection-git
Version         : 1:1.48.0.3532.42bf17a-1

[jtojnar]

This simple code produces the same error.

#!/usr/bin/python2
from gi.repository import GLib
print(GLib.filename_to_utf8("test", -1, 0, 0))

If I remove the last two arguments it returns ('test', bytes_read=4L, bytes_written=4L) correctly.

[achadwick]

🌈🌠 ... and that is why we wrap certain inconsistently annotated GLib calls... :SIGH:

At least upstream are fixing it for G-I uses. Sadly they're fixing it in a way that breaks stuff... *args, **kwargs anyone? Still, other languages won't be able to do that.

This is where the call's coming from: https://github.com/mypaint/mypaint/blob/v1.2.0/lib/glib.py#L57. I'm guessing we should try

    # See https://github.com/mypaint/mypaint/issues/634
    try:
        ustring = GLib.filename_to_utf8(opsysstring, -1)
    except TypeError:
        ustring = GLib.filename_to_utf8(opsysstring, -1, 0, 0)

in its place.

I'm using python-gobject 3.18.2-2, python-gi 3.18.2-2+b1, gir1.2-glib-2.0 1.46.0-4, and libglib2.0-0 2.48.0-1 on Debian, and the 4 argument version works for me whereas the 2-arg version spits out the TypeError. The 2-arg one is better to my eyes (NULL optional means unspecified to me).

@jtojnar please can you try:

>>> from gi.repository import GLib
>>> GLib.filename_to_utf8(opsysstring="test2", len=-1, bytes_read=0, bytes_written=0)
'test2'

to see if your typelib binding still accepts the junk args as keywords? Because that might be a nicer way of writing it.

If not we can catch the exception as above & do it that way.

[jtojnar]

Nope.

TypeError: GLib.filename_to_utf8() got an unexpected keyword argument 'bytes_read'

[achadwick]

If I remove the last two arguments it returns ('test', bytes_read=4L, bytes_written=4L) correctly.

Noted! The Python return type will be different if the params are annotated as [out] now.

[achadwick]

@jtojnar our versions look very similar, but I'm happy to add a try/except fix to master for now even if it's puzzling. If you have the time, can you comment with details of the plumbing of of the G-I mess on your system?

1. Which package+version on your system provides GLib-2.0.typelib? For me, this file is in gir1.2-glib-2.0 1.46.0-4.
2. Which package and version provides gi/overrides/GLib.py for Python 2 on your system? For me, that's python-gi 3.18.2-2+b1.
3. Whether that gi.overrides.GLib adds a compat layer for [g_] filename_to_utf8()?

It could just be something that Arch/Manjaro do but Debian don't...

[jtojnar]

1. /usr/lib/girepository-1.0/GLib-2.0.typelib is owned by gobject-introspection-git 1:1.48.0.3532.42bf17a-1
2. /usr/lib/python2.7/site-packages/gi/overrides/GLib.py is owned by python2-gobject 3.20.0-1
3. How can I determine that?

[achadwick]

@jtojnar Thanks for the info. Looks like it's caused by 1.46→1.48 in the package owning GLib-2.0.typelib, or at least it has the shape of it.

You can see if there's an override by opening the /usr/lib/python2.7/site-packages/gi/overrides/GLib.py file on your system in a text editor and searching. If it's adding some special override, it will hopefully be obvious.

[achadwick]

@jtojnar
Can you test master again with commit a0ff399, both starting MyPaint and then with:

$ cd path/to/mypaint/repo
$ nosetests --with-doctest lib

That should light up with all sorts of errors if there's anything else we need to do for 3.20.

[achadwick]

👣 🔍 I detectived!

GLib fixed up their annotations in 25a7c81 (mirror: GNOME/glib@25a7c81), included in 2.48. The generated typelib files are updating in Debian too, but they're in experimental only right now: http://packages.debian.org/gir1.2-glib-2.0.

[jtojnar]

There is only filename_from_utf8 override:

# backwards compatible API with default argument, and ignoring bytes_read
# output argument
def filename_from_utf8(utf8string, len=-1):
    return GLib.filename_from_utf8(utf8string, len)[0]

I tried downgrading the gobject-introspection-git to 1.46.0 and can confirm it is working. It just does not work with version ≥ 1.47 so your assumption about 1.46→1.48 seems to be correct.

[achadwick]

Thanks - hopefully the new MyPaint master will work on 1.48 systems. Let me know how it goes 😄

[jtojnar]

Yes it works, thank you.

[achadwick]

Thanks. Applied for 1.2.1 ☺

[achadwick]

Downstream instance: https://bugs.debian.org/822784. Better get 1.2.1 proper out of the door soon :(

[joseflaviojr]

To not downgrade Glib, I removed in /opt/local/share/mypaint/lib/glib.py:

ustring = GLib.filename_to_utf8(opsysstring, -1, 0, 0)
if ustring is None:
    raise UnicodeDecodeError(
        "GLib failed to convert %r to a UTF-8 string. "
        "Consider setting G_FILENAME_ENCODING if your file system's "
        "filename encoding scheme is not UTF-8."
        % (opsysstring,)
    )

And just return:

return opsysstring

But the filenames must be compatible with the UTF-8.

I advise:

import unicodedata
def convert_to_utf( string ):
    if type(string) is bytes: string = string.decode(sys.getfilesystemencoding())
    return unicodedata.normalize('NFC', string)