compile_glib_schema_files UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined> #7833

coolcatco888 · 2023-08-09T06:00:58Z

I am encountering an error when running pyinstaller:
pyinstaller --noconsole --add-data="resource/ui/maingui.glade;resource/ui" --splash "resource/images/DisplayInSplash.png" --icon="resource/images/DisplayInIcon.ico" --onedir displayin.py

Error
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined>

I am not sure why this is happening. Some notable things about the program I am trying to compile:

I am using the msys2 mingw64 environment
The program is written with Python GTK 3.0
The program uses MSVC gstreamer binaries
I compiled opencv4.7.0 to use those gstreamer binaries

I should note that those factors don't seem to be causing the error as it is failing at compile_glib_schema_files, which seems to be related to GTK. Please help!

Full Error Log

user@localhost MINGW64 /c/Users/user/git/DisplayIn/src/displayin
$ ./build_win.sh 
1112 INFO: PyInstaller: 5.13.0
1112 INFO: Python: 3.11.4
1112 INFO: Platform: Windows-10-10.0.22621-SP0
1112 INFO: wrote C:/Users/user/git/DisplayIn/src/displayin/displayin.spec
1127 INFO: Extending PYTHONPATH with paths
['C:/Users/user/git/DisplayIn/src/displayin']
2324 INFO: Appending 'datas' from .spec
2325 INFO: checking Analysis
2325 INFO: Building Analysis because Analysis-00.toc is non existent
2325 INFO: Initializing module dependency graph...
2331 INFO: Caching module graph hooks...
2371 INFO: Analyzing base_library.zip ...
3378 INFO: Loading module hook 'hook-encodings.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
3770 INFO: Loading module hook 'hook-heapq.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
4704 INFO: Loading module hook 'hook-pickle.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
6287 INFO: Caching module dependency graph...
6366 INFO: running Analysis Analysis-00.toc
6383 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by C:/tools/msys64/mingw64/bin/python.exe
6403 INFO: Analyzing C:/Users/user/git/DisplayIn/src/displayin/displayin.py
6456 INFO: Loading module hook 'hook-cv2.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
7061 INFO: Loading module hook 'hook-numpy.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/numpy/_pyinstaller'...
7635 INFO: Loading module hook 'hook-difflib.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
7870 INFO: Loading module hook 'hook-multiprocessing.util.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
7970 INFO: Loading module hook 'hook-xml.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
8623 INFO: Loading module hook 'hook-platform.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
8647 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
9753 INFO: Processing pre-safe import module hook gi from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.py'.
9776 INFO: Loading module hook 'hook-gi.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
9870 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gtk.py'.
9872 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
9872 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
9873 INFO: Processing pre-find module path hook pyi_splash from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_find_module_path/hook-pyi_splash.py'.
9874 INFO: Adding pyi_splash module to application dependencies.
9923 INFO: Loading module hook 'hook-sounddevice.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
10866 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
10971 INFO: Processing pre-safe import module hook distutils from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-distutils.py'.
10974 INFO: Processing pre-find module path hook distutils from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
10983 INFO: Loading module hook 'hook-distutils.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
11088 INFO: Loading module hook 'hook-distutils.util.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
11336 INFO: Loading module hook 'hook-OpenGL.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
15132 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
15151 INFO: Processing module hooks...
15329 INFO: Loading module hook 'hook-setuptools.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
17153 INFO: Loading module hook 'hook-pkg_resources.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
17734 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
18287 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
18289 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
18290 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
18291 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/io-wmf.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/io-wmf.dll': The specified procedure could not be found.
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-heif.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-heif.dll': The specified procedure could not be found.
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.dll': The specified procedure could not be found.
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-tiff.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-tiff.dll': The specified procedure could not be found.
19661 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
19670 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
20054 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
20066 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
20580 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
20951 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
21363 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
22565 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
22567 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
22959 INFO: Processing pre-safe import module hook gi.repository.HarfBuzz from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.HarfBuzz.py'.
23304 INFO: Loading module hook 'hook-setuptools.msvc.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24127 INFO: Loading module hook 'hook-_tkinter.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24128 INFO: checking Tree
24128 INFO: Building Tree because Tree-00.toc is non existent
24129 INFO: Building Tree Tree-00.toc
24192 INFO: checking Tree
24192 INFO: Building Tree because Tree-01.toc is non existent
24192 INFO: Building Tree Tree-01.toc
24200 INFO: checking Tree
24200 INFO: Building Tree because Tree-02.toc is non existent
24201 INFO: Building Tree Tree-02.toc
24203 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24586 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24971 INFO: Loading module hook 'hook-gi.repository.HarfBuzz.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
25372 INFO: Processing pre-safe import module hook gi.repository.freetype2 from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.freetype2.py'.
25378 INFO: Loading module hook 'hook-gi.repository.freetype2.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
25810 INFO: Looking for ctypes DLLs
26107 INFO: Analyzing run-time hooks ...
26118 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
26124 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
26129 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
26134 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
26139 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
26143 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
26147 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
26150 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
26157 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_setuptools.py'
26161 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgres.py'
26169 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth__tkinter.py'
26336 INFO: Compiling collected GLib schema files in 'C:/Users/user/git/DisplayIn/src/displayin/build/displayin/_pyi_gschema_compilation'...
Traceback (most recent call last):
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/utils/hooks/gi.py", line 391, in compile_glib_schema_files
    p = subprocess.run(
        ^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/subprocess.py", line 1196, in communicate
    stdout = self.stdout.read()
             ^^^^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/encodings/cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:/tools/msys64/mingw64/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/__main__.py", line 194, in _console_script_run
    run()
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/__main__.py", line 180, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/__main__.py", line 61, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1019, in main
    build(specfile, distpath, workpath, clean_build)
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 944, in build
    exec(code, spec_namespace)
  File "C:/Users/user/git/DisplayIn/src/displayin/displayin.spec", line 7, in <module>
    a = Analysis(
        ^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 429, in __init__
    self.__postinit__()
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 664, in assemble
    self.datas = compile_glib_schema_files(self.datas, os.path.join(CONF['workpath'], "_pyi_gschema_compilation"))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/utils/hooks/gi.py", line 402, in compile_glib_schema_files
    logger.warning("Extra output from glib-compile-schemas:\n%s", p.stdout)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'p' where it is not associated with a value

The text was updated successfully, but these errors were encountered:

rokm · 2023-08-09T08:02:10Z

Does it help if you add errors='ignore' to the list of arguments to the subprocess.run in your installed PyInstaller copy:

pyinstaller/PyInstaller/utils/hooks/gi.py

Lines 391 to 398 in 58b8235

    
           p = subprocess.run( 
        
               cmd_args, 
        
               stdin=subprocess.DEVNULL, 
        
               stdout=subprocess.PIPE, 
        
               stderr=subprocess.STDOUT, 
        
               check=True, 
        
               text=True, 
        
           )

The root cause of this seem to be warning messages that use U+201C and U+201D quotation marks:

Warning: Schema “org.freedesktop.ibus” has path “/desktop/ibus/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.

coolcatco888 · 2023-08-09T18:28:19Z

Does it help if you add errors='ignore' to the list of arguments to the subprocess.run in your installed PyInstaller copy:

pyinstaller/PyInstaller/utils/hooks/gi.py

Lines 391 to 398 in 58b8235

p = subprocess.run(

cmd_args,

stdin=subprocess.DEVNULL,

stdout=subprocess.PIPE,

stderr=subprocess.STDOUT,

check=True,

text=True,

)

The root cause of this seem to be warning messages that use U+201C and U+201D quotation marks:
Warning: Schema “org.freedesktop.ibus” has path “/desktop/ibus/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.

Thanks for the quick response! That fixed it! Thank you!

coolcatco888 added the triage Please triage and relabel this issue label Aug 9, 2023

rokm added bug and removed triage Please triage and relabel this issue labels Aug 9, 2023

rokm mentioned this issue Aug 9, 2023

hookutils: gi: fix error handling in compile_glib_schema_files #7834

Merged

rokm closed this as completed in #7834 Aug 9, 2023

coolcatco888 mentioned this issue Aug 9, 2023

Gtk libpixbufloader-png.dll load path is not relative - exe will crash if run on another machine #7838

Closed

6 tasks

github-actions bot locked as resolved and limited conversation to collaborators Oct 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

compile_glib_schema_files UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined> #7833

compile_glib_schema_files UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined> #7833

coolcatco888 commented Aug 9, 2023

rokm commented Aug 9, 2023

coolcatco888 commented Aug 9, 2023

compile_glib_schema_files UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined> #7833

compile_glib_schema_files UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined> #7833

Comments

coolcatco888 commented Aug 9, 2023

rokm commented Aug 9, 2023

coolcatco888 commented Aug 9, 2023