ord('œ̃')... TypeError when processing ligatures

[timeis-byte]

I have front.stack, which is a dictionary with some ligatures, french pronunciations
I also have front.bulk, which is a pyglet layout

first I take some texts from front.stack and create a formatted pyglet document like:

bulk = pgl.text.decode_html(
        
        text = str(charger_dict['bulk'])
        
        )

second I change the current document of existing pyglet layout:
front.bulk.document = bulk

when done on text without ligatures, that works.

When there are ligatures, it raises TypeError, somewhere deep down the line there happens ord(' some ligature here' )
I understand why it happens technically - ligature is viewed by python as a string of length 2
but what should I do with that? remove all the ligatures from the original text somehow? or modify pyglet module code somehow?

I am not very good at that, so if you have any advice, I would like to hear 'em
thanks in advance ^-^

i got this, by the way:
Exception ignored on calling ctypes callback function: <function Win32Window.get_window_proc..f at 0x0776D610>
Traceback (most recent call last):
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\win32_init.py", line 720, in f
result = event_handler(msg, wParam, lParam)
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\win32_init_.py", line 994, in _event_lbuttondown
return self.event_mousebutton(
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\win32_init.py", line 988, in _event_mousebutton
self.dispatch_event(ev, x, y, button, self.get_modifiers())
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window_init.py", line 1333, in dispatch_event
if EventDispatcher.dispatch_event(self, *args) != False:
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\event.py", line 411, in dispatch_event
self._raise_dispatch_exception(event_type, args, handler, exception)
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\event.py", line 476, in _raise_dispatch_exception
raise exception
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\event.py", line 408, in dispatch_event
if handler(*args):
File "B:\PyPro\doors\engine\moves.py", line 496, in on_mouse_press_if_charger
load_next_from_stack()
File "B:\PyPro\doors\engine\moves.py", line 474, in load_next_from_stack
place_the_text(front.stack[ int(front.stack['now']) ] )
File "B:\PyPro\doors\engine\moves.py", line 751, in place_the_text
front.bulk.document = bulk
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 920, in _set_document
self._init_document()
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 1034, in _init_document
self._update()
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 957, in _update
lines = self._get_lines()
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 933, in _get_lines
glyphs = self._get_glyphs()
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 1076, in _get_glyphs
glyphs.extend(font.get_glyphs(text[start:end]))
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\base.py", line 386, in get_glyphs
self.glyphs[c] = glyph_renderer.render(c)
File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\win32.py", line 427, in render
ord(text), ord(text), byref(abc)):
TypeError: ord() expected a character, but string of length 2 found

[benmoran56]

Thanks for opening the ticket.

The current font renderer is GDI based. It may be able to be updated to support this, but I'm not sure yet.
We also have have some new experimental DirectWrite based font rendering that was just merged. (See this pull request: #359)
I haven't checked this yet, but it's worth a try as well.

If possible, can you share some example text? Or, a simple example that shows the crash? That would be very helpful to troubleshoot.

[benmoran56]

I just realized that you did share some example text, in the issue title:

ord('œ̃')

I can confirm your error.
We actually just merged some experimental support for DirectWrite based font rendering. I have tested it, and it does seem to work with this text. If you want to try it, it's in the pyglet-1.5-maintenance branch. To enable DireceWrite rendering, add the following pyglet option at the start of your project:

import pyglet

pyglet.options["advanced_font_features"] = True

[timeis-byte]

thank you, benmoran... or is it just Ben?
I will try to merge it a bit later and write back here on the result

[timeis-byte]

I installed from github pyglet-1.5-maintenance;
the version that was installed is pyglet-1.5.16, the previous (the one I had before) was 1.5.14

when I run the code, I get this (and this time it's just some text, I haven't gotten to the ligatures yet)

Exception ignored on calling ctypes callback function: <function Win32Window._get_window_proc.<locals>.f at 0x07689658>
Traceback (most recent call last):
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\win32\__init__.py", line 720, in f
    result = event_handler(msg, wParam, lParam)
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\win32\__init__.py", line 995, in _event_lbuttondown
    return self._event_mousebutton(
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\win32\__init__.py", line 989, in _event_mousebutton
    self.dispatch_event(ev, x, y, button, self._get_modifiers())
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\window\__init__.py", line 1333, in dispatch_event
    if EventDispatcher.dispatch_event(self, *args) != False:
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\event.py", line 411, in dispatch_event
    self._raise_dispatch_exception(event_type, args, handler, exception)
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\event.py", line 476, in _raise_dispatch_exception
    raise exception
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\event.py", line 408, in dispatch_event
    if handler(*args):
  File "B:\PyPro\doors\main.py", line 183, in on_mouse_press
    activate_the_menu(x, y)
  File "B:\PyPro\doors\engine\moves.py", line 570, in activate_the_menu
    load_the_charger()
  File "B:\PyPro\doors\engine\moves.py", line 516, in load_the_charger
    load_next_from_stack()
  File "B:\PyPro\doors\engine\moves.py", line 486, in load_next_from_stack
    place_the_text(front.stack[ int(front.stack['now']) ] )
  File "B:\PyPro\doors\engine\moves.py", line 812, in place_the_text
    bulk_list[number] = pgl.text.layout.TextLayout(
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 811, in __init__
    self.document = document
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 920, in _set_document
    self._init_document()
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 1034, in _init_document
    self._update()
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 957, in _update
    lines = self._get_lines()
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 933, in _get_lines
    glyphs = self._get_glyphs()
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 1072, in _get_glyphs
    for start, end, (font, element) in runs.ranges(0, len(text)):
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\runlist.py", line 410, in ranges
    starts, ends, values = zip(*[next(i) for i in iterators])
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\runlist.py", line 410, in <listcomp>
    starts, ends, values = zip(*[next(i) for i in iterators])
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\document.py", line 723, in ranges
    ft = font.load(font_name, font_size,
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\__init__.py", line 144, in load
    font = _font_class(name, size, bold=bold, italic=italic, stretch=stretch, dpi=dpi)
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\win32.py", line 487, in __init__
    super(GDIPlusFont, self).__init__(name, size, bold, italic, stretch, dpi)
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\win32.py", line 228, in __init__
    self.logfont = self.get_logfont(name, size, bold, italic, dpi)
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\win32.py", line 259, in get_logfont
    logfont.lfItalic = italic
TypeError: an integer is required (got type NoneType)

after I close the window, I get this:

Exception ignored in: <function GDIPlusFont.__del__ at 0x0AACB580>
Traceback (most recent call last):
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\win32.py", line 527, in __del__
    super(GDIPlusFont, self).__del__()
  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\win32.py", line 242, in __del__
    gdi32.DeleteObject(self.hfont)
AttributeError: 'GDIPlusFont' object has no attribute 'hfont'

[timeis-byte]

moreover, when I add the option pgl.options["advanced_font_features"] = True
in the beginning of my program, I get this:

_Traceback (most recent call last):

  File "B:\PyPro\doors\main.py", line 5, in <module>
    from engine.config import (

  File "B:\PyPro\doors\engine\config.py", line 42, in <module>
    step_counter = pgl.text.Label(text = 'Step:0', x = 5, y = 5,

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\__init__.py", line 449, in __init__
    super(Label, self).__init__(document, x, y, width, height,

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\__init__.py", line 268, in __init__
    super(DocumentLabel, self).__init__(document,

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 811, in __init__
    self.document = document

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 920, in _set_document
    self._init_document()

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 1034, in _init_document
    self._update()

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 957, in _update
    lines = self._get_lines()

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 933, in _get_lines
    glyphs = self._get_glyphs()

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\layout.py", line 1069, in _get_glyphs
    self._document.get_font_runs(dpi=self._dpi),

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\document.py", line 608, in get_font_runs
    ft = self.get_font(dpi=dpi)

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\text\document.py", line 618, in get_font
    return font.load(font_name, font_size,

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\__init__.py", line 144, in load
    font = _font_class(name, size, bold=bold, italic=italic, stretch=stretch, dpi=dpi)

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\directwrite.py", line 1906, in __init__
    self._font_index, self._collection = self.get_collection(name)

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\directwrite.py", line 2177, in get_collection
    cls._initialize_direct_write()

  File "b:\miniconda3\envs\doors\lib\site-packages\pyglet\font\directwrite.py", line 2090, in _initialize_direct_write
    DWriteCreateFactory(DWRITE_FACTORY_TYPE_SHARED, IID_IDWriteFactory5, byref(cls._write_factory))

  File "_ctypes/callproc.c", line 1010, in GetResult

OSError: [WinError -2147467262] Интерфейс не поддерживается_

it's worth noting, that this specific label was succesfully loaded before, without the option active...

[caffeinepills]

Post your system specs (including windows build), python specs, the font you are using (if other than default), the label arguments including the full text of what you are trying to display. There is too little information to go off of right now.

[timeis-byte]

yeah, of course

Windows 8.1 Professional
Processor: Intel(R) Celeron(R) CPU N2840 @ 2.16 GHz
RAM: 4.00 Gb (1,89 Gb accessible)
System type: 32 bit, processor x64 bit

Python 3.9.1 (in a Conda environment)

I was using the default font, also I tried Times New Roman, nothing changed

there are 2 different dropouts: the first without pgl.options["advanced_font_features"] = True and the second with

1.
in the 1st dropout I have my label

step_counter = pgl.text.Label(text = 'Step:0', x = 5, y = 5, 
                              batch = batch, group = foreground)

successfully loaded and later on the text of this label is being changed, it works

then I try to load this

    word = pgl.text.decode_html(
        
        text = '<h3>de</h3>'
        
        )
    
    word_layout = pgl.text.layout.TextLayout(
        
            document = word, 
            multiline = True, wrap_lines = True, 
            width = 320, height = 200,
            batch = batch, group = foreground
            
            )

it crashes on word_layout...

2.
the second dropout happens, as I mentioned when I enable the option advanced_font_features
I did not change any more lines, and it crashes on the 1st label, the one I brought above
here it is again:

step_counter = pgl.text.Label(text = 'Step:0', x = 5, y = 5, 
                              batch = batch, group = foreground)

[timeis-byte]

and, by the way, I did not install all the windows' recommended system updates.... since 2013 I think... or 2016... the update is switched off on my PC

maybe it has something to do with the problem?

[caffeinepills]

Ok that narrowed it down. I found the issues, I put in a pull request to fix them, we will have to wait until Benjamin takes a look at it and merges it in. Then you will have to update again to test it out, thank you.

[timeis-byte]

thank you, caffeinePills
)

[caffeinepills]

#363 Was merged into 1.5 branch. Try it out and see if you are able to run it.

[timeis-byte]

Well, it kinda worked...))

I updated.
If I don't use pyglet.options["advanced_font_features"] = True, the script still stumbles over the ligatures, just like it was when I used pyglet 1.14

If I enable the option, then the script shows me the ligatures, but... it seems it outright ignores font names and plays with the text format to its heartful delight, he
that is what I see when the option is NOT enabled:

this is the same page, with the option being enabled:

it doesn't look that bad, at first sight, just some jumping letters, overall bolder, some are suddenly very bold, broken distances between the letters as well

it shows the ligatures changed a bit, spreads them out in several letters in some cases... below, for example, œ̃ turned to oe (glued together, without ~ above)

I don't know...
maybe I should swap all ligatures in the original text with something more or less suitable, with two or more letters... ɑ̃ was swapped by a with the script, I can do it as well, if I'm gonna drop the idea of preserving the original view....

also it feels a bit heavier now, my computer takes a tiny pause before loading the next word (when the option is enabled, I mean)... maybe I can somehow send a code to the program... I mean a unicode sign of the ligature... the thing is python prints all the ligatures happily, why should there be a problem in the first place... those are just rhetoric questions, sorry, people

Thank you for your help, open to any advice ^-^

[caffeinepills]

Could you provide the HTML string that is causing this issue?

[timeis-byte]

below are the dictionaries I use, each keyword is brought into a separate pyglet document and separate pyglet layout

 'meanings': '< p>< b>from, of, with, by, to...< /b>< /p>',
 'word': '< h3>de< /h3>',
 'book': '< p>(Harry Potter et la Chambre des Secrets)< /p>',
 'ex_list': ["< p>'mourir de faim', 'to die of hunger'< /p>",
  "< p>'du pain et de la confiture', 'bread and jam'< /p>",
  "< p>'rouge de colère', 'red with anger'< /p>"],
 'info': '[də, d] [de, d’]',
 'quote': "< p>< i>...de la magie ! Dobby se souvient comment c'était quand Celui-Dont-On-Ne-Doit-Pas-Prononcer-Le-Nom était au sommet de sa puissance ! Nous, les elfes ...< /i>< /p>" 

 'meanings': < p>< b>the, him, her, it, them< /b>< /p>
 'word': '< h3>le< /h3>',
 'book': "< p>(Harry Potter et le Prisonnier d'Azkaban)< /p>",
 'ex_list': ["< p>'se casser la jambe', 'to break one’s leg'< /p>",
  "< p>'le soir', 'in the evening'< /p>",
  "< p>'le quart de', 'a quarter of'< /p>"],
 'info': '[lə, la] [le, la,  l’,  les]',
 'quote': '< p>< i>...que tu racontes ? Hermione se précipita vers la table et retourna le pot au lait. Criant et se ...< /i>< /p>'

I'm not sure it's about html though... the html label with simple text is also being shown bolder, its letters closer to each other. But I did not test it with unformatted documents, well (or attributed, if it matters... I did not work with attributed at all, did not grasp the conception, he)
should I?

p.s. in each of the html tag I added a space, otherwise the redactor ran them no matter what.... how can I tell it to ignore the tags?

[caffeinepills]

Could you try the new 1.5 maintenance branch and see if that makes a difference? I made some more changes and some fixes. I apologize as I have been busy these past few weeks.

[caffeinepills]

@timeis-byte Newest release should have all of the latest fixes.

[benmoran56]

I'm going to close this now due to age and lack of reply, but please let us know if you still have issues after updating.

[timeis-byte]

hi everybody!)

as the one who raised the issue, I hereby confirm: now everything works just fine ^-^
sorry for getting lost for so long... and I really appreciate your help (benmorran, caffeinpills & co if any)
thank you one more time!

AK