Error re.error: global flags not at the start of the expression at position ...

[Alexey-T]

CudaText plugin:
https://github.com/cudaText-addons/cuda_markdown_preview

is giving error under Python 3.11 and Win10:

Init: cuda_markdown_preview
Traceback (most recent call last):
  File "C:\APPLIS\cudatext\py\cuda_markdown_preview\__init__.py", line 12, in <module>
    md = markdown.Markdown(extensions=ext)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\APPLIS\cudatext\py\cuda_markdown_preview\markdown\core.py", line 96, in __init__
    self.registerExtensions(extensions=kwargs.get('extensions', []),
  File "C:\APPLIS\cudatext\py\cuda_markdown_preview\markdown\core.py", line 125, in registerExtensions
    ext.extendMarkdown(self)
  File "C:\APPLIS\cudatext\py\cuda_markdown_preview\mdx_partial_gfm\__init__.py", line 62, in extendMarkdown
    gfm.AutolinkExtension().extendMarkdown(md)
  File "C:\APPLIS\cudatext\py\cuda_markdown_preview\gfm\autolink.py", line 75, in extendMarkdown
    md.inlinePatterns.register(AutolinkPattern(URL_RE, md), "gfm-autolink", 100)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\APPLIS\cudatext\py\cuda_markdown_preview\markdown\inlinepatterns.py", line 209, in __init__
    self.compiled_re = re.compile(r"^(.*?)%s(.*)$" % pattern,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "re\__init__.py", line 227, in compile
  File "re\__init__.py", line 294, in _compile
  File "re\_compiler.py", line 743, in compile
  File "re\_parser.py", line 980, in parse
  File "re\_parser.py", line 455, in _parse_sub
  File "re\_parser.py", line 841, in _parse
re.error: global flags not at the start of the expression at position 6
ERROR: Exception in CudaText for cuda_markdown_preview.run: error: global flags not at the start of the expression at position 6

What can I do?

[waylan]

is giving error under Python 3.11

We do not yet officially support Python 3.11 and it appears that as of Python 3.11 global flags are only allowed to exist at the start of a regex (apparently this was deprecated in Python 3.6; see second to last item here). A few of our regex will need to be refactored to support Python 3.11.

[Alexey-T]

thanks, I will wait for a fix.

[waylan]

So I just looked at our source code and we are not using global flags inline in our regex. However, looking more closely at your traceback, I see that the error occurs when registering the gfm.autolink extension. A quick search demonstrates that that extension is using flags. I suggest reporting this issue to the appropriate extension.

In the meantime, I will leave this issue open until we verify that the base library runs fine on Python 3.11.

[Alexey-T]

I cannot find Github repo for py-GFM extension, where is it?

[waylan]

As a more general concern, the regex for each inline pattern is compiled using this code:

markdown/markdown/inlinepatterns.py

Lines 209 to 210 in a8ad6fc

	self.compiled_re = re.compile(r"^(.*?)%s(.*)$" % pattern,
	re.DOTALL | re.UNICODE)

That includes any inlinepatterns defined by extensions. If an extension wants/needs additional flags, as of Python 3.11 they cannot define them within the regex. Of course, extensions can always redefine this method in their own subclasses and then define any flags they want. While not as easy as defining them anywhere within the rexeg itself, it is still possible to do. Do we want to address this somehow, and if so, how?

[waylan]

I cannot find Github repo for py-GFM extension, where is it?

I have no idea. I was not aware of the extension until today.

[waylan]

The pypi page links to zopieux/py-gfm.

[Alexey-T]

But "This repository will not receive bug fixes, and might become read only soon...
Oct 25, 2020"

[waylan]

Yeah I saw that. However, we can not be responsible for third party extensions. That is the choice of the developers of the extension. Your choices include not updating to Python 3.11, finding an alternate extension, patching the extension yourself, or building your own replacement extension.

[Alexey-T]

@veksha Do you use Markdown Editing plugin for CudaText? if so, consider to patch the mentioned extension for py 3.11.

[facelessuser]

What kind of flags are being used, can you use scoped flags? (?aiLmsux-imsx:...)

[waylan]

@facelessuser the extension is using (?i:...) as can be seen here

[facelessuser]

Scoped flags should not throw a global flag warning. This is PY 3.11 below, no error.

Python 3.11.0 (v3.11.0:deaf509e8f, Oct 24 2022, 14:43:23) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> re.compile(r'^(?i:www)stuff')
re.compile('^(?i:www)stuff')

So, I don't know what is actually getting run.

But I'd advise using the InlineProcessor instead of Pattern as it does not wrap the pattern:

markdown/markdown/inlinepatterns.py

Line 253 in a8ad6fc

class InlineProcessor(Pattern):

[facelessuser]

Just an FYI on InlineProcessor, the return is a bit different as it was designed to report back the found start and end position allowing you to use a pattern, but also grab more content than the pattern matched. This allows for more advanced inline parsing which helped to alleviate some complex pattern handling we needed.

[waylan]

But I'd advise using the InlineProcessor instead of Pattern as it does not wrap the pattern

Ah right. So that is the answer for any third party extensions which need this: Use the new InlineProcessor class.

As the py311 tests are passing that I added in #1313, I am closing this. There is no action we need to take.