Url parsing mangles urls with parenthesis in regex

[janne-tervo]

Describe the bug
Spectacular schema generation mangles urls with parenthesis in parameter patterns

To Reproduce
Add to url patterns:
url(r'^jsi18n/(?P[a-z]{2}(-[a-z]{2})?)/$', JavaScriptCatalog.as_view()),
Attempt to generate schema fails.

Expected behavior
Url regex patterns with parenthesis are parsed correctly.

[tfranzel]

hi @janne-tervo!

url(r'^jsi18n/(?P[a-z]{2}(-[a-z]{2})?)/$', JavaScriptCatalog.as_view()),

we do not account for that i suppose. pretty sure that this is an illegal regex pattern written like that

>>> import re
>>> re.compile(r'^jsi18n/(?P[a-z]{2}(-[a-z]{2})?)/$')
re.error: unknown extension ?P[ at position 9
>>> re.compile('^jsi18n/(?P<some_name>[a-z]{2}(-[a-z]{2})?)/$')
>>> # no error

i would guess that making requests there would also fail, maybe even starting the server.

[janne-tervo]

Markdown bit me here.

This gets mangled by spectacular url parsing:
url(r'^jsi18n/(?P<language>[a-z]{2}(-[a-z]{2})?)/$',

This does not get mangled:
url(r'^jsi18n/(?P<language>[^/]+)/$',

[janne-tervo]

This is caused by:
plumbing.py:767 regex=re.sub(r'(?P<(\w+)>.+?)', r'(?P<\1>[^/]+)', pattern._regex),
which stops at first parentheses in variable regex.

[tfranzel]

ahh now i get it. makes sense.

[tfranzel]

turns out this was fixed once before (#168) but it was incomplete. i chose to write a parser for it and replace the old RE.

i believe it would have been possible to construct a extended RE, but it would have been (imho) way more complicated with low readability. i hope this solution now is at least somewhat readable and more importantly correct.

can you check if this works for you now?

[janne-tervo]

This commit fixes my issue.
Alternative fix with extra dependency could be created using e.g. recursive extension from https://pypi.org/project/regex/. This problem can't be solved using python re package because it can't recognize this language.

[tfranzel]

awesome!

i love the regex package and use it myself. i briefly tried but was not able to get it right and as you said, adding deps is always problematic. if my solution turns out to be suboptimal, we can still switch to regex.

[tfranzel]

i'll close this issue for now. feel free to circle back if you notice any problems.