Failing to parse a path requirement without an extra space

[alexifm]

I have a pretty simple example that shows what's going wrong. The problematic line is being produced by poetry and the difference between file:///path/to/package/; and file:///path/to/package/ ; seems like it shouldn't matter but the former is not parseable.

In [1]: from packaging.requirements import Requirement

In [2]: s = 'test @ file:///Users/alex/test/; python_version >= "3.8" and python_version < "3.9"'

In [3]: Requirement(s)
---------------------------------------------------------------------------
ParseException                            Traceback (most recent call last)
~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/packaging/requirements.py in __init__(self, requirement_string)
    101         try:
--> 102             req = REQUIREMENT.parseString(requirement_string)
    103         except ParseException as e:

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/pyparsing.py in parseString(self, instring, parseAll)
   1954                     exc.__traceback__ = self._trim_traceback(exc.__traceback__)
-> 1955                 raise exc
   1956         else:

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/pyparsing.py in parseImpl(self, instring, loc, doActions)
   3813         if loc < len(instring):
-> 3814             raise ParseException(instring, loc, self.errmsg, self)
   3815         elif loc == len(instring):

ParseException: Expected stringEnd, found 'p'  (at char 33), (line:1, col:34)

During handling of the above exception, another exception occurred:

InvalidRequirement                        Traceback (most recent call last)
<ipython-input-3-fcc89c1c6706> in <module>
----> 1 Requirement(s)

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/packaging/requirements.py in __init__(self, requirement_string)
    102             req = REQUIREMENT.parseString(requirement_string)
    103         except ParseException as e:
--> 104             raise InvalidRequirement(
    105                 f'Parse error at "{ requirement_string[e.loc : e.loc + 8]!r}": {e.msg}'
    106             )

InvalidRequirement: Parse error at "'python_v'": Expected stringEnd

In [4]: s = 'test @ file:///Users/alex/test/ ; python_version >= "3.8" and python_version < "3.9"'

In [5]: Requirement(s)
Out[5]: <Requirement('test@ file:///Users/alex/test/ ; python_version >= "3.8" and python_version < "3.9"')>

I ran some more tests to try to identify the specific problem. Seems to be something to do with the python_version spec and the semicolon:

n [8]: Requirement('name @ https://github.com/pypa ;os_name=="a"')
Out[8]: <Requirement('name@ https://github.com/pypa ; os_name == "a"')>

In [9]: Requirement('name @ https://github.com/pypa;os_name=="a"')
Out[9]: <Requirement('name@ https://github.com/pypa;os_name=="a"')>

In [10]: Requirement('name @ file:///github.com/pypa;os_name=="a"')
Out[10]: <Requirement('name@ file:///github.com/pypa;os_name=="a"')>

In [11]: Requirement('name @ file:///github.com/pypa ;os_name=="a"')
Out[11]: <Requirement('name@ file:///github.com/pypa ; os_name == "a"')>

In [12]: Requirement('name @ file:///github.com/pypa; python_version >= "3.8" and python_version < "3.9"')
---------------------------------------------------------------------------
ParseException                            Traceback (most recent call last)
~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/packaging/requirements.py in __init__(self, requirement_string)
    101         try:
--> 102             req = REQUIREMENT.parseString(requirement_string)
    103         except ParseException as e:

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/pyparsing.py in parseString(self, instring, parseAll)
   1954                     exc.__traceback__ = self._trim_traceback(exc.__traceback__)
-> 1955                 raise exc
   1956         else:

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/pyparsing.py in parseImpl(self, instring, loc, doActions)
   3813         if loc < len(instring):
-> 3814             raise ParseException(instring, loc, self.errmsg, self)
   3815         elif loc == len(instring):

ParseException: Expected stringEnd, found 'p'  (at char 32), (line:1, col:33)

During handling of the above exception, another exception occurred:

InvalidRequirement                        Traceback (most recent call last)
<ipython-input-12-394adcb9387d> in <module>
----> 1 Requirement('name @ file:///github.com/pypa; python_version >= "3.8" and python_version < "3.9"')

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/packaging/requirements.py in __init__(self, requirement_string)
    102             req = REQUIREMENT.parseString(requirement_string)
    103         except ParseException as e:
--> 104             raise InvalidRequirement(
    105                 f'Parse error at "{ requirement_string[e.loc : e.loc + 8]!r}": {e.msg}'
    106             )

InvalidRequirement: Parse error at "'python_v'": Expected stringEnd

In [13]: Requirement('name @ file:///github.com/pypa ; python_version >= "3.8" and python_version < "3.9"')
Out[13]: <Requirement('name@ file:///github.com/pypa ; python_version >= "3.8" and python_version < "3.9"')>

In [14]: Requirement('name @ https://github.com/pypa;python_version >= "3.8" and python_version < "3.9"')
---------------------------------------------------------------------------
ParseException                            Traceback (most recent call last)
~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/packaging/requirements.py in __init__(self, requirement_string)
    101         try:
--> 102             req = REQUIREMENT.parseString(requirement_string)
    103         except ParseException as e:

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/pyparsing.py in parseString(self, instring, parseAll)
   1954                     exc.__traceback__ = self._trim_traceback(exc.__traceback__)
-> 1955                 raise exc
   1956         else:

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/pyparsing.py in parseImpl(self, instring, loc, doActions)
   3813         if loc < len(instring):
-> 3814             raise ParseException(instring, loc, self.errmsg, self)
   3815         elif loc == len(instring):

ParseException: Expected stringEnd, found '>'  (at char 46), (line:1, col:47)

During handling of the above exception, another exception occurred:

InvalidRequirement                        Traceback (most recent call last)
<ipython-input-14-ed256381ba74> in <module>
----> 1 Requirement('name @ https://github.com/pypa;python_version >= "3.8" and python_version < "3.9"')

~/.pyenv/versions/3.8.5/envs/modeling/lib/python3.8/site-packages/packaging/requirements.py in __init__(self, requirement_string)
    102             req = REQUIREMENT.parseString(requirement_string)
    103         except ParseException as e:
--> 104             raise InvalidRequirement(
    105                 f'Parse error at "{ requirement_string[e.loc : e.loc + 8]!r}": {e.msg}'
    106             )

InvalidRequirement: Parse error at "'>= "3.8"'": Expected stringEnd

In [15]: Requirement('name @ https://github.com/pypa ;python_version >= "3.8" and python_version < "3.9"')
Out[15]: <Requirement('name@ https://github.com/pypa ; python_version >= "3.8" and python_version < "3.9"')>

[uranusjr]

This is by design. ; is a valid URL character, so a space is needed to separate the two parts; otherwise there's no way to tell whether the semicolon is a part of the URL or not.

[pradyunsg]

And, following what @uranusjr said above:

>>> r1 = Requirement('name @ file:///github.com/pypa;os_name=="a"')
>>> r2 = Requirement('name @ file:///github.com/pypa ;os_name=="a"')
>>> r1 == r2
False
>>> r1.url
'file:///github.com/pypa;os_name=="a"'
>>> r2.url
'file:///github.com/pypa'

[alexifm]

@uranusjr thanks for letting me know. much appreciated. Will send this over to poetry so they can fix their requirements export