poetry-core generates invalid PEP 508 requirements when extras are used

[luca-medeiros]

• Poetry version: 1.2.2
• Python version: 3.8
• OS version and name: Ubuntu 18.04
• pyproject.toml: https://github.com/luca-medeiros/poetry_monorepo

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Installing the repository by poetry install works fine, but when running pip install . the following error message shows up:

Output message

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing /home/luca/reposNuvi/test_monorepo/lib
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/packaging/requirements.py", line 102, in __init__
    req = REQUIREMENT.parseString(requirement_string)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pyparsing/core.py", line 1141, in parse_string
    raise exc.with_traceback(None)
pip._vendor.pyparsing.exceptions.ParseException: Expected string_end, found 'extra'  (at char 11), (line:1, col:12)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3101, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/packaging/requirements.py", line 104, in __init__
    raise InvalidRequirement(
pip._vendor.packaging.requirements.InvalidRequirement: Parse error at "'extra =='": Expected string_end

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 369, in run
    requirement_set = resolver.resolve(
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 237, in get_dependencies
    return [r for r in candidate.iter_dependencies(with_requires) if r is not None]
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 237, in <listcomp>
    return [r for r in candidate.iter_dependencies(with_requires) if r is not None]
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 495, in iter_dependencies
    valid_extras = self.extras.intersection(self.base.dist.iter_provided_extras())
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_internal/metadata/pkg_resources.py", line 202, in iter_provided_extras
    return self._dist.extras
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2978, in extras
    return [dep for dep in self._dep_map if dep]
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3033, in _compute_dependencies
    reqs.extend(parse_requirements(req))
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3094, in parse_requirements
    yield Requirement(line)
  File "/home/luca/anaconda3/envs/split_na2/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3103, in __init__
    raise RequirementParseError(str(e))
pip._vendor.pkg_resources.RequirementParseError: Parse error at "'extra =='": Expected string_end

[dimbleby]

probably just a missing space before the semicolons, here

pull request welcome, though note that fixing this alone is insufficient, you'll still have to get past #5273

[radoering]

I don't think a whitespace is required according to PEP 508. Further, pip does not require whitespace in a requirements.txt.

[dimbleby]

I think you are mistaken and without whitespace the semicolon is mistaken for being part of the url

there's code a few lines up that tries to add a space for direct dependencies but (surely accidentally?) omits directory dependencies. Simplest is likely to include the space unconditionally, it should be benign always if only required sometimes

[radoering]

You're right. I missed that part. Whitespace is required in some cases (e.g. with urls). I agree we should simply add it in any case.

[luca-medeiros]

Strangely, I'm still getting the same error message about extras on the repo that I've linked.

[neersighted]

The change is not yet in a released version of Poetry. Merged does not mean it has shown up on your machine yet.

[luca-medeiros]

I supposed installing from the source on the main branch would do the trick.

[radoering]

Only if the fix is in poetry itself. This fix is in poetry-core. When installing poetry from source you still get a released version of poetry-core. Replacing poetry-core with the version from its main branch should do the trick.

[neersighted]

That won't work either as this is poetry-core as the build-system. You'll need to update your build-system requirement to use a Git reference.

[luca-medeiros]

Thanks for the build-system trick! Seems to be working now.

[neersighted]

Just note that by using a non-SHA git reference there is inherent instability; please do not release versions with such a build dependency (see #6741 for an example of why this is bad).

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.