[BUG] Prospector fails if pyproject.toml contains git dependencies with rev instead of version

[bellmatt]

Describe the bug A clear and concise description of what the bug is.

If prospector is run in a Poetry project with git dependencies where rev is used to specify the dependency version instead of version, it fails to run.

Note: the root cause may be a bug in requirements-detector (dependency of prospector) that has just been uncovered by a change in behaviour in prospector in 1.8.0+.
I noticed this in the release notes which might be relevant, but there's no PR linked, so it's hard to confirm for sure:

Autodetction of libraries, to automatically use pylint plugins, will no [now?] work on projects using a pyproject.toml ; also it has been turned on by default, it seemed to have accidentally been set to off by default some time ago.

To Reproduce Steps to reproduce the behavior:

• Add this to your pyproject.toml, doesn't need to be a real package and you don't need to update/install anything in your environment (but same behaviour is visible if it is a real installed package):
  mytestdependency = {git = "ssh://git@github.com/MyOrg/MyRepo.git",rev = "1.2.3"}
• Run prospector and see the error message in the screenshot section below.
• Remove all git dependencies with "rev" from pyproject.toml
• Prospector works as expected

Note: the same is probably the case for tag which is also valid for git dependencies: https://python-poetry.org/docs/dependency-specification/

Expected behavior A clear and concise description of what you expected to happen.

• Prospector runs successfully

Screenshots If applicable, add screenshots to help explain your problem.

Traceback:

Traceback (most recent call last):
  File "/repo/.venv/bin/prospector", line 8, in <module>
    sys.exit(main())
  File "/repo/.venv/lib/python3.8/site-packages/prospector/run.py", line 189, in main
    config = ProspectorConfig()
  File "/repo/.venv/lib/python3.8/site-packages/prospector/config/__init__.py", line 32, in __init__
    self.libraries = self._find_used_libraries(self.config, self.profile)
  File "/repo/.venv/lib/python3.8/site-packages/prospector/config/__init__.py", line 201, in _find_used_libraries
    for found_dep in autodetect_libraries(self.workdir):
  File "/repo/.venv/lib/python3.8/site-packages/prospector/autodetect.py", line 90, in autodetect_libraries
    libraries = find_from_requirements(path)
  File "/repo/.venv/lib/python3.8/site-packages/prospector/autodetect.py", line 72, in find_from_requirements
    reqs = find_requirements(path)
  File "/repo/.venv/lib/python3.8/site-packages/requirements_detector/detect.py", line 68, in find_requirements
    requirements = from_pyproject_toml(poetry_toml)
  File "/repo/.venv/lib/python3.8/site-packages/requirements_detector/detect.py", line 116, in from_pyproject_toml
    spec = spec["version"]
KeyError: 'version'

Environment (please complete the following information):

• OS: macOS
• Tool: all
• Prospector version: 1.8.0, 1.8.1, 1.8.2
• Python version: 3.8

Additional context Add any other context about the problem here. Putting the list of
dependencies installed, e.g. the output of pip freeze also helps.

[drbartling]

I see this same issue in prospector 1.8.3

[bellmatt]

I created a PR for landscapeio/requirements-detector#38 in requirements-detector (a dependency of prospector) but unsure how actively it is maintained.

The change in behaviour seen in prospector >1.8.0 was because autodetection of requirements was enabled by default, which was previously disabled (see this change in the release notes). There's no PR, commit or issue I can find that relates to that change.

In the meantime, I have found a workaround which seems to preserve the behaviour from <1.8.0 and allows prospector to run without raising the KeyError - setting this in prospector.yaml:

autodetect: false

From looking through https://github.com/PyCQA/prospector/blob/master/prospector/autodetect.py - I think this autodetect feature is only useful if you have code that depends on one of POSSIBLE_LIBRARIES = ("django", "celery", "flask") anyway.

[bellmatt]

Closing as this was fixed in 1.10