Read poetry.lock file

[pawamoy]

I would like safety to be able to read dependencies from a poetry.lock file. It is written in TOML format.

Here is an example of how a dependency is written:

[[package]]
category = "dev"
description = "A configurable sidebar-enabled Sphinx theme"
name = "alabaster"
optional = false
python-versions = "*"
version = "0.7.12"

Here is a complete file:

poetry.lock

Currently the only way to use safety when developing a project with poetry is to actually install the dependencies (which is costly in CI) and run something like:

poetry run pip freeze | poetry run safety check --stdin

I would like to be able to run directly safety check -r poetry.lock or something equivalent 🙂 .

[a-recknagel]

+1 on the feature request, I would love to have this as well.

As a side note, from poetry==1.0.0a0 onward you can use poetry export -f requirements.txt to generate a requirements.txt file that can be consumed by safety check. But it is still a bit inconvenient to install poetry in a CI job just to generate that file.

[pydolan]

If using pre-commit, here's a hook to using the stdin approach @pawamoy described:

- repo: local
    hooks:
      - id: python-safety-dependencies-check
        name: Run safety on project packages.
        entry: bash -c "poetry run pip freeze | poetry run safety check --stdin"
        pass_filenames: false
        language: system

[kierun]

Running the above command with the latest beta of poetry does not work.

; poetry run pip freeze | poetry run safety check 
--stdin

[FileNotFoundError]
[Errno 2] No such file or directory
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

and

; poetry debug   

Poetry
Version: 1.0.0b2
Python:  3.7.4

Virtualenv
Python:         3.7.4
Implementation: CPython
Path:           /home/usr/.cache/pypoetry/virtualenvs/usignals-py3.7
Valid:          True

System
Platform: linux
OS:       posix
Python:   /home/usr/.pyenv/versions/3.7.4

[kierun]

Please ignore me. Not enough coffee before I start work… Safety was not installed.

I shall go sit in a corner wearing a dunce hat.

/sigh

[rafaelpivato]

It is clear there is work to be done here. One problem I see so far is that Safety was supposed to be using pyupio/dparse more than it is now, and such library should indeed have Poetry support.

The work around looks great anyway!

[pawamoy]

For reference:

• issue in pyup: support pyproject.toml and poetry pyup#332
• issue in dparse: Add supprort for Poetry (pyproject.toml and poetry.lock) dparse#36

[pawamoy]

I'm now using

poetry export -f requirements.txt | safety check --stdin

...instead.

• pip freeze was also showing the current package as an editable install, that I had to filter out with an additionnal grep -v pipe (with ugly regex to match underscores/dashes and upper/lower case). poetry export -f requirements.txt don't show dev dependencies, which is good since I don't care if they have CVEs as they are not part of the published package, and it also doesn't show the current package, fixing the previous pain point.
• safety is not installed as a dev dependency anymore, because doing so make it vulnerable to malicious packages installed in the same env, effectively canceling its usefulness (the malicious packages can override its check function to hide themselves). See https://nvd.nist.gov/vuln/detail/CVE-2020-5252. Instead, I install it with pipx so it's isolated. You could also always use pipx if you want: poetry export -f requirements.txt | pipx run safety check --stdin

[cjolowicz]

safety is not installed as a dev dependency anymore, because doing so make it vulnerable to malicious packages installed in the same env

You can run safety in an isolated environment, and still track it as a development dependency in Poetry:

# (steps to create and activate virtualenv omitted)
poetry export --dev --format=requirements.txt --output=requirements.txt
pip install --constraint=requirements.txt safety
safety check --file=requirements.txt

This works best when automated with Nox.

Here is an article that describes the technique in more detail (disclaimer: I'm the author):
https://medium.com/@cjolowicz/hypermodern-python-3-linting-e2f15708da80

[yrro]

Having just found out about CVE-2020-5252, I'm migrating from installing safety into my poetry-managed environment to having it installed into a separate environment.

In the process, I noticed that safety check (when run in the poetry-managed environment) misses out on a known vulnerability in the cryptography package.

$ poetry run python -m safety check --full-report
[...]

  Safety v2.3.5 is scanning for Vulnerabilities...
  Scanning dependencies in your environment:

  -> /home/sam/.cache/pypoetry/virtualenvs/hitron-exporter-ugtVc_Mk-py3.11/lib64/python3.11/site-packages
  -> /home/sam/.cache/pypoetry/virtualenvs/hitron-exporter-ugtVc_Mk-py3.11/lib/python3.11/site-packages

  Using non-commercial database
  Found and scanned 34 packages
  Timestamp 2023-02-18 16:46:04
  0 vulnerabilities found
  0 vulnerabilities ignored
+======================================================================================================================================================================+

 No known security vulnerabilities found. 

+======================================================================================================================================================================+
[...]

When I use poetry export and feed the output to safety, it correctly detects the following vulnerabilities:

$ poetry export --with=dev -E container -E freeipa-vault | safety check -r -
[...]

  Safety v2.3.5 is scanning for Vulnerabilities...
  Scanning dependencies in your files:

  -> <stdin>

  Using non-commercial database
  Found and scanned 48 packages
  Timestamp 2023-02-18 16:57:12
  2 vulnerabilities found
  0 vulnerabilities ignored

+======================================================================================================================================================================+
 VULNERABILITIES FOUND 
+======================================================================================================================================================================+

-> Vulnerability found in cryptography version 37.0.4
   Vulnerability ID: 52173
   Affected spec: >=37.0.0,<38.0.3
   ADVISORY: Cryptography versions from 37.0.0 and before 38.0.2 include a statically linked copy of OpenSSL that has known
   vulnerabilities.https://github.com/pyca/cryptography/security/advisories/GHSA-39hc-v87j-747x
   CVE-2022-3786
   For more information, please visit https://pyup.io/v/52173/f17


-> Vulnerability found in cryptography version 37.0.4
   Vulnerability ID: 52174
   Affected spec: >=37.0.0,<38.0.3
   ADVISORY: Cryptography versions from 37.0.0 and before 38.0.2 include a statically linked copy of OpenSSL that has known
   vulnerabilities.https://github.com/pyca/cryptography/security/advisories/GHSA-39hc-v87j-747x
   CVE-2022-3602
   For more information, please visit https://pyup.io/v/52174/f17

 Scan was completed. 2 vulnerabilities were found. 
[...]

I've double-checked and the Poetry-managed environment does contain /home/sam/.cache/pypoetry/virtualenvs/hitron-exporter-ugtVc_Mk-py3.11/lib64/python3.11/site-packages/cryptography-37.0.4.dist-info and the corresponding cryptography directory containing the package's Python modules.

I was going to file a separate issue about this but I figured I should mention it in here, since proper support for Poetry would make a separate issue irrelevant.

[dgutson]

@Franco0700 pls check this out, especially the last comment rgd poetry-audit.