Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace safety with pip audit #645

Merged
merged 9 commits into from
Nov 24, 2022
Merged

Replace safety with pip audit #645

merged 9 commits into from
Nov 24, 2022

Conversation

severo
Copy link
Collaborator

@severo severo commented Nov 23, 2022

https://github.com/pyupio/safety is updated only once per month (you have to pay to have more frequent updates). https://github.com/pypa/pip-audit has fewer stars (684 against 1.4k) but use open data and is maintained by https://github.com/pypa.

Note that we upgrade poetry to 1.2.2 in this PR.

We have some issues with pip-audit (see below), and the fixes are a bit hacky (editing the requirements.txt file with sed). Ideally, it would be managed in a proper poetry plugin (see https://github.com/opeco17/poetry-audit-plugin/ for a plugin based on safety), but I think it's not worth creating a new repo for now.


Some notes:

That's why we do:

bash -c 'poetry run pip-audit -r <(poetry export -f requirements.txt --with dev)'

We still have an issue, though: the requirements.txt file contains duplicates when the same package is required both with and without "extras", e.g. with requests (which is not considered a bug by poetry: python-poetry/poetry-plugin-export#129, python-poetry/poetry-plugin-export#157, reason: python-poetry/poetry#5688 (comment)):

requests==2.28.1 ; python_full_version == "3.9.6" \
    --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 \
    --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349
requests[socks]==2.28.1 ; python_full_version == "3.9.6" \
    --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 \
    --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349

but pip-audit fails in this case:

ERROR:pip_audit._cli:package requests has duplicate requirements: requests[socks]==2.28.1 (from RequirementLine(line_number=1992, line='requests[socks]==2.28.1 ; python_full_version == "3.9.6"     --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983     --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349', filename=PosixPath('/dev/fd/63')))

I added a comment here: pypa/pip-audit#84 (comment)

@codecov-commenter
Copy link

codecov-commenter commented Nov 23, 2022

Codecov Report

Base: 83.04% // Head: 85.90% // Increases project coverage by +2.85% 🎉

Coverage data is based on head (b6d4c8a) compared to base (38070c7).
Patch has no changes to coverable lines.

❗ Current head b6d4c8a differs from pull request most recent head 4a48536. Consider uploading reports for the commit 4a48536 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #645      +/-   ##
==========================================
+ Coverage   83.04%   85.90%   +2.85%     
==========================================
  Files          22       47      +25     
  Lines         920     1830     +910     
==========================================
+ Hits          764     1572     +808     
- Misses        156      258     +102     
Flag Coverage Δ
jobs_mongodb_migration 84.00% <ø> (?)
services_admin 78.38% <ø> (ø)
services_api 93.47% <ø> (?)
workers_first_rows 83.96% <ø> (ø)
workers_splits 89.44% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...grations/_20221117223000_cache_generic_response.py 64.86% <0.00%> (ø)
.../migrations/_20221116133500_queue_job_add_force.py 74.54% <0.00%> (ø)
...bs/mongodb_migration/src/mongodb_migration/plan.py 100.00% <0.00%> (ø)
services/api/src/api/routes/splits.py 91.17% <0.00%> (ø)
services/api/src/api/authentication.py 100.00% <0.00%> (ø)
jobs/mongodb_migration/tests/test_collector.py 100.00% <0.00%> (ø)
...ngodb_migration/src/mongodb_migration/collector.py 100.00% <0.00%> (ø)
...db_migration/migrations/_20221110230400_example.py 66.66% <0.00%> (ø)
services/api/src/api/utils.py 98.41% <0.00%> (ø)
.../mongodb_migration/src/mongodb_migration/config.py 100.00% <0.00%> (ø)
... and 15 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@severo severo merged commit 18a51af into main Nov 24, 2022
@severo severo deleted the replace-safety-with-pip-audit branch November 24, 2022 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants