Pinning to packaging < 22.0

[dmaljovec]

• safety version: 2.3.5
• Python version: All
• Operating System: All

Description

We are attempting to create virtual environments for our developers that include all production dependencies for their package plus all of our static analyzers and security checks, so they can reproduce our CI system locally. It has recently come up that the latest version of safety pins to packaging<22.0 which is incompatible requires some extra work to operate with some of our existing package dependencies.

Can you shed some more light on the comment in this commit: aa1b153? What exactly is meant by "preventing issues?" Is this something that you could use help on or is on your roadmap to support?

What I Did

First look at my input file (reduced to show minimal reproducible error):

> cat test.in
pandera

Next, compile that file to a lockfile:

> pip-compile --no-emit-index-url --no-header --quiet --pip-args --no-input --resolver=legacy --output-file=test.txt test.in        
> cat test.txt                                                                                                                      
mypy-extensions==0.4.3
    # via typing-inspect
numpy==1.23.5
    # via
    #   pandas
    #   pandera
packaging==22.0
    # via pandera
pandas==1.5.2
    # via pandera
pandera==0.13.4
    # via -r test.in
pydantic==1.10.2
    # via pandera
python-dateutil==2.8.2
    # via pandas
pytz==2022.6
    # via pandas
six==1.16.0
    # via python-dateutil
typing-extensions==4.4.0
    # via
    #   pydantic
    #   typing-inspect
typing-inspect==0.8.0
    # via pandera
wrapt==1.14.1
    # via pandera

Now to show the dev requirements:

> cat test-dev.in                                                                                                                   
-c test.txt

safety

And finally, attempt to lock that file:

> pip-compile --no-emit-index-url --no-header --quiet --pip-args --no-input --resolver=legacy --output-file=test-dev.txt test-dev.in
Could not find a version that matches packaging<22.0,==22.0,>=21.0 (from -c test.txt (line 7))
Tried: 14.0, 14.0, 14.1, 14.1, 14.2, 14.2, 14.3, 14.3, 14.4, 14.4, 14.5, 14.5, 15.0, 15.0, 15.1, 15.1, 15.2, 15.2, 15.3, 15.3, 16.0, 16.0, 16.1, 16.1, 16.2, 16.2, 16.3, 16.3, 16.4, 16.4, 16.5, 16.5, 16.6, 16.6, 16.7, 16.7, 16.8, 16.8, 17.0, 17.0, 17.1, 17.1, 18.0, 18.0, 19.0, 19.0, 19.1, 19.1, 19.2, 19.2, 20.0, 20.0, 20.1, 20.1, 20.2, 20.2, 20.3, 20.3, 20.4, 20.4, 20.5, 20.5, 20.6, 20.6, 20.7, 20.7, 20.8, 20.8, 20.9, 20.9, 21.0, 21.0, 21.1, 21.1, 21.2, 21.2, 21.3, 21.3, 22.0, 22.0
There are incompatible versions in the resolved dependencies:
  packaging==22.0 (from -c test.txt (line 7))

I realize there are some pretty simple workarounds in this case, and we are investigating tools like pip-compile-multi to solve this problem more robustly with our internal tooling, but I just wanted to flag in case others do not have the luxury of dodging a specific version of packaging. It could be someone is reliant on a new feature. I also acknowledge that this new version of packaging is hot off the press, and will take time to onboard. Let me know if I can help, I really like this tool!

[qs5779]

I am sure I am not alone, but today github's dependabot updated one of my projects, dependency to packaging = ^22.0, which of course will not work in conjunction with safety 2.3.5.

[yeisonvargasf]

Hi @dmaljovec and @qs5779,

Thanks for your interest in this issue.

Can you shed some more light on the comment in this commit: aa1b153? What exactly is meant by "preventing issues?" Is this something that you could use help on or is on your roadmap to support?

Packaging 22.0 removed LegacyVersion, which Safety in some situations uses. It's in the roadmap, and we should solve it in the following minor releases.

There is already work in progress related to this issue. Indeed I'll request your help checking it when it gets released.

Thanks.

[dmaljovec]

Awesome! Thanks for the quick reply and look forward to the update.

[dashmug]

@yeisonvargasf Please let us know through this issue once that issue is solved. Thank you.

We have temporarily removed safety from our projects until this gets fixed.

[yeisonvargasf]

@dashmug, the next version will solve this issue. We are near to releasing this new version.

[yeisonvargasf]

Update here: we released a beta version (Feb 26) with significant changes and additional improvements.

This version ( https://pypi.org/project/safety/2.4.0b1/) should fix this issue. Could you try?

Let me know if this version works for you.

[nim65s]

Hi,
I have the same issue, and this pre-release fix it for me, thanks !

[nim65s]

Hi again,

5 months after my previous comment, packaging v23.1 is out, and the pre-release workaround is not working anymore with it:

poetry lock
Updating dependencies
Resolving dependencies... (11.5s)

Because safety (2.4.0b1) depends on packaging (>=21.0,<=23.0)
 and no versions of safety match >2.4.0b1,<3.0.0, safety (>=2.4.0b1,<3.0.0) requires packaging (>=21.0,<=23.0).
So, because <my-package> depends on both packaging (^23.1) and safety (^2.4.0b1), version solving failed.

[yeisonvargasf]

Hi @nim65s, I'm sorry you're experiencing this issue; we revamped Safety CLI in the coming version 3.0 (this month); this 3.0 version will fix this issue with packaging and add more features and services.

I recommend you pin packaging if possible. If multiple users are having the same issue with 2.4.0.b1, we may look at how we can release a 2.4.0b2 version with a fix for this; otherwise, it's better to wait for Safety 3.0 this month, which is in a closed beta phase.

[Nachtalb]

Like some previous users, we also have version conflicts. One of our dependencies uses packaging ^23.1 while safety forces it to <=23, which of course won't work. It's quite vexing when a well-known package like Safety rigidly pins to an outdated version of a dependency that's even more widely adopted. So we hope you can at least push an update to b2 that is able to use the latest version or at least 23.1.

[grutts]

Hi do you have an update on when this will be fixed? We're running into a packaging mismatch with several libraries because of this

[yeisonvargasf]

Hi, @Nachtalb and @grutts; there is a 2.4.0b2 version that removes the upper clause restriction for the packaging dependency.

[washeck]

@yeisonvargasf What is the recommended solution at the moment? We are a paying custumer of PyUp.io and we want to switch from pipenv check to a tool working in poetry. I thought safety would be the right tool but this issues makes me feel bit concerned about the stability and maintenance policy of safety.

It was reported more than a year ago, 2.4.0b1 was published as workaround which stopped working, now 2.4.0b2 seems to be the workardound. 3.0 was promised in August 2023 and it is still in alpha version. The reason to use such a tool is to make us pass our software audit more easily and adding dependency on alpha or beta versions is contrary to this goal.

Why don't you release 2.4.0 as proper fix until 3.0 is ready?

[Jwomers]

@yeisonvargasf What is the recommended solution at the moment? We are a paying custumer of PyUp.io and we want to switch from pipenv check to a tool working in poetry. I thought safety would be the right tool but this issues makes me feel bit concerned about the stability and maintenance policy of safety.

It was reported more than a year ago, 2.4.0b1 was published as workaround which stopped working, now 2.4.0b2 seems to be the workardound. 3.0 was promised in August 2023 and it is still in alpha version. The reason to use such a tool is to make us pass our software audit more easily and adding dependency on alpha or beta versions is contrary to this goal.

Why don't you release 2.4.0 as proper fix until 3.0 is ready?

Hi @washeck. Please accept my apology for this sequence of events and the inconvenience caused. We are releasing version 3.0.0 within the next 24 hours, which supports the same safety check command from 2.4.0b2, so you will be able to upgrade to this version this week. Safety 3.0.0 includes native support for reporting on Poetry files. Full details on Safety 3.0.0, what changes it includes, and how to upgrade will be detailed here in GitHub and in our Documentation hub.

In addition to the release of version 3 this week, we will also be removing the beta flag on version 2.4.0, to support those who want to remain on version 2.x of safety.

We did not communicate clearly here, and I acknowledge that we must do better, both for Safety customers and the open source community. To ensure this won't happen again, we will also be moving to a public roadmap and maintenance schedule for Safety CLI 2.x and 3.x going forward, and adopting a trunk based development approach. In the meantime, I appreciate the feedback here and invite you to reach out at justin@safetycli.com with any questions you have about safety cli, or any account-related questions.

Justin - CEO, Safety Cybersecurity