`mix deps.get` when violating policy but already installed.

[maennchen]

• Add {:cowlib, "~> 2.17"} dependency
• HEX_POLICY="" mix deps.get - Installs, warning, exit 0
• HEX_POLICY="hexpm:erlefsecuritywg/strict" mix deps.get - Shows Policy Breakdown, Warning, exit 0

I believe we should have an exit 1 no matter if a dependency is locked already if it violates a policy.

[ericmj]

Maybe that should be setting on the policy? I have always been working under the assumption that mix.lock should be trusted.

If not your CI/CD could suddenly stop working because a package author retired a version you are using. This is something we generally want to avoid, see left_pad.

[maennchen]

@ericmj For cooldown I can follow that argument, but for retirement and vulnerabilities, it’s normal that those arise after an initial installation.

If not, what does the policy actually guarantee?

[ericmj]

The policy guarantees you don't move to a version that doesn't match your policy restrictions, currently it doesn't do anything if the package if the package is already locked.

I can see that you would want a hard blocker on security advisories, but for renamed/deprecated/invalid retirements I would want my CD to break because an author decided their package is deprecated.

I can think of two improvements:

1. Add a policy setting "Strict enforcement" for each restriction which also checks your lock
2. Add a flag to mix hex.audit --policy which check against the specified policy, mix hex.audit always checks against the lock.

[maennchen]

mix hex.audit currently always exits non-zero. So essentially the policy would make it more lax instead of more strict. But adding policies to it would be nice anyways.

Strict enforcement sounds more like something you would want to enable / disable in the client and not in the policy itself.

[josevalim]

There is a very practical concern here: if mix deps.get breaks all your builds because of the policy, we will see people immediately adding flags to remove the policy, because being unable to test or merge any other ongoing work until the violation is addressed is going to be counter-productive. For example, imagine the vulnerability is reported, someone on my team is already working on it, but now the designer cannot do any work because mix deps.get is failing on their machine (or worse, they remove the policy).

On the other hand, we do want to encourage people to run something like mix hex.audit or mix deps.get --enforce or whatever.

And I assume this only applies if you have a policy?