Adding pip-audit fails with pre-commit.ci due to pip support

[cooperlees]

describe your issue

I added pip-audit to my pre-commit configuration for bandersnatch and the CI doesn't like how it uses pip. Is there anyway we can tune the environment here or is this outside what pre-commit.ci offers? If so, should we document this and have users move back to running pre-commit in my own action/ci job?

• This all runs fine locally and I would imagine would in my own GitHub action

PR with failing action added: pypa/bandersnatch#1116
pre-commit.ci job: https://results.pre-commit.ci/run/github/133377409/1651359348.J_Nu59G_RqyOHsV2bHdyoQ

Sorry if this is the wrong place to log issues for the .ci offering. This just seemed the best place from my quick searching etc.

pre-commit --version

Latest? Don't know ...

.pre-commit-config.yaml

https://github.com/pypa/bandersnatch/blob/pre-commit-pip-audit/.pre-commit-config.yaml#L41-L45

  - repo: https://github.com/trailofbits/pip-audit
    rev: bafa0f8
    hooks:
      -   id: pip-audit
          args: ["-r", "requirements.txt"]

pip-audit config: https://github.com/trailofbits/pip-audit/blob/main/.pre-commit-hooks.yaml

~/.cache/pre-commit/pre-commit.log (if present)

In the cloud.

[asottile]

the issues for https://pre-commit.ci are at https://github.com/pre-commit-ci/issues (linked at the bottom of every pre-commit.ci page)

network is forbidden at runtime for the free tier because it invites abuse (crypto mining, etc.) and makes runtime unpredictable. you can utilize ci: skip: to avoid such hooks

more information: https://github.com/pre-commit-ci/issues/issues?q=is%3Aissue+network+is%3Aclosed

[asottile]

fwiw, imo pip-audit is much too slow to be added as a pre-commit check and is probably better suited for a check in tox / etc. -- hooks should be speedy or your contributors will be more likely to turn them off entirely

[woodruffw]

fwiw, imo pip-audit is much too slow to be added as a pre-commit check and is probably better suited for a check in tox / etc. -- hooks should be speedy or your contributors will be more likely to turn them off entirely

Just as a note: pip-audit can be either fast or slow, depending on how you use it. If you have a preconfigured environment, pip-audit will be nearly instantaneous; if you run it via pip-audit -r ... without --require-hashes or --no-deps, it'll take roughly as long as pip install -r ... (since that's effectively what it does, with a small amount of isolation). It also caches, so the cost of pip-audit on a project is mostly front-loaded.

I'm not too familiar with pre-commit, but if users can be pointed towards the former use case rather than the latter that'll probably be a much better experience for the average user. Otherwise yes, it's effectively identical to an environment setup and therefore is quite slow (at least at first) and perhaps not ideal for pre-commit 🙂

[asottile]

pre-commit will always run it from an isolated environment so it doesn't really make sense at all

[woodruffw]

pre-commit will always run it from an isolated environment so it doesn't really make sense at all

That makes sense for the preconfigured environment case. For users who have frozen/completely hashed environments, either pip-audit -r ... --require-hashes or pip-audit -r ... --no-deps (upcoming: pypa/pip-audit#255) will be roughly as fast since they don't involve any dependency resolution.

But either way I leave it to you; there's no significant advantage AFAICT to having pip-audit as a commit hook versus a CI task.