Use pre-commit for Git hooks

[andrewhare]

Fixes #3271.

NOTE: This is blocked by a pending 2.2.0 release of pre-commit to support post-checkout hooks.

This introduces a new flag to the dvc install command:

$ dvc install --use-pre-commit-tool

Running this command will do several things:

• Verify that pre-commit is installed on the system
• Write the existing DVC git hooks bash files to .dvc/tmp/hooks instead of to .git/hooks
• Build a pre-commit configuration that calls the hook files in .dvc/tmp/hooks.
• Check the current directory for an existing .pre-commit-config.yaml, if one is found, merge the configuration into it, otherwise dump the configuration to a new .pre-commit-config.yaml.

[efiop]

@andrewhare Looks really good! Thank you so much for the PR!

For the record: We've discussed during the meeting with @andrewwhare, that it might be worth creating those hook files somewhere (e.g. in .dvc/ or .dvc/tmp/) and triggering those from pre-commit config, instead of writing shell wrapper directly into them.

[andrewhare]

@efiop Thanks for the feedback!

I like the idea of writing the hook files to disk (possibly reusing/refactoring _install_hook to write to the correct directory) and calling them there.

[efiop]

I wonder if we really need to run pre-commit for the user, maybe config is enough + a hint now run 'pre-commit install' to install git hooks? Just a question.

[efiop]

This one is needed in case user has .pre-commit-hooks.yaml, but no config? Or am I missing something?

[pared]

I would consider adding test asserting that hooks work.
I tried to check their behavior and upon running this script:

#!/bin/bash

rm -rf repo storage git_repo
mkdir repo git_repo

main=$(pwd)
pushd git_repo
git init --quiet --bare
popd

pushd repo

git init --quiet
dvc init -q
git remote add origin $main/git_repo
dvc remote add -d str $main/storage

dvc install --use-pre-commit-tool

echo data >> data

dvc add data -q

git add -A
git commit -am "something"

I get the following error:

An error has occurred: InvalidConfigError: 
==> File .pre-commit-config.yaml
==> At Config()
==> At key: repos
==> At Repository(repo='local')
==> At key: hooks
==> At Hook(id='dvc-post-checkout')
==> At key: stages
==> At index 0
=====> Expected one of commit, commit-msg, manual, merge-commit, prepare-
commit-msg, push but got: 'checkout'
Check the log at /home/prd/.cache/pre-commit/pre-commit.log

[pared]

Is there some particular reason to use unittest? We re moving towards using pytest, I think it would be good to make it test also pytest-based.

[efiop]

@pared Did you install the latest pre-commit version? As mentioned in the header, it requires 2.2.0

EDIT: can reproduce, looking into it.

[efiop]

For the record: taking over this PR.

[efiop]

Currently you need to dvc install --use-pre-commit-tool on each clone, as pre-commit won't have .dvc/tmp/hook/* scripts to run without it. Looks like it would be better to run hooks directly through dvc. Looking into it.

[efiop]

Will be fixed after merge.

[efiop]

Ok, merging to adjust on master. So far works good for me, but it is tied to github repo, so need to try out on our master.

Docs will come separately after.

[shcheklein]

@efiop do we need adjust any docs because of this change?

[efiop]

@shcheklein Yes, hence "Docs will come separately after."

[skshetry]

It'd be good to have --use-pre-commit or just --pre-commit. Tool is kind of redundant here.

[efiop]

@skshetry Im worried about it being confused with pre-commit hooks.