Support artifact checksum pinning

[jin]

From Bazel slack: https://bazelbuild.slack.com/archives/CE6AZNQP7/p1556024364000500

Coursier supports it: coursier/coursier#625

[jin]

I'm probably doing this incorrectly, but the --checksum flag doesn't seem to be doing what I think it should do:

$ sha1sum /usr/local/google/home/jingwen/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar
88e3c593e9b3586e1c6177f89267da6fc6986f0c  /usr/local/google/home/jingwen/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar

$ cat /usr/local/google/home/jingwen/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar.sha1 
88e3c593e9b3586e1c6177f89267da6fc6986f0c

$ coursier fetch com.google.errorprone:error_prone_annotations:2.2.0 --checksum 88e3c593e9b3586e1c6177f89267da6fc6986f0c
mirrorConfFiles=List(MirrorConfFile(/usr/local/google/home/jingwen/.config/coursier/mirror.properties, true))
allMirrors0=List()
Resolution error: Error downloading com.google.errorprone:error_prone_annotations:2.2.0
  not found: /usr/local/google/home/jingwen/.ivy2/local/com.google.errorprone/error_prone_annotations/2.2.0/ivys/ivy.xml
  checksum not found: 

[jin]

Looks like Coursier CLI doesn't support this yet, according to the referenced issue. We may have to do the checksum verification out of band, after downloading the jars and possibly while generating the BUILD targets.

[arjantop]

Would this issue also cover "locking" versions of all transitive dependencies? (Like Cargo.lock etc.)

[jin]

@arjantop That's #107

[JaredNeil]

@jin #107, does not seem to be about generating or using a .lock file.

[jin]

My understanding is that locking and pinning are similar to each other. #107 talks about locking top level artifacts to specific versions, but doesn't cover transitive dependencies. Let's continue discussions there?

[arjantop]

@jin If I understand #107 it is about overriding/pinning specific transitive dependencies to some version, .lock file would be generated automatically, pinning all versions to the resolved ones

[jin]

Yes, and I imagine lock file generation will involve Bazel WORKSPACE resolve files to a large extent.

[mandrean]

@jin Maybe this issue should be renamed "Support artifact checksum pinning" or something to differentiate it from #134 which is more actually verification IMHO

[JaredNeil]

One benefit of a lock file not yet discussed here is that the artifacts are pre-resolved to an artifact URL and shasum. This can save a lot of time every time you would normally need to re-run the coursier resolver (several minutes in our repository). Doing the resolution once and storing that information to a lock file saves us a ton of time in our CI builds.

[borkaehw]

@jin any thought on the benefit of a lock file that @JaredNeil mentioned?

[jin]

I agree. This is something that I'll discuss with @aehlig, who implemented the Bazel-wide lock file for external repositories (--experimental_workspace_resolved_file).

[borkaehw]

@jin just a follow-up. Any update on the lock file?

[jin]

Sorry, no updates on this yet.

[jin]

Work in progress to support artifact checksum pinning in a file read and exported by maven_install: #169