Update coursier to 2.0.7

[cheister]

For #471 update coursier to a non-RC version.

Collected release notes from https://get-coursier.io/versions are

v2.0.7

Optimization
Memo-ize classes that can be created at a high rate
coursier.core.{Dependency, Module, Publication} are now memo-ized (using weak references). This can lower the amount of duplicate instances of these objects, and lower GC pressure.

Added in #1900, thanks to @jtjeferreira.

Fixes
Un-escape characters in file URLs
When using local repositories with special characters in their paths, such as spaces, escaped characters were not properly handled, which created spurious not-found errors.

v2.0.6

Fixes
Persisted checksums written outside of the coursier cache
Persisted checksums were added in 2.0.5. These could mistakenly be written outside of the coursier cache, under ~/.ivy2/local for example. This release fixes that.

Fixed in #1909, thanks to @oyvindberg.

Changing artifacts in cs get command
The cs get command allows to download things via the coursier cache:

$ cs get https://repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/maven-metadata.xml
~/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/maven-metadata.xml
If a URL it is passed ends with ?changing=true, cs get should assume the artifact is changing (and may check for updates if the last check is older than the TTL). This release includes a fix for that. Prior to it, these were not assumed to be changing, so were not checked for updates.

Fixed in #1907, thanks to @martijnhoekstra.

v2.0.5

Optimizations
Cache computed checksums
Computed checksums of downloaded files are now written in the cache, and read from there later on. This short-circuits checksum computation for anything already in cache, which can significantly speed up resolutions with large dependency graphs.

Added in #1801, thanks to @oyvindberg.

Changes
New cs channel command
This release adds a new cs channel command. This command allows to add channels, like

$ cs channel --add my.company:cs-channel
or list them with

$ cs channel --list
This deprecates the --add option of the cs install command (which is still available, and prints a deprecation message).

Added in #1899, thanks to @shubhamJay.

Allow to pick particular files in archives as prebuilt binaries in application descriptors
It is now possible to specify that prebuilt binaries for applications correspond to particular files in archives, like

  "prebuilt": "zip+https://github.com/sbt/sbt/releases/download/v${version}/sbt-${version}.zip!sbt/bin/sbtn-${platform}"
This example picks the file sbt/bin/sbtn-${platform} (where ${platform} is replaced by x86_64-apple-darwin on macOS, …) from the zip archive at URL https://github.com/sbt/sbt/releases/download/v${version}/sbt-${version}.zip (where ${version} is replaced by 1.4.1 for example).

v2.0.4

Changes
Update zsh completions
New commands (java, java-home, …) are now proposed as completions, and removed ones (spark-submit) were dropped. Some removed options were removed from completions too. (Changed in #1897, thanks to @note.)

cs launch --python changes
cs launch has a --python option, that sets jna.library.path prior to starting the app, so that it can load libpython.so via JNA. This release also sets jna.nosys to false, so that libpython.so can be loaded from the standard locations, most notably PATH on Windows.

Add --python option to cs bootstrap
When generating bootstraps, passing --python ensures jna.library.path and jna.nosys are set prior to starting the app, so that libpython.so can be loaded from JNA.

Add jna field in app descriptors
App descriptors now accept an optional jna field, whose value must be a list of strings. If this list contains "python", jna.library.path and jna.nosys are set prior to starting the app, so that libpython.so can be loaded from JNA.

v2.0.3

Fixes
Fix javax.net.ssl.SSLHandshakeException thrown when downloading things from github.com from the native Windows launcher

v2.0.2

Changes
Add sbtn to the default application list, installed during cs setup
Fixes
Interpret some SSL handshake errors as not found errors, when looking for prebuilt binaries during cs install and cs setup (this has been a problem when installing cs on Windows since 2.0.1 was tagged, when using the native launcher)
Miscellaneous
Adapt some tests to make them pass on Windows (with or without CRLF to LF conversion in particular, #1890, thanks to @martijnhoekstra)
Mention in the documentation how to pass custom Java options to apps installed by cs install (#1892, thanks to @Poorva17)

v2.0.0

No changes since 2.0.0-RC6-27.

v2.0.0-RC6-27

Changes
Add support for the arm and aarch64 JVM architectures in the java and java-home commands (#1849, thanks to @matsluni)
Build native launchers with GraalVM 20.1.0 Java 11 (former coursier versions used GraalVM 19.3.1 Java 8)
Add --jvm-index option to the java / java-home / setup commands, allowing to pass a custom JVM index URL, also accepting aliases such as jabba (current default) and cs (supporting less JVM flavors, but more up-to-date)
Other
coursier now uses GitHub actions for its CI, rather than Travis CI and Appveyor
The switch to GraalVM 20.1.0 and GitHub actions fixes the generation of native launchers on the CI, which should be shipped as GitHub release assets again (which was broken in 2.0.0-RC6-25 and 2.0.0-RC6-26)

v2.0.0-RC6-24

Fixes
Fix handling of version intervals in JSON report (dependencies with a version interval as version had empty dependencies).
Ensure latest.stable doesn't match milestones or alpha / beta versions.
Accept space characters as separator in COURSIER_REPOSITORIES, so that Ivy repositories with different metadata and artifact patterns can be used.
Fix update and list command on Windows.
Enhancements
Don't print empty line when the result of the fetch command is empty.
In the java and java-home commands, don't re-download JVM index when not necessary. (If the cached index has a matching JVM, it's used straightaway, and the index is not updated. Pass --update to force updating the index.)
Add aliases for GCS repositories (gcs, gcs-eu, gcs-asia)

v2.0.0-RC6-23

Fix possible IllegalArgumentException: toMillis not allowed error, that can sometimes happen with infinite TTLs (which became the default from the CLI in 2.0.0-RC6-22 when an application rather than Maven coordinates is passed)

v2.0.0-RC6-22

Changes
Drop support for legacy cache location ~/.coursier/cache
From version 2.0.0-RC6-14 up to 2.0.0-RC6-21, the coursier CLI prints a warning if the legacy cache location (pre Nov 2017),

~/.coursier/cache
is being used, pointing to cache-migration to help migrate to the new cache location.. Support for ~/.coursier/cache is now dropped altogether.

New configuration directory on macOS
The configuration directory changed from

~/Library/Preferences/Coursier
to

~/Library/Application Support/Coursier
The latter is more correct per the Apple documentation, see the discussion here for more details.

The former location might be deprecated in a late future. Files there are still accepted for the time being.

Fixes
Concurrency issues
Work around rare "Resource deadlock avoided" errors
Wait for concurrent downloads when downloading things from bootstraps, instead of loudly failing
Other issues
Don't crash if the cache directory is a symbolic link
Don't crash when printing zsh completions from the native launcher
Accept JDK archives not putting files under Contents/Home on macOS, so that cs java --jvm adopt:1.10.0-2 -version now works.
Fix handling of 1. prefix in JVM versions, which was faulty in some cases, so that cs java --jvm openjdk:10 -version now works, in particular.
New features
New get command
Prints the path of a file in the coursier cache, downloading it if necessary. Use like

$ cs get https://repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/2.0.0-RC6-21/coursier-cli_2.12-2.0.0-RC6-21.jar
/Users/alexandre/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cli_2.12/2.0.0-RC6-21/coursier-cli_2.12-2.0.0-RC6-21.jar
Prefer local JVMs over latest ones from the JVM index
In the java and java-home commands, when passing short versions like adopt:11 or graalvm:20, if a valid JVM matching that version is already unpacked locally, it is used straightaway. In particular, if a newer matching version is available per the JVM index, the local one is still preferred. Pass --update to force using the latest matching version from the index (former behavior).

Less superfluous network requests with applications, and managed JVMs
In the java and java-home commands, if a valid JVM is available locally, it is used straightaway (see previous section), and no attempt is made to update the index. Previously, the index was updated if the last check was older than the TTL (24 hours by default). The index is now only checked when --update is passed.

When passing applications to the resolve, fetch, launch, or bootstrap commands, like cs launch scala, the TTL now defaults to Inf (infinite). That means that if all the necessary files to start an application are in the cache, it is launched straightaway, even if a newer version could have been downloaded. Pass an explicit TTL value to recover the former behavior, like cs launch scala --ttl 24h.

v2.0.0-RC6-21

Slightly less secure defaults for credentials (those passed via --credentials and --credentials-file now don't require https by default, and are automatically tried when getting an authentication error on their host)
Allow to read JVM option files upon startup in bootstraps (pass --jvm-option-file .jvmopts to the bootstrap command to have the bootstrap read .jvmopts if it exists)

v2.0.0-RC6-20

Add jcenter repository alias (use like cs resolve -r jcenter …)
Install vanilla sbt-launcher by default in cs setup, rather than the non-standard coursier-based sbt-launcher

v2.0.0-RC6-19

Keep colors when printing trees in more cases by default (keep them on CIs, and when the output of coursier is piped)
Ensure cs list doesn't crash if the app directory doesn't exist
Fix an issue in the way execve works (should make launch with explicit JVMs, like cs launch ammonite --jvm 11, work)
Fix some wrong conflicts being sometimes reported

v2.0.0-RC6-18

Fix global exclusions (-E option) not excluding root dependencies themselves in the CLI
Add support for global exclude node in Ivy XML files

v2.0.0-RC6-16

Update Scala.JS to 1.0.1 (from 0.6.32)
Return more detailed results in coursier.{Artifacts, Fetch}

v2.0.0-RC6-15

Fix some spurious and unexpected "concurrent download errors" exceptions (coursier is fine with concurrent downloads otherwise). Seems to be a regression since 2.0.0-RC6-2. Fixed in #1704, thanks to @jodersky.

v2.0.0-RC6-14

Deprecate cache directory ~/.coursier/cache
This impacts users who used a coursier version released before Nov. 2017. These users should have a ~/.coursier/cache directory on their disk. For these users, this directory should be picked as default cache directory most of the time, per the rules introduced along with the newer cache location.

As of this version, and until 2.0.0 final, running the coursier CLI will print a warning if ~/.coursier/cache is found and used as default location per the rules picking the default cache location. That warning message points to the cache-migration tool, that helps migrate ~/.coursier/cache to the newer cache location.

You might have a ~/.coursier/cache directory, and not get that warning. This happens when you first ran a post Nov. 2017 coursier version on your system, that created the OS-specific cache directory, then ran a pre Nov. 2017 one, that created a ~/.coursier/cache nonetheless. You can run cache-migration too, if you'd like to migrate what's under ~/.coursier/cache to the newer cache.

Optimizations
Optimize some data structures used during resolution, thanks to @jtjeferreira.

v2.0.0-RC6-13

Take into account the COURSIER_INSTALL_DIR environment variable in the install command (thanks to @zheh12)
Fix reflection issue preventing the Windows native launcher to access the cache
On Windows, if cs is installed with cs install cs or via cs setup, running just cs now prints the help like on Linux and macOS, instead of running the setup command

v2.0.0-RC6-12

Add list command to list installed applications (thanks to @ckipp01)
Print the help rather than nothing when no arguments are passed to the CLI (thanks to @regadas)
Fix GraalVM version mismatch when installing GraalVM-based applications
Accept clojars as a repository, pointing to Clojure's clojars
Rework CLI documentation

v2.0.0-RC6-11

Manifest JAR -based launchers
The bootstrap command can now generate launchers that rely on the Class-Path field of manifests to set the launcher classpath. This requires all the required JARs to be already in the coursier cache, and these launchers can typically only be run on the current machine, as they hardcode the path to the cache.

Example

$ cs bootstrap coursier -o coursier --manifest-jar
$ ls -lh coursier
-rwxr-xr-x  1 user  group   2,1K 20 mar 16:14 coursier
$ unzip -l coursier
  Length      Date    Time    Name
---------  ---------- -----   ----
     7939  03-20-2020 16:14   META-INF/MANIFEST.MF
---------                     -------
     7939                     1 file
In the generated JAR, META-INF/MANIFEST.MF contains things like

Manifest-Version: 1.0
Class-Path: /Users/alexandre/.coursier/cache/v1/https/repo1.maven.org/
 maven2/io/get-coursier/coursier-cli_2.12/2.0.0-RC6-10/coursier-cli_2.
 12-2.0.0-RC6-10.jar /Users/alexandre/.coursier/cache/v1/https/repo1.m
 aven.org/maven2/org/scala-lang/scala-library/2.12.10/scala-library-2.
 12.10.jar …
Main-Class: coursier.cli.Coursier
Publish the coursier-launcher module in 2.11
This module allows to generate bootstrap, assemblies, etc. The bootstrap command relies on it. It has no dependencies, so it's straightforward to compile it and publish it in 2.11 too. (The 2.11 version should be used by ammonite-runner in particular.)

v2.0.0-RC6-10

Accept exclusions for individual dependencies on the CLI, like
$ cs resolve org:name:version,exclude=something%foo
Ensure resolve command-specific options aren't accepted from the fetch, launch, and bootstrap commands. This includes the --tree, --reverse-tree, and --what-depends-on options in particular.

Add retry helpers in the resolve command, allowing to retry resolving a number of times until resolution succeeds, like

$ cs resolve org:name:ver --retry 1min --attempts 30
That allows to wait for a sync to Maven Central with a single command for example.

v2.0.0-RC6-9

Fix issue appending directories to PATH on Windows, in the install / java / setup commands

v2.0.0-RC6-8

Various changes and fixes in the java, install, and setup commands (see discussion in 2.0.0-RC6-2)

v2.0.0-RC6-7

Various changes and fixes in the java, install, and setup commands (see discussion in 2.0.0-RC6-2)

v2.0.0-RC6-6

Various changes and fixes in the java, install, and setup commands (see discussion in 2.0.0-RC6-2)

v2.0.0-RC6-5

Order artifacts by default in fetch, launch, and bootstrap commands (classpath order, #1546, thanks to @dwijnand)
Various changes and fixes in the new java and setup commands (see discussion in 2.0.0-RC6-2)

v2.0.0-RC6-4

Like 2.0.0-RC6-3, this is mostly a convenience release, to more easily test the latest experimental developments. (See the release notes for 2.0.0-RC6-2 for more discussion.)

v2.0.0-RC6-3

Mostly a convenience release, to more easily test the latest experimental developments. (See the release notes for 2.0.0-RC6-2 for more discussion.)

v2.0.0-RC6-2

Native launcher changes
Don't let native-image interpret the -D options passed to the native launcher, so that we can interpret those ourselves as normal options, like the JAR-based launcher does.
Have the launch command accept Java options via --java-opt / -J, that are passed to java when launching a JVM application from the native launcher, like cs launch -J -Xmx3g ammonite.
In progress changes
The changes below are meant to be described more extensively and documented in later releases, while being more tested in the mean time. These are mentioned here for the sake of comprehensiveness.

Major in-progress overhaul of the internals of the bootstrap and install commands.
Early support for pre-built native launchers for the install command.
Experimental java and java-home commands. java fetches and starts a custom JVM (like cs java --jvm graalvm@19.3+ -version), and java-home fetches and prints the home directory of a custom JVM (like cs java-home --jvm 11).
Add support for generating GraalVM native images to the bootstrap command. Use like cs bootstrap --native-image coursier -o custom-cs.
Add experimental setup command, installing a JVM if none is found locally, updating JAVA_HOME and PATH if needed, and installing a few applications via the install command (cs, ammonite, sbt, in particular).
Other changes
Add a --scala alias for --scala-version, in most CLI commands. Use like cs launch ammonite --scala 2.13.

v2.0.0-RC6-1

Fix ugly List(…) in the command listing of the help message

v2.0.0-RC6

Drop scala 2.11 support
Handle JDK version intervals in Maven profile activation (#1504)
Fix regression in the handling of properties when both parent POMs and Maven import scope are involved (regression since 2.0.0-RC5-4, #1505)
Add support for mirrors when using the coursier.Versions API (#1515, thanks to @fthomas)
Switch to Scala.JS 0.6.32
Don't issue HEAD requests in local mode (--mode local from the CLI, #1522, thanks to @fthomas)
Fix progress bars in the Windows command terminal (regression since around 2.0.0-RC3-3)
Build native CLI launchers with GraalVM 19.3.1, build Windows launchers too (#1525, see instructions on the website)

v2.0.0-RC5-6

Fix rare occurrence of attempts to download the same URL twice in parallel, which is rejected by the default logger (regression in 2.0.0-RC5-4)

v2.0.0-RC5-4

Fix disprecancy with Maven in the handling of Maven properties, affecting dependencies substituting properties in their parent POM (#1457)
Add --candidate-urls option to resolve command, printing the URLs of possible artifacts. Note that optional artifacts are printed too, so some of those URLs may return 404 Not Found. Authentication isn't printed / reported either.
Mark the artifact of dependencies with an explicit classifier or artifact type as non optional. This means things like cs fetch org.scala-lang:scala-library:2.13.1,classifier=foo now fail loudly.

[cheister]

@jin I don't believe I have access to upload to mirror.bazel.build can you upload this version?

[jin]

Yes, let me patch and apply

[cheister]

@jin any ideas what's going wrong with Windows? Unfortunately I don't have a Windows machine I can test on.

[jin]

Unfortunately I still haven't been able to get a Windows machine right now either :(

[cheister]

Looks like calling coursier 2.0.6 without arguments on Windows was hanging on CI because it was awaiting input for

Checking if ~\AppData\Local\Coursier\data\bin is in PATH
  Should we add ~\AppData\Local\Coursier\data\bin to your PATH? [Y/n]

by adding --help to the coursier command it no longer asks to update the PATH and runs successfully.

[jin]

Nice catch!! Thank you. Do you mind adding a comment on the --help append to mention this issue?

[cheister]

Done. And updated to 2.0.7 since it was released since this PR was opened. Can you update 2.0.7 to the google mirror?

[jin]

Done.

$ bazel run //:mirror_coursier
DEBUG: /usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/bazel_toolchains/rules/rbe_repo/version_check.bzl:68:14:
Current running Bazel is ahead of bazel-toolchains repo. Please update your pin to bazel-toolchains repo in your WORKSPACE file.
DEBUG: /usr/local/google/home/jingwen/.cache/bazel/_bazel_jingwen/8484bc4fff18ee4a905b69a9ddb0e143/external/bazel_toolchains/rules/rbe_repo/checked_in.bzl:125:14: buildkite_config not using checked in configs; Bazel version 3.7.1 was picked/selected but no checked in config was found in map {"0.20.0": ["8.0.0"], "0.21.0": ["8.0.0"], "0.22.0": ["8.0.0", "9.0.0"], "0.23.0": ["8.0.0", "9.0.0"], "0.23.1": ["8.0.0", "9.0.0"], "0.23.2": ["9.0.0"], "0.24.0": ["9.0.0"], "0.24.1": ["9.0.0"], "0.25.0": ["9.0.0"], "0.25.1": ["9.0.0"], "0.25.2": ["9.0.0"], "0.26.0": ["9.0.0"], "0.26.1": ["9.0.0"], "0.27.0": ["9.0.0"], "0.27.1": ["9.0.0"], "0.28.0": ["9.0.0"], "0.28.1": ["9.0.0"], "0.29.0": ["9.0.0"], "0.29.1": ["9.0.0", "10.0.0"], "1.0.0": ["9.0.0", "10.0.0"], "1.0.1": ["10.0.0"], "1.1.0": ["10.0.0"], "1.2.0": ["10.0.0"], "1.2.1": ["10.0.0"], "2.0.0": ["10.0.0"], "2.1.0": ["10.0.0"], "2.1.1": ["10.0.0", "11.0.0"], "2.2.0": ["11.0.0"], "3.0.0": ["11.0.0"], "3.1.0": ["11.0.0"], "3.2.0": ["11.0.0"], "3.3.0": ["11.0.0"], "3.3.1": ["11.0.0"], "3.4.1": ["11.0.0"], "3.5.0": ["11.0.0"]}
INFO: Analyzed target //:mirror_coursier (13 packages loaded, 49 targets configured).
INFO: Found 1 target...
Target //:mirror_coursier up-to-date:
  bazel-bin/mirror_coursier
INFO: Elapsed time: 0.899s, Critical Path: 0.03s
INFO: 3 processes: 3 internal.
INFO: Build completed successfully, 3 total actions
INFO: Build completed successfully, 3 total actions

Copying file://external/coursier_cli_v2_0_7/file/downloaded [Content-Type=application/octet-stream]...
- [1 files][ 34.0 MiB/ 34.0 MiB]
Operation completed over 1 objects/34.0 MiB.

[cheister]

Regarding

Drop support for legacy cache location ~/.coursier/cache

Not sure if we want to update our local Coursier cache settings to also remove support for ~/.coursier/cache?

[cheister]

I'm also not sure what is going on with the Windows tests. The coursier windows executable name changed between versions from standalone-x86_64-pc-win32.zip to cs-x86_64-pc-win32.exe but I don't think that is being used by the tests?

[jin]

Let's fork the cache discussion to a follow up issue/PR?

[jin]

Looks like this update may be able to fix issues in #446, will rebase #446 on this after merging.