Opam downloads twice the same source

[vblot]

When installing package b that depends on a but has the same source, this source is downloaded twice in parallel. However, if package a is installed first then a subsequent install of b does not redownload the source.
When starting multiple downloads in parallel, opam should check that they are not redundant.
Here is an example, with a=coq and b=coqide:

user@computer:~$ opam --version
2.0.1
user@computer:~$ opam switch create test 4.07.1

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ocaml-base-compiler.4.07.1] downloaded from cache at https://opam.ocaml.org/cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> installed base-bigarray.base
-> installed base-threads.base
-> installed base-unix.base
-> installed ocaml-base-compiler.4.07.1
-> installed ocaml-config.1
-> installed ocaml.4.07.1
Done.
# Run eval $(opam env) to update the current shell environment
user@computer:~$ eval $(opam env)
user@computer:~$ opam install coqide
The following actions will be performed:
  - install camlp5             7.06.10-g84ce6cc4 [required by coqide]
  - install conf-pkg-config    1.1               [required by conf-gtksourceview]
  - install conf-m4            1                 [required by ocamlfind]
  - install conf-gtksourceview 2                 [required by coqide]
  - install ocamlfind          1.8.0             [required by coq, lablgtk]
  - install num                1.1               [required by coq]
  - install lablgtk            2.18.7            [required by coqide]
  - install coq                8.9.0             [required by coqide]
  - install coqide             8.9.0
===== 9 to install =====
Do you want to continue? [Y/n] y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[camlp5.7.06.10-g84ce6cc4] downloaded from cache at https://opam.ocaml.org/cache
[lablgtk.2.18.7] downloaded from cache at https://opam.ocaml.org/cache
[num.1.1] downloaded from cache at https://opam.ocaml.org/cache
[ocamlfind.1.8.0] downloaded from cache at https://opam.ocaml.org/cache
[coq.8.9.0] downloaded from cache at https://opam.ocaml.org/cache
[coqide.8.9.0] downloaded from cache at https://opam.ocaml.org/cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> installed conf-m4.1
-> installed conf-pkg-config.1.1
-> installed conf-gtksourceview.2
-> installed ocamlfind.1.8.0
-> installed num.1.1
-> installed lablgtk.2.18.7
-> installed camlp5.7.06.10-g84ce6cc4
-> installed coq.8.9.0
-> installed coqide.8.9.0
Done.
user@computer:~$ opam remove coq
The following actions will be performed:
  - remove coqide 8.9.0 [uses coq]
  - remove coq    8.9.0
===== 2 to remove =====
Do you want to continue? [Y/n] y

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> removed   coqide.8.9.0
-> removed   coq.8.9.0
Done.
user@computer:~$ opam clean
Cleaning up switch test
Clearing cache of downloaded files
Clearing logs
user@computer:~$ opam install coq
The following actions will be performed:
  - install coq 8.9.0

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[coq.8.9.0] downloaded from cache at https://opam.ocaml.org/cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> installed coq.8.9.0
Done.
user@computer:~$ opam install coqide
The following actions will be performed:
  - install coqide 8.9.0

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[coqide.8.9.0] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> installed coqide.8.9.0
Done.

[dra27]

While discussing a related issue yesterday, we had another thought. As well as downloading the archive multiple times, opam also has to extract the archive multiple times. It might be worth investigating if it's worth a single extraction followed by cp -a in this instance. It's also another potential reason to integrate ocaml-tar, as that would trivially allow a single extraction to write the files to multiple targets.