dist: make installation asynchronous

[djc]

cc @FranciscoTGouveia

async-install.mov

[rami3l]

These changes look pretty reasonable and definitely an improvement over #4567’s attempt of spilling Arc everywhere.

I imagine that if correctly implemented, it would be equivalent to #4471 in terms of performance because as long as we are spawning the installation task, the extra overhead introduced by #4567 should be minimal.

It’s worth another bench nonetheless though.

PS: Would you mind investigating the CI failures? This implementation doesn’t seem to have covered all the edge cases.

[djc]

Running time rustup-init toolchain install beta (compiled with --release) on #4471:

4.78s user 14.11s system 101% cpu 18.604 total

On this PR:

5.12s user 14.54s system 138% cpu 14.214 total

So it looks like this PR is a bit faster, by making better use of parallelism.

On main:

5.21s user 14.69s system 103% cpu 19.154 total

[djc]

PS: Would you mind investigating the CI failures? This implementation doesn’t seem to have covered all the edge cases.

Oops, solved those.

[rami3l]

Thanks! I really want to test this with my own machine but I don't have my fiber installed until tomorrow... I hope you won't mind my delaying merging this for a bit just to be sure 🙏

[djc]

No problem! Hope the fiber install will work out better for you tomorrow!

[rami3l]

@djc Just tested a few times after connecting all my cables. The thing did work which is super cool, but I have some concerns when downloading the cargo component, which seems to be stuck all of the time.

However, considering the fact that this implementation shouldn't have set any bias of any kind in that regard, I'm considering this to be a particular environment-related or remote-related problem (the same thing happens for stable rustup on my side and not just latest). Anyway, I think we can merge this first and see how it goes.

Many thanks again for all the refactoring work to make the final implementation more clearer and much more elegant than the original :)