Optimize disk access during install

[brson]

Installation is noticeably slow. I believe there is redundant disk work during installation, associated with the transaction machinery.

[brson]

I don't specifically know what is inefficient here, but somebody could do some stracing on Linux and see if there's anything obvious.

[iamthad]

stat appears to get called 3 times for each file in set_file_perms:

• One to get the file permissions in needs_x

  rustup.rs/src/rustup-dist/src/component/package.rs

  Line 131 in 67d740f

  meta.permissions().mode() & 0o700 == 0o700 // If it is rwx for the user, it gets the X bit

• One inside utils::is_directory

  rustup.rs/src/rustup-dist/src/component/package.rs

  Line 141 in 67d740f

  let is_dir = utils::is_directory(dest_path);

• One to get the file permissions (again) before setting the mode

  rustup.rs/src/rustup-dist/src/component/package.rs

  Line 151 in 67d740f

  let mut perm = meta.permissions();

This could probably be reduced to just one stat call.

[kinnison]

Given that the majority of filesystem operations which Rustup performs are reading and writing large (10s of MBs) datasets, I think the stat() inefficiency is by no means the most important thing to consider for speed.

I do, however, think that we should look at consider if there's anything we can do to improve install performance on Windows.

[mati865]

Installation on Linux filesystems is fast even on HDD so measuring performance there won't help much for Windows but still it'd be nice to make it even faster.

I don't know which one is more limiting NTFS or AV software but shipping less files would be the way to go here.
Nightly on my Linux machine consists of 17957 files, 16617 of them docs. It'd be significant improvement for Windows if they could be fused together into few bigger files.

[iamthad]

If I understand correctly, the set_file_perms function walks the entire tree of files after they have been extracted to ensure their permissions are correct. This is especially costly in the docs, since there are a tremendous number of files there as @mati865 points out. I would expect tens of thousands of fewer system calls in an installation operation to be noticeable, especially e.g. on NFS.