Add _from variants to support threaded tests

[rbtcollins]

These permit abstracting over some common global process state and allow
the use of home in project where in-process threaded tests are
desirable, and where rustup / cargo state is changed during the tests.

[rbtcollins]

Hmm, missed a site, updating it...

[rbtcollins]

All verified with my branch of rustup that is doing in-process testing.

[tesuji]

Sorry for late reply!

Overall I feel like this patch should remain in rustup.
These new APIs are wrapper for process unit testing of rustup only.
Does other tools like cargo and its plugins benefit from these APIs?

[rbtcollins]

Hi @lzutao without this patch we will have to stop using this crate. For now we've moved to using a forked copy of it.

Yes, the motivation for the patch is to facilitate testing, but because the state that home consults is process global state, we need a trait to influence it to be able to perform threaded tests in-process.

Cargo and other tools would be able to write in-process threaded tests using this facility as well.

We haven't yet taken full advantage of this capability - our interlocks to deal with process invocation file locking races are still present - but even with that in place we've gotten ~8% test improvement with this, and are targeting much higher numbers as we can now move to unit tests rather than full stack tests, without having to invert the entire code base to introduce explicit dependency injection everywhere. (And even if we did that, we'd still have to modify home to permit injection into home for the state it consults).

[rbtcollins]

Another way of thinking about this is as follows: imagine that rustup had implemented this in-process threaded testing support before this library was broken out to unify the code. We would still have wanted to do the unification of the domain logic, but supporting this testing feature would not be a matter of question or debate. So I don't think it should be now either.

The how can certainly be - if you have different interfaces or means you'd like to see, just let me know.

[tesuji]

I am sorry for taking long to get back here.
This PR was on my TODO list. I kept ignoring it for several reasons.

Part of that is that I really don't really get
what problems the new added API going to solve.
You said about in-process thread tests. What are they?
Do they run rustup binary in each threaded of integration tests?
The rustup code that use the new APIs?

The second part is that why rustup couldn't add a wrapper around "home" crate API
for #[cfg(test)] only that could be used by unit tests?

That's said. Not receiving enough context makes me to be not confident about
adding new APIs and so I kept ignoring this PR.

PS: I don't satisfy with current new API documentation. But that another problem
before I clear out my misunderstandings first.

PS: Opensource is not easy. I will try my best to make API useful, easy-to-use.
So please bear with me.

[kinnison]

One reason for not wanting to special-case the operations in our test suite is that it'd alter the logic for handling our detection logic since we'd essentially have to special-case it in the test suite instead of trusting that the same logic will be run against our mocked environment. If we were to start putting down locks and manipulating the environment instead then it'd cause problems for any other unlocked access to the environment, and locking all access to the mocked environment would partially defeat the point of permitting more parallel in-process testing (to speed up our test cycle)

I can understand your reticence to increase your API surface, but I feel this addition will be helpful and not complex to maintain. Also I, and I imagine @rbtcollins would be prepared to help maintain things if you are very busy. We did briefly discuss suggesting that bringing home into the rust-lang stable might be a good idea, putting it into the collective maintenance wheelhouse of the dev-tools team since it's used by rustup and cargo (and hopefully eventually by other tooling)

[rbtcollins]

Hi @lzutao the challenge we have is that the rustup test suite is very slow; it is very slow because many of our tests involve forking and running an entire process, often many times. To fix this and have unit tests is going to be a very large amount of work for 2 reasons.
one because we need to rearrange the code to perform dependency injection, so that all dependencies like e.g. home directory, are not accessed by a global-state-function, but are instead a contextual lookup on some injected dependency.

and two, because we need to go through the code and figure out what each test that current invokes the CLI is actually trying to test, rewrite it as a narrow layer specific test, figure out if that reduces test coverage, if so backfill other narrow tests for the things left uncovered and so on.

This patch is in support of step one. There are basically two ways that dependency injection can be done: via thread local state, or via passed-down-state. The code churn to do passed-down-state is many thousands of lines of code change in rustup: we have to move the lookup of home directory, current working directory, username - everything like that - out of the library and into the top level entry point, store it in some sort of state struct, and then pass that down into every single function through the entire code base. It is doable but it makes the entire code base vastly more noisy and hard to read. We tried that first.

To see our in-process test support, have a look in the rustup code base; as I said, we've save about 8% on test runtime already simply by not forking every time we want to invoke rustup during a test run. Yes, we invoke rustup that many times.

A test only wrapper would be we have to compile rustup twice during tests, and @kinnison is already unhappy with compile times; secondly it would as @kinnison says reduce confidence in our test results. As it is we can inspect https://github.com/rust-lang/rustup/blob/a93284b929d2d6716f8bedebcbf3658b09763e9c/src/utils/utils.rs#L487 and see that it is the same code path in test or release builds.

[tesuji]

Sorry for the late review again! My remote build machine has died recently and I had a long time to set it up again.

I let a few thoughts. Most are just nitpicking.

[tesuji]

I, and I imagine @rbtcollins would be prepared to help maintain things

That's really nice to see your interests. I help maintain things but I don't have ownership of this repository.
You may want to ask @brson for transfering this repository to you or rust-lang.

[kinnison]

You may want to ask @brson for transfering this repository to you or rust-lang.

We'd only be doing this if between @brson and @lzutao maintenance is not really viable. Given how busy you both seem to be, it might be worthwhile doing, and putting ownership into the dev-tools group which I think @lzutao is a member of anyway.

What do you both think? If you'd rather keep control then I have no objections.

[cbeuw]

FYI this is partly blocking rustup from being packaged in distros, since rustup is depending on a specific commit on this PR branch, and distros are unlikely to package this on its own.

It's sad to read the disagreement over the default implementation for Env, looks like the original author has followed the steps of many burnt-out open source maintainers. But now @brson has repo ownership so hopefully we can push this forward and fix rustup's dependencies?

My (potentially ignorant, since I knew nothing about this crate) opinion is that the current state (no default for Env) is fine, and it's frankly what rustup has been using for 2 years. If we want to eventually provide the default implementation it's not even a breaking change, since all existing implementers wouldn't be affected - and in any case this crate not 1.0 yet.

[LucioFranco]

Closing this in favor of #29