rustup CLI design

[brson]

Edit: Here's the interface as of 3/20. Here's a comparison to pyenv.

Here's what I'm thinking on the rustup UI.

A few main principles.

• The changes are mostly going to be reorganizing the existing
  commands. The current output is generally good enough.
• The most important thing is updating the channels, so the simple
  command rustup will just update all the things.
• Custom toolchain management will be disconflated from
  release channel toolchain management. The --copy-local,
  --link-local, and --installer features will be dealt with differently.
  rustup toolchain link <path> will link in development toolchains
  (and they will automatically get a fallback cargo).
• Commands that work on channels use the current override by default,
  so you don't have to e.g. specify the toolchain when adding a target.
  Such commands will probably have --channel flags.
• We're going to try to use more sub-sub-commands, like the "multirust
  self" command, to free up the root namespace for more features and bury lesser features.

TLDR

This is the important stuff

• rustup. Upgrades all tracking toolchains and self-upgrades,
  tracking toolchain statuses.
• rustup default <toolchain>. Sets the default toolchain, installing it if it isn't already. Same as multirust default but without the custom toolchain parts.
• rustup update [toolchain]. Updates the current toolchain or a given toolchain, installing if it isn't already. Mostly the same as multirust update but without custom toolchain parts.
• rustup toolchain link. Symlinks a toolchain. Useful for development. The same as multirust update --link-local.
• rustup toolchain remove
• rustup toolchain list. Lists the installed toolchains in some useful way.
• rustup target list/add/remove. Modify target architectures for a toolchain.
• rustup override add <toolchain>. Sets the toolchain for cwd. Same as multirust override <toolchain>.
• rustup override remove
• rustup run <toolchain> [args ..]. Runs a command with the environment of a given toolchain. Same as multirust.
• rustup self uninstall
• rustup self update
• rustup self upgrade-data

Basic usage

After installation, the user already has a toolchain installed
("stable" by default). Many users don't need to interact with rustup
again until the next version of Rust is released. At that time they
will upgrade Rust by writing

$ rustup

rustup upgrades every tracking toolchain installed, so by default it
upgrades stable; then it displays the status of every tracking
toolchain, then it quietly upgrades itself.

The bare rustup command is non-interactive. It just does all the
business. It's not clear yet whether it will update ndk components.

The rustup update <toolchain> can still be used to update a single toolchain.

At some point they want to try out nightly so they write

$ rustup default nightly

FIXME: The word 'default' could also be something like 'use'
or 'activate'.

FIXME: I might want a more convenient way to 'try out' toolchains
than switching the default or running rustup run nightly bash.

Now they want to try android by installing another target.

$ rustup target list
$ rustup target add arm-linux-androideabi

All commands related to changing a toolchains targets are now
subcommands of 'target'. They can also rustup target remove.

Finally after making all these changes, they want to be reminded what
the state of the world is, they write

rustup status

It's not clear what all this shows, but perhaps the state of the
tracking channels and the current override information. It replaces
show-default and show-override.

Toolchain managament

• rustup toolchain list
• rustup toolchain update [toolchain]. FIXME
• rustup toolchain link <path> - the equivalent of update --link-local
• rustup toolchain remove
• rustup toolchain show [toolchain] - the same info displayed
  after updates. I often want to know this. subset of rustup status

I want to hold off on adding equivalents of --copy-local and
--installer.

Overrides

• rustup override add <toolchain>
• rustup override remove
• rustup override list
• rustup override show. Subset of rustup status.

Misc

update-data and delete-data will sink into the self subcommand.

I think run and proxy can be merged.

cc @Diggsey @alexcrichton

Looking for feedback.

Edit: There are some contradictory bits in here about whether to preserve functionality for just showing the current override and other lesser functions. I'm not sure where I stand.

[alexcrichton]

The thrust of this all sounds great to me. I wasn't personally deeply familiar with all the bells and whistles of multirust.sh, so for example I don't know what it means to add an override. Consolidating the top level commands into sub-sub commands seems good.

I will say that multirust run nightly rustc ... is pretty wordy to type, but I can't really thing of a way to shorten it other than my own personal shell aliases.

For switching between toolchains and such ti may be worth taking a look at tools like RVM or NVM, maybe there's a "standard precedent"?

[gyscos]

then it quietly upgrades itself

I guess there will be an option to disable it, for when rustup itself is not writeable? (For instance when installed by the package manager).

[brson]

I guess there will be an option to disable it, for when rustup itself is not writeable? (For instance when installed by the package manager).

Yes. Good idea.

[brson]

I think I'd still like to have a rustup update [toolchain] command. I still want to be able to update a single toolchain, because often I only care about nightly, and I know stable isn't going to change. rustup update toolchain is a lot nicer than rustup toolchain update [toolchain].

Perhaps the bare rustup shouldn't be updating and reporting on every channel, but only the default.

[chriskrycho]

@brson, in line with @alexcrichton's suggestion about looking at rvm/nvm above, I've been following this with some interest and am also a heavy user of rbenv/pyenv/nodenv, and those are also worth looking at for comparison. It's a pretty well-designed UI.

• Where we have override here, they have local, with a global setting matching our default
• They give you the ability to specify for a given shell, which is really nice, e.g. pyenv shell 2.7.10
• You can list the current version (pyenv version) or all installed versions (pyenv versions)

The overall structure for installed versions is pretty similar to what you've already been doing with multicast, too, with a dedicated directory at e.g. ~/.pyenv.

A common pyenv workflow would look something like this:

$ pyenv install 2.7.10
$ pyenv install 3.5.1
$ pyenv global 2.7.10
$ pyenv version
2.7.10 (set by /Users/chris/.pyenv/version)
$ python --version
2.7.10
$ mkdir my_project
$ cd my_project
$ pyenv local 3.5.1
$ pyenv version
3.5.1 (set by /Users/chris/my_project/.python_version
$ python --version
Python 3.5.1

Obvious there are some substantial tweaks that would be needed to account for the special notions we have in Rust for our stable/beta/nightly channels approach, so I'm not suggesting you just adopt it wholesale, but again: it's a great UI, so probably worth stealing from as makes good sense.

[brson]

Thanks for the feedback @chriskrycho.

I've taken your pyenv examples and translated them to both multirust-rs and rustup as implemented.

# Install (or update?) a toolchain
$ pyenv install 2.7.10
$ multirust update 2.7.10
$ rustup update 2.7.10

$ pyenv install 3.5.1
$ multirust update 3.5.1
$ rustup update 3.5.1

# Set the default / global toolchain
$ pyenv global 2.7.10
$ multirust default 2.7.10
$ rustup default 3.7.10

# Get the active toolchain version and reason
$ pyenv version
2.7.10 (set by /Users/chris/.pyenv/version)

$ multirust show-override
default toolchain: nightly
default location: /home/brian/dev/.multirust/toolchains/nightly

rustc 1.9.0-nightly (b12b4e4e3 2016-03-17)
cargo 0.10.0-nightly (ece4e96 2016-03-17)

$ rustup show? 'rustup override show'? (not implemented) 'pyenv version' is good

... snip commands for other tools ...

$ pyenv local 3.5.1
$ multirust override 3.5.1
$ rustup override add 3.5.1

# The extra `add` in rustup is unfortunate. To remove it's `rustup override remove`
# whereas in pyenv has `pyenv override --unset`, multirust is `multirust remove-override`.
# penv is more convenient for common cases here. Maybe that says bad things about
# sub-sub commands.

... snip already covered stuff ...

[brson]

I noticed that pyenv stores the local toolchain overrides inside the project directories. I don't remember why I didn't do that in multirust (perhaps I didn't want people to check that information in) but it has obvious advantages. Probably doesn't impact the cli much though.

[brson]

I like pyenv's scheme that puts the override version at ~/.pyenv/version or .python-version, making it part of the pyenv interface that you can frob manually. In multirust the default is stored at ~/.multirust/default, but the overrides shouldn't be edited by hand. ~/.multirust/version is taken by the multirust metadata version.

[brson]

@chriskrycho pyenv shell 2.7.10 is rustup run 2.7.10 bash in rustup. I've considered a shell command but it was hard to justify saving the word 'bash'. This is related to @alexcrichton's desire to write rustup run nightly rustc easier. Are both rustup shell nightly and rustup rustc nightly important enough to get their own subcommands? Hopefully that's where we draw the line at rustup run aliases (though rustup shell does have the extra shell detection - it's not a simple alias).

Edit: After further reading it seems that shell does more than just launch a shell. It allows certain shells to always use certain versions. There must be more to this than I understand still. Why does one want to tie bash specifically to a particular version of Rust?

[brson]

For conversation's sake here is the --help output for what I've already implemented:

nightlybrian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup --help
rustup 0.0.5
Diggory Blake
The Rust toolchain installer

USAGE:
    rustup [FLAGS] [SUBCOMMAND]

FLAGS:
    -v, --verbose    Enable verbose output
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    default      Set the default toolchain
    update       Install or update a toolchain from a Rust distribution channel
    run          Run a command with an environment configured for a given toolchain
    target       Modify a toolchain's supported targets
    toolchain    Modify the installed toolchains
    override     Modify directory toolchain overrides
    self         Modify the rustup installation
    help         Prints this message or the help message of the given subcommand(s)


brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup default --help
rustup-default
Set the default toolchain

USAGE:
    rustup default [FLAGS] <toolchain>

FLAGS:
    -h, --help    Prints help information

ARGS:
    <toolchain>


brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup update --help
rustup-update
Install or update a toolchain from a Rust distribution channel

USAGE:
    rustup update [FLAGS] [ARGS]

FLAGS:
    -h, --help    Prints help information

ARGS:
    [toolchain]



brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup run --help
rustup-run
Run a command with an environment configured for a given toolchain

USAGE:
    rustup run [FLAGS] <toolchain> <command>... [--]

FLAGS:
    -h, --help    Prints help information

ARGS:
    <toolchain>
    <command>...


brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup target --help
rustup-target
Modify a toolchain's supported targets

USAGE:
    rustup target [FLAGS] [SUBCOMMAND]

FLAGS:
    -h, --help    Prints help information

SUBCOMMANDS:
    add       Add a target to a Rust toolchain
    help      Prints this message or the help message of the given subcommand(s)
    list      List installed and available targets
    remove    Remove a target  from a Rust toolchain



brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup toolchain --help
rustup-toolchain
Modify the installed toolchains

USAGE:
    rustup toolchain [FLAGS] [SUBCOMMAND]

FLAGS:
    -h, --help    Prints help information

SUBCOMMANDS:
    help      Prints this message or the help message of the given subcommand(s)
    link      Create a custom toolchain by symlinking to a directory
    list      List installed toolchains
    remove    Uninstall a toolchain



brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup override --help
rustup-override
Modify directory toolchain overrides

USAGE:
    rustup override [FLAGS] [SUBCOMMAND]

FLAGS:
    -h, --help    Prints help information

SUBCOMMANDS:
    add       Set the override toolchain for a directory
    help      Prints this message or the help message of the given subcommand(s)
    list      List directory toolchain overrides
    remove    Remove the override toolchain for a directory



brian@ip-10-235-20-183:~/dev/multirust-rs⟫ rustup self --help
rustup-self
Modify the rustup installation

USAGE:
    rustup self [FLAGS] [SUBCOMMAND]

FLAGS:
    -h, --help    Prints help information

SUBCOMMANDS:
    help            Prints this message or the help message of the given subcommand(s)
    uninstall       Uninstall rustup.
    update          Downloadand and install updates to rustup
    upgrade-data    Upgrade the internal data format.

[brson]

@alexcrichton nvm calls rustup run nightly rustc a nvm run nightly, and uses nvm exec nightly [command] for running any command. I don't think this is appropriate for Rust though since the nvm run command will actually run a script with node. I might rather have a rustup rustc or rustup cargo command.

[brson]

We could switch from 'default' / 'override' terminology to 'global' / 'local' since there's such strong precedence. My main resistence is because of potential confusion when discussing system installs of rust, which are also in a sense 'global'. I do think 'default' / 'override' is clearer than 'global' / 'local'.

nvm uses 'use' for this, which is the word I like best, but doesn't convey the globalness of default/global, nor have an obvious partner word like override/local.

[alexcrichton]

I've personally always been a little skeptical of some of the things rvm/nvm do, specifically directory-level overrides, shell overrides, etc. They're convenient to use, but they come at a toll. Awhile back I noticed that a new shell windows took noticeably longer to give me a prompt, and it got way faster once I stopped loading nvm. Similarly, I wouldn't want all normal rustc and cargo commands to all of a sudden have to do a ton of filesystem work before they hit the actual command themselves (we've talked about this before though).

To me, so long as the startup time is or can be super optimized, I'm fine with a global/local concept (although I will agree that just having one "default" is easier to understand).

In terms of shell vs run vs rustc vs cargo, I agree that rustup run nightly isn't appropriate for us because Cargo/rustc are used so interchangeably (there's not really one default to sweeten), and rustup rustc seems slightly odd because where would the channel argument get passed? rustup rustc nightly seems like you're passing an argument to rustc and rustup nightly rustc might be ambiguous.

[chriskrycho]

@brson:

I do think 'default' / 'override' is clearer than 'global' / 'local'.

👍 on that; I agree 100%.

Edit: After further reading it seems that shell does more than just launch a shell. It allows certain shells to always use certain versions. There must be more to this than I understand still. Why does one want to tie bash specifically to a particular version of Rust?

Huh. Maybe I'm misunderstanding the source and the help, but I believe all that pyenv shell does is set an environment variable, PYENV_VERSION, which is the highest-ranked way of deciding what shell to use out of those installed. (I think it just goes PYENV_VERSION > /local/path/hierarchy/.python-version > ~/.pyenv/version.)

Since PYENV_VERSION could be set by a shell config, you could override it universally, but you definitely don't have to; I never have. My normal use case with Python or Ruby is to switch from whatever local or global version to a specific version useful for a specific tool, e.g. homebrew recipes which depend on Python 2.7.

That particular use case is likely to be much less common in Rust, unless people decide to start using it as their built tool glue (!), but it's definitely more convenient IMO for a series of commands using a non-default/non-override toolchain than actually launching a full shell instance just for those tasks, a la rustup run nightly bash. That said, as long as there's a way to do it, I'll be happy!

---

@alexcrichton:

To me, so long as the startup time is or can be super optimized, I'm fine with a global/local concept (although I will agree that just having one "default" is easier to understand).

I agree about startup times; zsh plugins sometimes make me sad. That said, I would guess that the local overrides are going to be really useful to people. Thinking, for example, about using something like Diesel in one project, but not in another (and so, nightly in one project but not another): it's very convenient for both the shell itself and anything launched from it (editors, e.g.) to be able to set the relevant paths appropriately.

---

One other point: Any chance you could make install an alias for update? (I can submit a presumably-straightforward PR for it if you want.) It's a little bit of UI that initially confused me about multirust: to me, at least, it wasn't intuitive that I needed to update to get the first install.

[Diggsey]

I think multirust has existed long enough, and seen enough use, for its own conventions to be just as important as rbenv conventions (eg: global/local vs default/override) and in that case, I think that default/override are better names anyway.

[brson]

Huh. Maybe I'm misunderstanding the source and the help, but I believe all that pyenv shell does is set an environment variable, PYENV_VERSION, which is the highest-ranked way of deciding what shell to use out of those installed. (I think it just goes PYENV_VERSION > /local/path/hierarchy/.python-version > ~/.pyenv/version.)

Guh, you seem to be right, but I'm still confused. I don't understand how running pyenv shell foo effects the environment in its parent's shell. Just exporting an enviroment variable in a child process is not enough for it to be reflected in the parent process. Obviously there's just something about this I don't understand. (I would just run it and see but so far I've been unable to install pyenv).

That particular use case is likely to be much less common in Rust, unless people decide to start using it as their built tool glue (!), but it's definitely more convenient IMO for a series of commands using a non-default/non-override toolchain than actually launching a full shell instance just for those tasks, a la rustup run nightly bash. That said, as long as there's a way to do it, I'll be happy!

I totally agree! pyenv shell seems really useful. If I understand how it's implemented we will add it :) Hopefully it works on windows too.

[brson]

Any chance you could make install an alias for update? (I can submit a presumably-straightforward PR for it if you want.) It's a little bit of UI that initially confused me about multirust: to me, at least, it wasn't intuitive that I needed to update to get the first install.

I've been thinking about this, and whether update should just be renamed to install. I do feel like update is again right for Rust here though. Our tool is very much about updating our release channels - it's even in the name "rustup". I'd like to think about this question more before deciding to duplicate the command with a different name.

We could just switch to 'global' / 'local' and 'install' to maintain familiarity. I'm not sure how important it is for rustup to work like pyenv. @chriskrycho to be clear, do you prefer to keep 'default' / 'override' or switch to 'global' / 'local'?

[brson]

I think multirust has existed long enough, and seen enough use, for its own conventions to be just as important as rbenv conventions (eg: global/local vs default/override) and in that case, I think that default/override are better names anyway.

Thanks for the feedback. I do agree that we have to weigh our own precedent against others' and our existing users are already familiar with our terminology.

[kbknapp]

@brson @Diggsey

Since I saw it mentioned in #88, you can now order the arguments in whatever order you want, instead of just the default which is alphabetical.

This can help emphasize arguments or prioritize common options and commands. The ordering can be done on an argument by argument basis (or sucommand), or be derived from the declaration order.

Arguments can also be hidden from the help text, in case there are rare or developmental arguments and commands.

Finally, what clap calls two separate categories flags and options can actually be combined into a single options category similar to a docopt help message.

Combining all of the above one should be able to get a good help message.

[chriskrycho]

@brson, I think you're right (as are @Diggsey and @alexcrichton) about sticking with default and override. The semantics there are nice and clear. I offered pyenv just as a nice UI to compare and steal from where it makes sense; I definitely don't think we need to be constrained by those choices.

install and update

Even if you don't make it a full alias, having it respond in some way would probably help with first-time use. Multirust using update was initially confusing to me for two reasons:

1. You're not updating anything on the first install of any given channel. When I go from no nightly to some nightly, it's not intuitive from a UI perspective that I'm "updating" that channel. It seems like I'm "installing" it instead. (The flip side is: once I have a channel installed, I'm not installing, but updating one in place, which is quite different from how e.g. pyenv install works.)

   I think sticking with update is good because of what it does the majority of the time, but having install respond with something like that "Maybe you mean update?" would help.

2. Neither with multirust nor with the original rustup did I think of it as being primarily about updating my Rust channels, though I see that now. (Until you said that, I always thought of "rustup" as "Rust up!" sort of like "Saddle up!" however ridiculous that might sound. 😛)

How shell works

So, the way pyenv works is by using a series of shell script shims in the ~/.pyenv/shims directory, very similar to the shims in the old multirust bin directory. Those shims are what actually get run when you call python or 2to3 or pip or whatever else.

They just run pyenv exec <program> <args>, and the pyenv script just checks in order for PYENV_VERSION, the version specified in a file-tree-local .python-version file, or the version specified in ~/.pyenv/version to determine what version to use.

Once it has a version, it uses it to build the path to the actual program being shimmed in ~/.pyenv/versions. For example, if your installed versions included 2.7.10 and 3.5.1, you'd have full installations of each of those at ~/.pyenv/versions/2.7.10 and ~/.pyenv/versions/3.5.1 and it would just build the path to those accordingly.

So pyenv shell doesn't affect the surrounding shell environment at all other than setting that PYENV_VERSION environment variable and using that as the highest priority is determining what to run from the shims.

[brson]

Since I saw it mentioned in #88, you can now order the arguments in whatever order you want, instead of just the default which is alphabetical.

Great. I've already changed the rustup binary to use my ordering.

So pyenv shell doesn't affect the surrounding shell environment at all other than setting that PYENV_VERSION

@chriskrycho that 'other than setting PENV_VERSION' is what I don't understand. You can't call a subprocess, have it change the environment then exit, and have that environment variable still be set in the parent process.

Ok, I just went and tested it and this is exactly what happens - calling rbenv shell foo sets the parent processes RBENV_VERSION environment variable. I'm so confused. This upends my understanding of environment variables.

After further investigation I've figured out how it works. The key is this:

Installs the sh dispatcher. This bit is also optional, but allows rbenv and plugins to change variables in your current shell, making commands like rbenv shell possible. The sh dispatcher doesn't do anything crazy like override cd or hack your shell prompt, but if for some reason you need rbenv to be a real script rather than a shell function, you can safely skip it.

Every time the shell starts it calls eval rbenv init, and that creates an rbenv shell function that intercepts calls to rbenv. Then when you run rbenv shell it doesn't actually run rbenv just sets the environment variable.

Thanks for the other feedback. We'll do something for rustup install for sure.

[brson]

Here's an issue for shell.

[brson]

Here's an issue for install.

[chriskrycho]

OH! I had totally forgotten about eval (rb|py|nod)env init. Of course, that makes a ton of sense.

I totally forgot about the way the environment variable/subprocess bits related there (if I ever knew it). TIL.

Thanks for the thoughtful interactions about it! 👍

[brson]

This issue has served it's purpose. The new CLI is landed.