RFC: Add `cargo install`

[alexcrichton]

Add a new subcommand to Cargo, install, which will install [[bin]]-based
packages onto the local system in a Cargo-specific directory.

Rendered

[nagisa]

Totally needs a $CARGO_INSTALL_HOME which overrides everything, because nobody wants to get binaries stored in their $XDG_CONFIG_HOME to which their $CARGO_HOME is very likely to be set to.

[alexcrichton]

Ah yes I had forgotten about my intention to do this. I'm thinking something along the lines of:

• --root is an argument to the install and uninstall subcommands to specify the root manually.
• The environment variable CARGO_INSTALL_ROOT can specify the root
• The configuration key install.root can specify the root
• Finally, $CARGO_HOME/bin is used.

How does that sound?

[SimonSapin]

Would binaries go into $root or $root/bin? With the latter, cargo install could later write things in $root/share or $root/lib, and there is precedent in ./configure scripts to name it prefix rather than root.

[alexcrichton]

Hm yes I do think that going into $root/bin is probably the best option for now to give ourselves some room to expand in the future like this. So long as it's predictable I don't think it's the end of the world.

[nagisa]

This is a pretty big downside.

[Gankro]

It's not a downside really. Just not included.

[alexcrichton]

@nagisa can you elaborate on your concern a bit? Do you think it's enough of a downside to require fixing now versus punting to a later date? I'm of the impression that most native dependencies built as part of the build process are statically linked, in which case this is not a problem. I'm unaware of any native dependency that builds a dynamic library as part of the build process.

Put another way, I expect this limitation to not actually affect many crates today, so I don't feel it's necessary to fix immediately.

[nagisa]

Now that you mention it, yes, most binaries will be linked fully statically. Except there will surely exist those few odd binaries that link to something dynamically and fail to run at all, no matter what.

Sure, that’s probably not something worth putting a lot of attention to for the first viable release of the functionality.

[nagisa]

I’m somewhat against this. Just because we already have a few awesome system package managers that make installing “foreign” packages relatively painless and if you/your system doesn’t use one of these, it should totally start doing so. That is, I believe cargo (and similar tools such as pip, cabal, etc) are not a place for such functionality.

[huonw]

So, the alternative is making sure those sort of package managers have (cross-platform...) support for Rust code as effortless as the cargo install proposed here? Do you know what that will take? (I have no idea.)

[steveklabnik]

Really big 👍 here, this is something I've wanted since day 1.

[nagisa]

So, the alternative is making sure those sort of package managers have (cross-platform...) support for Rust code as effortless as the cargo install proposed here?

I subscribe to the idea of package manager providing the necessary glue. That strategy has worked out spectacularly for nix with cabal2nix, for example. We also already provide a good tool (cargo build), which does a good enough job at building and putting built binaries in a predictable location.

History has also proven that any notable packages (e.g. xmonad, pandoc) will find their way into all/most package repositories (many less notable ones into user-maintained repositories too!), following all the distribution’s guidelines, very quickly. I don’t see how that won’t be a case with cargo packages or why this doesn’t cover, lets say, 95% of use cases (installing a few notable packages from cargo.io) cargo install is trying to solve.

---

I certainly agree that building packages for all the package managers or trying to interface with all of them is outrageous. On the other hand, I don’t see why we should bother introducing tools, which are essentially an alias for cargo build && find -executable -exec "mv {} $CARGO_HOME/bin/$PACKAGE/" $PREDICTABLE_LOCATION and rm -rf $CARGO_HOME/bin/$PACKAGE and ls $CARGO_HOME/bin, either.

[Gankro]

$CARGO_HOME

[Gankro]

is to

[Gankro]

Also the lack of any kind of stable ABI means compatibility headaches.

[alexcrichton]

Yeah that's when Cargo would have to recompile libraries if the Rust compiler changes (e.g. via multirust or via a normal update). It already does this for local build directories but I think it definitely reduces the usefulness of cargo install for libraries as well.

[nixpulvis]

Unifying multirust and cargo would help dramatically with this. I very often find myself wanting a number of libraries for one of Rust files.

[jdub]

It'd be cool if cargo install blah was a teeny-weeny bit magic.

• If blah is a URL, do whatever makes sense based on the URL. You've only mentioned git, so assume --git for now.
• If ./Cargo.toml exists, check if blah is a binary. If so, install that binary.
• If blah is a local filesystem path, assume --path.
• Otherwise, assume --package.

apt has a nice modifier for package versions which could be an attractive shorthand for the --vers parameter: apt-get install blah=0.1.0.

[jarcane]

Personally speaking, I'm not sure I see what the advantage of locally installing libraries like this is. By day I work in Clojure, and I've rather become fond instead of the behavior of leiningen and cargo of instead grabbing and caching libraries as needed at build time, adding a library being as simple as adding a line to the dependencies (with the added bonus that the config and lock files will ensure we stick to a consistent version of that library for future builds). Perhaps I'm missing something, but it just doesn't seem like something I'm likely to use much.

I do like the sound of being able to add cargo sub-commands more easily, and perhaps this might be a solution for things like SDL which depend on external non-rust libs being more easily packaged (as opposed to now, where to build mooneye-gb I had to download SDL myself and manually extract the right files to the right Rust dirs).

[withoutboats]

👍 This is a great way to make it easy to distribute Rust binaries. Large Rust projects will work their way into the package repositories of many Linux distros, but that isn't the only use case for cargo install. For a small side project, or especially for the project of new coders, enabling an easy way to distribute binaries that integrates with the build tools you're already using is a real great advantage. It becomes very easy for someone to - say - make a little game or tool, and then just put it on crates.io to show it off. If it catches on, it'll filter out into a wider distribution system, but cargo install is a great launchpad.

Also, while linux distros have package managers, Windows really does not. cargo install will work on all platforms supported by cargo.

[SimonSapin]

Gratuitous bikeshedding: "vers" sounds weird (and spells like "worm" in French). I’d prefer "ver" or "version".

[alexcrichton]

We currently have precedent for this with cargo yank --vers, and unfortunately the best word, version, is already taken with the cargo --version flag :(

[SimonSapin]

How about --precise, like in cargo update?

[alexcrichton]

Hm perhaps, but precise was actually added first to cargo update for the SHA of a git repository, but it may apply here as well!

[tshepang]

We can still have --version without a conflict right?

$ cargo --version
0.4.0
$ cargo install --package foo --version 0.2.1
....

[withoutboats]

Why does cargo install require a -p or --package instead of pulling from the registry by default (which I would expect to be the most common use)? Is there an argument parsing issue with this being cargo install [options] <crate> [--vers VERS]?

[alexcrichton]

I don't think there's an ambiguity, but it seems relatively more consistent with --git and --path to have a flag to specify where you're pulling the crate from.

[nagisa]

Personally speaking, I'm not sure I see what the advantage of locally installing libraries like this is.

It doesn’t even install libraries, nor does it install native dependencies either. Only binaries (i.e. things defined in [[bin]] section).

Also, while linux distros have package managers, Windows really does not.

Sorry to disappoint you! OneGet is shipping with Windows 10, and is probably better than apt or rpm in featuring a functionality to discover and install packages from language specific package managers (and there’s Cargo/Rust on the TODO list).

[cosmo0920]

👍 for me, because windows folks dose not have package manager until Windows 10.
This feature helps Windows folks.

[killercup]

I'm in favor of this. Writing a small, useful CLI tool in Rust is pretty easy, but getting it into the big package managers is a lot of work. cargo install would offer a much easier and (for developers) more convenient way.

[jarcane]

Writing a small, useful CLI tool in Rust is pretty easy, but getting it into the big package managers is a lot of work. cargo install would offer a much easier and (for developers) more convenient way.

This is a use I hadn't considered. I was thinking about that problem the other day with wf; if it wanted to actually package it for distribution anywhere, or even just installing it to the right places locally, I'd have to muck with make or apt, or fiddle with a lot of stuff manually.

If Cargo offered some installation configuration as part of the Cargo.toml or as cargo install, one could then handle it more agnostically, or at least without having to mess with external tools. Perhaps even with target-based conditionals, so I didn't have to assume *nix like is common with these kinds of tools. But perhaps that's outside the intent of this RFC.

[Gankro]

@nagisa while package managers enable easy (un)installation of major stable software, they fail to adequately address minor or very unstable software, in my opinion. Cargo enables us all to share our random unstable one-off libraries. It seems reasonable to extend that to programs.

[Gankro]

This also encourages everyone to upload all their Rust programs to crates.io, which gives us more robust coverage for Crater.

[alexcrichton]

@nagisa

I’m somewhat against this. Just because we already have a few awesome system package managers that make installing “foreign” packages relatively painless and if you/your system doesn’t use one of these, it should totally start doing so.

I tried to discuss this in the alternatives section (although I could likely expand more), but I feel like the key part of this sentence is "it should totally start doing so". Despite whatever we feel, there's no way these sorts of agnostic package managers will make their way onto all platforms overnight, and it's becoming clear over time that we need a solution to this problem sooner rather than later.

I subscribe to the idea of package manager providing the necessary glue.

I 100% agree with this, but I'm not sure it precludes the existence of cargo install. I very much want commands like cargo debian or cargo rpm or cargo pacman which will generate appropriate packages for each package manager. The great part about cargo install is that it allows development of these subcommands outside of the main Cargo repo in a very quick and iterative fashion. It'll take some time for these tools to exist, but I think they definitely can exist!

History has also proven that any notable packages (e.g. xmonad, pandoc) will find their way into all/most package repositories (many less notable ones into user-maintained repositories too!)

I also totally agree with this, and I expect many popular Rust packages will follow suit.

I don’t see how that won’t be a case with cargo packages or why this doesn’t cover, lets say, 95% of use cases (installing a few notable packages from cargo.io) cargo install is trying to solve.

The key part here is ease of installation. As a user of Cargo who wants to quickly get some new subcommands or some ubiquitous rust-specific tools, cargo install is the easiest way to do so. If I instead have to go to a package manager, then you need:

• Each package everywhere needs to list the commands needed to install it. As in, the exact command line to install a crate is very different across all systems.
• Authors of crates now need to be responsible and upload their crate to as many package systems as possible, and one or two are inevitably going to be forgotten.

cargo install provides a 0-cost method (e.g. the author just needs to publish to crates.io, that's it) to install these kinds of binaries for Rust users. It's not meant to be a general mechanism to manage software, it's just enhancing the experience for Rust developers.

---

@jdub

It'd be cool if cargo install blah was a teeny-weeny bit magic.

I would be a little uneasy about adding a whole bunch of magic because it often leads to ambiguities. For example if we are given a URL, is it a git repo? mercurial? SVN? tarball? Having to say --git seems not that high overhead for a case like this and it allows us to dispel any form of ambiguity.

Similarly having to say -p for crates.io or --path for a path doesn't seem that high overhead, and Cargo could certainly provide some nice error messages to help out. In general you're probably not firing off hundreds of cargo install commands a day, so making it the most-ergonomic-as-possible may not be the highest priority.

---

@jarcane

I'm not sure I see what the advantage of locally installing libraries like this is.

Ah just to clarify, cargo install as proposed in this RFC explicitly doesn't allow installing libraries. It only allows installing binaries.

[SimonSapin]

I’d like to have this eventually (a convention of where files go, cargo install copying them, and maybe some helper functions for programs to find them), but it doesn’t have to be in the first iteration of cargo install.

[nrc]

I'd be happier to see install scripts supported in the same manner as build scripts, but agree that it can wait for later iteration,

[SimonSapin]

Should there be a --target argument to override this, separately from where Cargo keeps its cache files?

[SimonSapin]

Sorry, I meant --prefix, like in some ./configure scripts.

[alexcrichton]

Ah yeah @nagisa already mentioned a concern about this, and I'm curious what you think about my suggestion?

[SimonSapin]

Where would the target directory (for intermediate build artifacts) be for install -p or install --git? Is it temporary and removed after install, or preserved to speed up future updates?

[withoutboats]

Sorry to disappoint you! OneGet is shipping with Windows 10

This doesn't disappoint me at all. Your tone here is a bit antagonistic.

---

I do not think it is wise to "magic"ally inferring the type of source from user input; what kind of source each type of input refers to is not static across different projects and different times, and its better to be explicit here.

[alexcrichton]

@SimonSapin

Where would the target directory (for intermediate build artifacts) be for install -p or install --git? Is it temporary and removed after install, or preserved to speed up future updates?

I expect the implementation would cache all source files as Cargo does today already (e.g. have a local database of git repos, crates.io, etc), but the artifacts would be built as part of a temporary directory.

[ruuda]

I too feel that this should not try to replace system package managers. Even though

It's not meant to be a general mechanism to manage software, it's just enhancing the experience for Rust developers.

it will likely get abused. Too often I encounter a program with an installation section reading

$ <obscure-package-manager> install <pacakge-name>

So now I first have to install yet another package manager, add its binary directory to my path, etc. And after a system-wide update, I have to run the update command for each of those package managers … Preparing system packages might have high overhead for developers, but non-system packages have high overhead for users. On the other hand, one cannot expect the developer to maintain a package for every system package manager out there either and it is hard to get into official repositories, so I don’t know whether this problem can be solved at all.

[Stebalien]

How about adding a --no-manifest flag (for package build scripts)?

Sounds reasonable!

“Manifest” in this context refers to a list of installed files, right? It kinda conflicts with cargo build --manifest-path which refers to a Cargo.toml file :/

--no-metadata, --without-metadata, --unmanaged, ...

[wycats]

I don't see what we gain by having an install and an uninstall, but not an update (which would be semantically equivalent to an uninstall followed by an install).

In the absence of a strong argument not to provide a convenience for this common sequence, I would like to include it.

[alexcrichton]

The tools team has decided to place this RFC into its week-long final comment period.

[withoutboats]

It would be great if this had a --source flag that downloaded the source to the destination directory without compiling it (to be used for generating docs or reading the source of crates without online docs or a repository).

[gsingh93]

@withoutboats I'd like that feature as well, but I think it should apply to any type of crate (binary or library), and should be proposed separately.

[wycats]

I agree with @gsingh93

[eternaleye]

Personally, I feel that'd be better as a separate "cargo sources" subcommand or similar.

[withoutboats]

I'll look into writing something up.

[gsingh93]

I made a separate issue for the "cargo sources" command: rust-lang/cargo#1861

[RAOF]

@eternaleye

I'm actually not sure cargo install --list is even a good idea, and depending on how it's implemented
it could be a hazard to system PMs.

Properly designed, I rather think this is actually a prerequisite for good system PM integration - specifically, we'll want the system PM to be able to install rust libraries in such a way that cargo build knows how to use them for dependency resolution. This suggests that cargo will need to (a) store the relevant metadata and (b) will need to be able to consult a system-wide store of such metadata. Having the same metadata for binaries would be beneficial; after all, binaries can depend on other binaries.

Anyway, I'm glad that not only are other people thinking about distro-integration topics, they're being more active about it than I am 😄

[eternaleye]

Sure - mainly, my concerns revolve around the pitfalls it not being properly designed. Using a single central database that gets modified in-place, putting it somewhere contrary to FS guidelines, choosing a naming system that results in collisions, etc.

If it winds up being more like pkg-config, I entirely agree - though for Rust, I think it'd need an additional level of hierarchy, such as /usr/lib/crates/<cratename>/<hash>.cratemeta, with the cratemeta containing not just the file list (as cargo uninstall would need), but also the configuration map.

In addition, if more than one installed crate would satisfy a dependency, it's crucial that the PM is able to control which is used - on a source distro like Exherbo, for example, failing to ensure that could result in a library being spuriously treated as unused, breaking stuff up the dependency tree (in the case of dynamic linking).

[alexcrichton]

The conclusion of the libs team was to accept this RFC, so I shall merge.

[alexcrichton]

Er, tools team*

[mikeyhew]

I guess I'm late to the party, but the problem I see with cargo install is that, coming from node.js, cargo looks more like npm (a package manager) than it does Make, and cargo install <crate> would intuitively mean "download that crate", cargo install <crate> would download the crate and add it to your dependency list, and cargo install would download all of the dependency crates.

Since it's already been decided that cargo install should be similar to make install, I guess my only point is that you should make sure it's clear in the documentation, for people who view cargo as a package manager, that cargo install doesn't install crates, but instead installs the compiled binary.

[withoutboats]

@mikeyhew because cargo performs static linking and Rust requires explicit declaration of linked libraries in the code (the extern crate item), there isn't a concept analogous to installing a library as in npm or pip. It is unfortunate that a new user confused about how Rust's module system works might accidentally install a binary.

The most analogous command to npm install is the cargo add extension, which adds crate dependencies to the local Cargo.toml file.

[nixpulvis]

To make the goal of this command a bit more clear I'd update the description,

Install a crate onto the local system

to something illustrating that it's just compiling a binary and putting it in a bindir. Maybe something like

Install a crate's compiled binary on the local system

I know it explains more below, but this would make it clear from the get go. If this command ever does system wide library installation we could change the wording then.

[nixpulvis]

@withoutboats @alexcrichton

Why does cargo install require a -p or --package instead of pulling from the registry by default [...]

I also feel like this is cumbersome. People coming from other tools like gem or pip will expect crate install <package> to hit crates.io, this is also likely the most common use case, so avoiding a flag will help remembering the command.

[wycats]

I agree that we should allow leaving off the -p for crates.io packages.

[nixpulvis]

It might be too late for this, but I had an idea as I was reading through this. Since this command is really not installing a crate, it's just dealing with binaries, at least for now. And the details of dealing with binaries are different from the details of libraries, why not make the command cargo binaries or something like that.

This also might help with the currently odd cargo install --list option, which doesn't do any installing. Instead we'd have

cargo binaries list
cargo binaries install
cargo binaries uninstall

Just a thought. Another option for making the list command a bit better might be to have another subcommand cargo list.

[ticki]

@ruud-v-a, I agree. Whether you like it or not, people will use this as a replacement for another package manager.

I would rather have a way to generate the package for the specific package manager (such as homebrew, aptitude, pacman, rpm, and so on), as @alexcrichton mentions. Maybe using a subcommand like cargo make [package manager].

[hunterboerner]

I would rather have a way to generate the package for the specific package manager (such as homebrew, aptitude, pacman, rpm, and so on), as @alexcrichton mentions. Maybe using a subcommand like cargo make [package manager].

This would be interesting to see. It would eliminate the need for users to install cargo/manually install binaries.

[vyp]

@hunterboerner similarly, could also adopt nix/guix.

[Gankro]

I am of the belief that generating distributable packages with all the metadata and icons and whatnot is better handled by a tool on top of Cargo, and not Cargo itself.

[alexcrichton]

For those interested, I've now posted an implementation of cargo install. A few minor changes were made from this RFC, detailed in the PR description.