Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Add cargo install #1200

Merged
merged 5 commits into from
Aug 14, 2015
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
271 changes: 271 additions & 0 deletions text/0000-cargo-install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,271 @@
- Feature Name: N/A
- Start Date: 2015-07-10
- RFC PR: (leave this empty)
- Rust Issue: (leave this empty)

# Summary

Add a new subcommand to Cargo, `install`, which will install `[[bin]]`-based
packages onto the local system in a Cargo-specific directory.

# Motivation

There has [almost always been a desire][cargo-37] to be able to install Cargo
packages locally, but it's been somewhat unclear over time what the precise
meaning of this is. Now that we have crates.io and lots of experience with
Cargo, however, the niche that `cargo install` would fill is much clearer.

[cargo-37]: https://github.com/rust-lang/cargo/issues/37

Fundamentally, however, Cargo is a ubiquitous tool among the Rust community and
implementing `cargo install` would facilitate sharing Rust code among its
developers. Simple tasks like installing a new cargo subcommand, installing an
editor plugin, etc, would be just a `cargo install` away. Cargo can manage
dependencies, versions, updates, etc, itself to make the process as seamless as
possible.

Put another way, enabling easily sharing code is one of Cargo's fundamental
design goals, and expanding into binaries is simply an extension of Cargo's core
functionality.

# Detailed design

The following new subcommand will be added to Cargo:

```
Install a crate onto the local system

Installing new crates:
cargo install [options]
cargo install [options] [-p CRATE | --package CRATE] [--vers VERS]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gratuitous bikeshedding: "vers" sounds weird (and spells like "worm" in French). I’d prefer "ver" or "version".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently have precedent for this with cargo yank --vers, and unfortunately the best word, version, is already taken with the cargo --version flag :(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about --precise, like in cargo update?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm perhaps, but precise was actually added first to cargo update for the SHA of a git repository, but it may apply here as well!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can still have --version without a conflict right?

$ cargo --version
0.4.0
$ cargo install --package foo --version 0.2.1
....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does cargo install require a -p or --package instead of pulling from the registry by default (which I would expect to be the most common use)? Is there an argument parsing issue with this being cargo install [options] <crate> [--vers VERS]?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's an ambiguity, but it seems relatively more consistent with --git and --path to have a flag to specify where you're pulling the crate from.

cargo install [options] --git URL [--branch BRANCH | --tag TAG | --rev SHA]
cargo install [options] --path PATH

Managing installed crates:
cargo install [options] --list
cargo install [options] --update [SPEC | --all]

Options:
-h, --help Print this message
-j N, --jobs N The number of jobs to run in parallel
--features FEATURES Space-separated list of features to activate
--no-default-features Do not build the `default` feature
--debug Build in debug mode instead of release mode
--bin NAME Only install the binary NAME
--example EXAMPLE Install the example EXAMPLE instead of binaries
-p, --package CRATE Install this crate from crates.io or select the
package in a repository/path to install.
-v, --verbose Use verbose output

This command manages Cargo's local set of install binary crates. Only packages
which have [[bin]] targets can be installed, and all binaries are installed into
`$HOME/.cargo/bin` by default (or `$CARGO_HOME/bin` if you change the home
directory).

There are multiple methods of installing a new crate onto the system. The
`cargo install` command with no arguments will install the current crate (as
specifed by the current directory). Otherwise the `-p`, `--package`, `--git`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current directory thing is quite a neat default

and `--path` options all specify the source from which a crate is being
installed. The `-p` and `--package` options will download crates from crates.io.

Crates from crates.io can optionally specify the version they wish to install
via the `--vers` flags, and similarly packages from git repositories can
optionally specify the branch, tag, or revision that should be installed. If a
crate has multiple binaries, the `--bin` argument can selectively install only
one of them, and if you'd rather install examples the `--example` argument can
be used as well.

The `--list` option will list all installed packages (and their versions). The
`--update` option will update either the crate specified or all installed
crates.
```

## Installing Crates

Cargo attempts to be as flexible as possible in terms of installing crates from
various locations and specifying what should be installed. All binaries will be
stored in the **cargo-local** directory `CARGO_HOME/bin`. This is typically
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$CARGO_HOME

`$HOME/.cargo/bin` but the home directory can be modified via the `$CARGO_HOME`
environment variable.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally needs a $CARGO_INSTALL_HOME which overrides everything, because nobody wants to get binaries stored in their $XDG_CONFIG_HOME to which their $CARGO_HOME is very likely to be set to.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes I had forgotten about my intention to do this. I'm thinking something along the lines of:

  • --root is an argument to the install and uninstall subcommands to specify the root manually.
  • The environment variable CARGO_INSTALL_ROOT can specify the root
  • The configuration key install.root can specify the root
  • Finally, $CARGO_HOME/bin is used.

How does that sound?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would binaries go into $root or $root/bin? With the latter, cargo install could later write things in $root/share or $root/lib, and there is precedent in ./configure scripts to name it prefix rather than root.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm yes I do think that going into $root/bin is probably the best option for now to give ourselves some room to expand in the future like this. So long as it's predictable I don't think it's the end of the world.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a --target argument to override this, separately from where Cargo keeps its cache files?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I meant --prefix, like in some ./configure scripts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah @nagisa already mentioned a concern about this, and I'm curious what you think about my suggestion?


Cargo will not attempt to install binaries or crates into system directories
(e.g. `/usr`) as that responsibility is intended for system package managers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This exactly is what /usr/local is for.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll also note the per-user analogue, ${HOME}/.local - pip install --user makes use of this, as do others.


To use installed crates one just needs to add the binary path to their `PATH`
environment variable. This will be recommended when `cargo install` is run if
`PATH` does not already look like it's configured.

#### Crate Sources

The `cargo install` command will be able to install crates from any source that
Cargo already understands. For example it will start off being able to install
from crates.io, git repositories, and local paths. Like with normal
dependencies, downloads from crates.io can specify a version, git repositories
can specify branches, tags, or revisions.

#### Sources with multiple crates

Sources like git repositories and paths can have multiple crates inside them,
and Cargo needs a way to figure out which one is being installed. If there is
more than one crate in a repo (or path), then Cargo will apply the following
heuristics to select a crate, in order:

1. If the `-p` argument is specified, use that crate.
2. If only one crate has binaries, use that crate.
3. If only one crate has examples, use that crate.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would skip step 3, personally, seems a bit oblique. I hope having multiple crates with binaries is a rare-ish edge case and we could live with requiring -p there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this because I figured it'd be a good idea to add --example as an option to cargo install, but it's definitely non-critical and I'd be fine removing this as well.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like the feature to install examples. Various GUI libraries like GTK install a demo binary when they're installed, I'd like to have this same option when installing a package through cargo.

4. Print an error suggesting the `-p` flag.

#### Multiple binaries in a crate

Once a crate has been selected, Cargo will by default build all binaries and
install them. This behavior can be modified with the `--bin` or `--example`
flags to configure what's installed on the local system.

#### Building a Binary

The `cargo install` command has some standard build options found on `cargo
build` and friends, but a key difference is that `--release` is the default for
installed binaries so a `--debug` flag is present to switch this back to
debug-mode. Otherwise the `--features` flag can be specified to activate various
features of the crate being installed.

The `--target` option is omitted as `cargo install` is not intended for creating
cross-compiled binaries to ship to other platforms.

#### Conflicting Crates

Cargo will not namespace the installation directory for crates, so conflicts may
arise in terms of binary names. For example if crates A and B both provide a
binary called `foo` they cannot be both installed at once. Cargo will reject
these situations and recommend that a binary is selected via `--bin` or the
conflicting crate is uninstalled.

## Managing Installations

If Cargo gives access to installing packages, it should surely provide the
ability to manage what's installed! The first part of this is just discovering
what's installed, and this is provided via `cargo install --list`. A more
interesting aspect is the `cargo install --update` command.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, cargo shouldn't be a package manager.


#### Updating Crates

Once a crate is installed new versions can be released or perhaps the build
configuration wants to be tweaked, so Cargo will provide the ability to update
crates in-place. By default *something* needs to be specified to the `--update`
flag, either a specific crate that's been installed or the `--all` flag to
update all crates. Because multiple crates of the same name can come from
different sources, the argument to the `--update` flag will be a package id
specification instead of just the name of a crate.

When updating a crate, it will first attempt to update the source code for the
crate. For crates.io sources this means that it will download the most recent
version. For git sources it means the git repo will be updated, but the same
branch/tag will be used (if original specified when installed). Git sources
installed via `--rev` won't be updated.

After the source code has been updated, the crate will be rebuilt according to
the flags specified on the command line. This will override the flags that were
previously used to install a crate, for example activated features are not
remembered.

#### Removing Crates

To remove an installed crate, another subcommand will be added to Cargo:

```
Remove a locally installed crate

Usage:
cargo uninstall [options] SPEC
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I much prefer remove... faster to type and feels less awkward.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

install/uninstall is easier to remember than install/remove for me


Options:
-h, --help Print this message
--bin NAME Only uninstall the binary NAME
--example EXAMPLE Only uninstall the example EXAMPLE
-v, --verbose Use verbose output

The argument SPEC is a package id specification (see `cargo help pkgid`) to
specify which crate should be uninstalled. By default all binaries are
uninstalled for a crate but the `--bin` and `--example` flags can be used to
only uninstall particular binaries.
```

Cargo won't remove the source for uninstalled crates, just the binaries that
were installed by Cargo itself.

## Non-binary artifacts

Cargo will not currently attempt to manage anything other than a binary artifact
of `cargo build`. For example the following items will not be available to
installed crates:

* Dynamic native libraries built as part of `cargo build`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty big downside.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a downside really. Just not included.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nagisa can you elaborate on your concern a bit? Do you think it's enough of a downside to require fixing now versus punting to a later date? I'm of the impression that most native dependencies built as part of the build process are statically linked, in which case this is not a problem. I'm unaware of any native dependency that builds a dynamic library as part of the build process.

Put another way, I expect this limitation to not actually affect many crates today, so I don't feel it's necessary to fix immediately.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that you mention it, yes, most binaries will be linked fully statically. Except there will surely exist those few odd binaries that link to something dynamically and fail to run at all, no matter what.

Sure, that’s probably not something worth putting a lot of attention to for the first viable release of the functionality.

* Native assets such as images not included in the binary itself.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d like to have this eventually (a convention of where files go, cargo install copying them, and maybe some helper functions for programs to find them), but it doesn’t have to be in the first iteration of cargo install.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be happier to see install scripts supported in the same manner as build scripts, but agree that it can wait for later iteration,

* The source code is not guaranteed to exist, and the binary doesn't know where
the source code is.

Additionally, Cargo will not immediately provide the ability to configure the
installation stage of a package. There is often a desire for a "pre-install
script" which runs various house-cleaning tasks. This is left as a future
extension to Cargo.

# Drawbacks

Beyond the standard "this is more surface area" and "this may want to
aggressively include more features initially" concerns there are no known
drawbacks at this time.

# Alternatives

### System Package Managers

The primary alternative to putting effort behind `cargo install` it to instead
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is to

put effort behind system-specific package managers. For example the line between
a system package manager and `cargo install` is a little blurry, and the
"official" way to distribute a package should in theory be through a system
package manager. This also has the upside of benefiting those outside the Rust
community as you don't have to have Cargo installed to manage a program. This
approach is not without its downsides, however:

* There are *many* system package managers, and it's unclear how much effort it
would be for Cargo to support building packages for all of them.
* Actually preparing a package for being packaged in a system package manager
can be quite onerous and is often associated with a high amount of overhead.
* Even once a system package is created, it must be added to an online
repository in one form or another which is often different for each
distribution.

All in all, even if Cargo invested effort in facilitating creation of system
packages, **the threshold for distribution a Rust program is still too high**.
If everything went according to plan it's just unfortunately inherently complex
to only distribute packages through a system package manager because of the
various requirements and how diverse they are. The `cargo install` command
provides a cross-platform, easy-to-use, if Rust-specific interface to installing
binaries.

It is expected that all major Rust projects will still invest effort into
distribution through standard package managers, and Cargo will certainly have
room to help out with this, but it doesn't obsolete the need for
`cargo install`.

### Installing Libraries

Another possibility for `cargo install` is to not only be able to install
binaries, but also libraries. The meaning of this however, is pretty nebulous
and it's not clear that it's worthwhile. For example all Cargo builds will not
have access to these libraries (as Cargo retains control over dependencies). It
may mean that normal invocations of `rustc` have access to these libraries (e.g.
for small one-off scripts), but it's not clear that this is worthwhile enough to
support installing libraries yet.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the lack of any kind of stable ABI means compatibility headaches.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's when Cargo would have to recompile libraries if the Rust compiler changes (e.g. via multirust or via a normal update). It already does this for local build directories but I think it definitely reduces the usefulness of cargo install for libraries as well.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unifying multirust and cargo would help dramatically with this. I very often find myself wanting a number of libraries for one of Rust files.


Another possible interpretation of installing libraries is that a developer is
informing Cargo that the library should be available in a pre-compiled form. If
any compile ends up using the library, then it can use the precompiled form
instead of recompiling it. This job, however, seems best left to `cargo build`
as it will automatically handle when the compiler version changes, for example.
It may also be more appropriate to add the caching layer at the `cargo build`
layer instead of `cargo install`.

# Unresolved questions

None yet