Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve multi-crate project management #2122

Closed
alexcrichton opened this issue Nov 6, 2015 · 13 comments

Comments

Projects
None yet
5 participants
@alexcrichton
Copy link
Member

commented Nov 6, 2015

Cargo's story around having a project (or a repo) with many crates isn't the best right now, there are a number of papercuts that add up over time:

  • Each crate gets its own Cargo.lock, which means that each crate can be working with a different set of dependencies. Not only is it difficult to manage all these lockfiles but it's often not desired to have different sets of dependencies when working in the source tree.
  • By default crates have distinct target folders, meaning crates are recompiled as you move around the project tree.
  • Currently you can't run cargo test -p foo for a crate that has a dev-dependency #860
  • "test everything" isn't an easily accessible command

I'll likely add more over time!

@icorderi

This comment has been minimized.

Copy link

commented Feb 18, 2016

I keep running into needing some project level cargo for large projects.

I was thinking if something like this would help:

[project]
name = "theproject"
version = "0.1.0"
authors = ["people"]
readme = "README.md"
description = "..."

[crates]
onecrate = "0.1"
another = { git = "https://github.com/some/dude" }
relative = { path="./some/other" } # weird, but might happen

# feature crates
uhg = { optional = true }

[features]
nightly = ["uhg"]

Having the special [project] instead of [package] root, tells cargo to deal with this .toml differently.

  1. There are no [lib] or [bin]/[[bin]] entries
  2. cargo build would output target/debug/<crate>/...
  3. cargo test would test each crate (we could also check for a root level folder test for cross crate tests)
  4. cargo run with that .toml as-is would make no sense, but having an entry crate would be helpful for some kinds of projects. Such projects can add entry = onecrate to the project definition.
  5. What does cargo publish mean? (a) Recursive (explicit, with publish=true on project, requires all crates to be publishable too), (b) individual crates (implicit+default, except when publish=false on subcrate), (c) not allowed
  6. other subtleties...

We can also have a common crate cache, if a bunch of the project crates share the same dependencies there is no need compile them all over and over again.

@alexcrichton what are your thoughts on this? Could we get some traction to get "project" support on cargo?

cc: @jsgf

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Feb 20, 2016

Thanks for the comment @icorderi! I've been thinking about this recently and I hope to soon formulate what I'm thinking into an RFC. I suspect that this will basically manifest itself in somehow linking together Cargo.toml files into one common "parent".

To me most commands would continue to basically just work "as-is" today without many changes. Grouping projects together would primarily:

  • Share one output directory (dependencies aren't recompiled)
  • Share one Cargo.lock reducing divergence across directories and alleviating registry updates
  • Fixing testing transitive path dependencies by storing more information in the lock file

I think that some operations that are out of scope in the short term are:

  • cargo publish --all
  • cargo test --all
  • cargo bench --all
  • etc

Basically operations that work over an entire tree probably want to be held off for now. Perhaps one day we can add such support but for now it may be premature and just bog down the design.

@icorderi

This comment has been minimized.

Copy link

commented Feb 24, 2016

@alexcrichton looking forward to that RFC.

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Feb 26, 2016

Ok, I've got a draft now, yay! @icorderi curious as to your thoughts on -- https://github.com/alexcrichton/rfcs/blob/cargo-workspace/text/0000-cargo-workspace.md?

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Feb 26, 2016

er, wrong RFC, fixed the link

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Mar 3, 2016

I've now opened an RFC for this issue.

@alexbool

This comment has been minimized.

Copy link

commented Jun 15, 2016

Any good news? :)

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jun 16, 2016

Ah oops, I forgot to link the implementation PR! I have an implementation here: #2759

@alexbool

This comment has been minimized.

Copy link

commented Jun 16, 2016

Great!

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2016

Done! Workspaces landed

@matthiasbeyer

This comment has been minimized.

Copy link

commented Jul 21, 2016

@alexcrichton That's so awesome! Can't wait to try them out in my project!

@alexbool

This comment has been minimized.

Copy link

commented Jul 21, 2016

@alexcrichton --all commands will go as a seprate issue? If so, could you create them?

@alexcrichton

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2016

@alexbool indeed!

#2878

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.