Tracking issue for RFC 1977: public & private dependencies

[withoutboats]

This is the tracking issue for rust-lang/rfcs#1977 - public and private dependencies in cargo.

Blocking stabilization:

• Determining the impact & migration path
• Implementation
  • Command to update Cargo.lock to minimal versions
  • Make cargo publish use the minimal versions allowed by Cargo.toml
• Documentation

[Boscop]

Maybe it's useful to look at how this problem is approached in Haskell.
Here are some relevant slides: https://wiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdf

Here is an example of a subtle issue that occurs with private dependencies:

Btw, I found it through this post: http://harry.garrood.me/blog/purescript-why-bower/

[Eh2406]

Would Rust2018 be a good opportunity to make the braking changes to Cargo.toml?

[SimonSapin]

Maybe. Though such changes should be designed and proposed soon, have an easy migration path, and preferably only affect a minority of existing crates.

[Nemo157]

Thought I just had related to ecosystems like futures that provide a "core" stable crate along with less stable utilities re-exported from a "facade": items from a public dependency re-exported through a private dependency should be treated as public. (This is likely how the implementation would behave anyway, but having a testcase for this would be nice). Basically how I see using a tightly coupled ecosystem like that work is that you would declare both the "facade" crate and "core" crate as dependencies, but only the "core" crate as public:

[dependencies]
futures = { version = "10.17", public = false }
futures-core = { version = "1.2", public = true }

But then, in code you would never import items from the "core" crate, instead you would simply have public APIs that return types which have been re-exported through the "facade" from the "core" crate and rely on the compiler to warn you if you accidentally use something that didn't originate in the "core" crate.

[Eh2406]

I don't quite understand something about your example. Can you elaborate on what the Toml and the code in the futures crate look like?

[Nemo157]

@Eh2406 the setup's a little annoying to just explain, so here's a full compiling set of example crates: https://github.com/Nemo157/rust-44663-facade-example

In this example futures-core exports a single trait Future, futures re-exports this trait from futures-core along with a "utility" Foo.

Now mylib wants to use futures, including returning some Futures from its public API. But futures itself intends to have breaking changes to update utilities, users that care about interoperability should only use types from futures-core in their public APIs. Even as new major versions of futures are released they will all link against the same version of futures-core so the underlying traits will be interoperable.

So, to have the compiler enforce this restriction the developers of mylib want to use the public/private dependencies feature, they have both futures and futures-core as dependency but only futures-core marked as public. But for ergonomics it's nicer to just use the re-exports directly from futures, instead of having to remember which parts of the API need to be imported from futures and which from futures-core.

[Aaron1011]

I'd like to work on this.

[Eh2406]

Thanks for offering to work on this! @alexcrichton, do you have any mentoring advice?

I can at any time get a branch ready for the resolver part of cargo, it will be good enough to start experimentation, although not good enough to be ready to stabilize.

[alexcrichton]

Sure! @Aaron1011 so this is split roughly into two features, one being rustc knowing what crates are in the "public interface" and a second being the support in Cargo's resolver to make various decisions differently. Are you interested in one of those in particular?

[Aaron1011]

@alexcrichton: I've started working on the rustc part in a branch: https://github.com/Aaron1011/rust/commits/feature/pub-priv-dep, and I plan to submit a (WIP) PR soon. I might also be interested in working on the cargo part as well.

[Eh2406]

@Aaron1011 please cc me when you make that PR. I know nothing about the internals of Rust, so will be of no help, but I do want to follow the discussion.

I will redouble my efforts to get the Cargo's resolver part mergeable. I have been enjoying ( and hating ) it because it is the hardest algorithmic problem I have ever worked on. I would be happy for some help, perhaps a new perspective will get me un-stuck.

[alexcrichton]

@Aaron1011 ok and it seems like you're off to a great start! I'd be up for reviewing that PR, and if you've got any questions along the way just let me know

[Nemo157]

There was another usecase for this mentioned on i.rl.o, if cargo doc had a form of "local development" mode to build the documentation you care about when working on a crate, this would allow that mode to know exactly which dependencies it needs to produce the documentation for (all direct dependencies + their public dependencies).

Similarly, if you're self-hosting documentation for a crate somewhere you may want to be able to generate a documentation bundle including the crate and all its public dependencies so that it is fully self-contained.