New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Private dependencies #4035

Open
ezyang opened this Issue Oct 26, 2016 · 4 comments

Comments

Projects
None yet
2 participants
@ezyang
Contributor

ezyang commented Oct 26, 2016

CC @Ericson2314

I realized we didn't have a tracking ticket for private dependencies, so here it is.

Prior art by @kosmikus : https://wiki.haskell.org/HaskellImplementorsWorkshop/2011/Loeh (when the modular solver was originally introduced; slides have discussion of some tricky cases.)

Newest WIP proposal from @Ericson2314 https://github.com/Ericson2314/ghc-proposals/blob/private-deps/proposals/0000-private-dependencies.rst

I have some comments for @Ericson2314:

  • Notation definitions would be helpful, esp. IPub* (I assume * means transitive closure), r IPub (cast relation into set)
  • "As today, a valid Cabal build plan must never include multiple versions of the same package in the set of public dependencies." Strictly speaking this is true but we do, today, have qualified goals which, e.g., let setup-depends and build-tools be solved separately from the public dependencies.
  • Does your proposal handle the "encapsulation" problem described in the "Encapsulation example" slide in @kosmikus's slides? The situation is that if A depends on B and C, and C privately depends on B, if C adds a public dependency on D which depends on B, the encapsulation must be lost.
  • If freshening is done purely by allocating a new unit id to encapsulated dependencies, that would certainly solve the problem. You might be able to set up the requirements for private dependencies as, "if a package has a private dependency on some package, that package gets fresh identifiers for everything which don't match any other occurrences anywhere else." So maybe @kosmikus was wrong and we shouldn't stop encapsulating even if there is a public dependency that indirectly depends on the private package: they're just different types.
@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Oct 27, 2016

Collaborator

1, 2: I'll clarify thanks.

3, 4: is the encapsulation problem a problem? It is true that via type classes downstream can rely on types being _un_equal, but an easy solution is just ban reexports after all.

I wavered on this front, but I think not doing what you say for 4 is better. Consider if B has a public dep on Q, and the new dep D depends on Q instead of B. Then you have the same problem but one level deeper---Q from B->C->Q was unconstrained until C->D->Q makes Q a public dep. If you recursively always freshen, multiple "copies" of a package being in scope being fine, you end up endless bases. And that's no good.

Maybe packages get two types of private deps ("this dep should never unify with anything" and "unify with my personal public deps")?

Collaborator

Ericson2314 commented Oct 27, 2016

1, 2: I'll clarify thanks.

3, 4: is the encapsulation problem a problem? It is true that via type classes downstream can rely on types being _un_equal, but an easy solution is just ban reexports after all.

I wavered on this front, but I think not doing what you say for 4 is better. Consider if B has a public dep on Q, and the new dep D depends on Q instead of B. Then you have the same problem but one level deeper---Q from B->C->Q was unconstrained until C->D->Q makes Q a public dep. If you recursively always freshen, multiple "copies" of a package being in scope being fine, you end up endless bases. And that's no good.

Maybe packages get two types of private deps ("this dep should never unify with anything" and "unify with my personal public deps")?

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Oct 28, 2016

Contributor

Maybe "private dependencies" is the wrong way to go about thinking about this. What we want is a form of sealing, where we declare a dependency p to be sealed in some package, and this says, "If you depend on this package, it will not contribute any global constraints on p."

So let's take @kosmikus's encapsulation example. In this case, it is not that C has a private dependency on B, but that C wants to seal B. If C depends on D, which in turn depends on B, we cannot seal B without also sealing D (since any install plan that mentions D must also mention B). This avoids us from having to freshen infinitely, because we only freshen the reverse dependency closure of sealed packages up to our current one.

The downside of this approach is that if you seal a dependency, you might end up sealing some other dependencies that you wanted to be left unsealed, because those in turn depend on the sealed dependency.

Is this an improvement on your "we want the public dependencies of immediate private dependencies to agree"? I still admit to not fully understanding your formula.

Contributor

ezyang commented Oct 28, 2016

Maybe "private dependencies" is the wrong way to go about thinking about this. What we want is a form of sealing, where we declare a dependency p to be sealed in some package, and this says, "If you depend on this package, it will not contribute any global constraints on p."

So let's take @kosmikus's encapsulation example. In this case, it is not that C has a private dependency on B, but that C wants to seal B. If C depends on D, which in turn depends on B, we cannot seal B without also sealing D (since any install plan that mentions D must also mention B). This avoids us from having to freshen infinitely, because we only freshen the reverse dependency closure of sealed packages up to our current one.

The downside of this approach is that if you seal a dependency, you might end up sealing some other dependencies that you wanted to be left unsealed, because those in turn depend on the sealed dependency.

Is this an improvement on your "we want the public dependencies of immediate private dependencies to agree"? I still admit to not fully understanding your formula.

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Oct 28, 2016

Collaborator

The downside of this approach is that if you seal a dependency, you might end up sealing some other dependencies that you wanted to be left unsealed, because those in turn depend on the sealed dependency.

I don't think this is sound? D->B, C->D, and A->C are all public edges, and thus types can be leaked. C can compatibly use D's types (compatible with when it didn't use D at all) in its public interface provided its used in only in new functionality.

I do like the sealing language--I'm trying to reconcile it with the above.

Collaborator

Ericson2314 commented Oct 28, 2016

The downside of this approach is that if you seal a dependency, you might end up sealing some other dependencies that you wanted to be left unsealed, because those in turn depend on the sealed dependency.

I don't think this is sound? D->B, C->D, and A->C are all public edges, and thus types can be leaked. C can compatibly use D's types (compatible with when it didn't use D at all) in its public interface provided its used in only in new functionality.

I do like the sealing language--I'm trying to reconcile it with the above.

@ezyang

This comment has been minimized.

Show comment
Hide comment
@ezyang

ezyang Oct 28, 2016

Contributor

It's not unsound; I'm simply saying that if you wanted to use D "privately", you MUST also use C privately (if C depends on D.) I think this makes sense actually, because in @kosmikus's proposal, if C did depend on D that resulted in D becoming public again. That seems very unexpected; the private dep is effectively ignored.

Contributor

ezyang commented Oct 28, 2016

It's not unsound; I'm simply saying that if you wanted to use D "privately", you MUST also use C privately (if C depends on D.) I think this makes sense actually, because in @kosmikus's proposal, if C did depend on D that resulted in D becoming public again. That seems very unexpected; the private dep is effectively ignored.

@ezyang ezyang added this to the milestone Oct 29, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment