Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Purely functional metadata-rich derivation poisoning #1052

Closed
shlevy opened this Issue Sep 2, 2016 · 11 comments

Comments

Projects
None yet
3 participants
@shlevy
Copy link
Member

shlevy commented Sep 2, 2016

Problem: We want broken/unfree/etc. packages (and their dependents) to refuse to evaluate and build without specific user request and to have an unpolluted hydra eval errors tab. Exception throwing does anything more structured than "yes this evaluates"/"no this doesn't" violates the purely functional nature of nix: The order of evaluate affects the result that a function catching an exception will see.

Proposed solution: Add a "poisoned derivation" type, which infects any derivations that depend on it in a non-eval-order dependent way and which can be introspected for arbitrary metadata but which by default simply causes an exception to be thrown.

Example: Package foo is marked broken, package bar depends on foo. nix-build -A bar throws "foo is marked broken, add allowBroken to try anyway" as soon as foo is evaluated, whereas:

let
  maybe-poisoned = builtins.diagnosePoison pkgs.bar;
in if maybe-poisoned.poisoned && lib.only-broken-or-unfree maybe-poisoned.poisons
  then { /* some set which doesn't reference bar */}
  else { /* some set which does reference bar (via maybe-posoned.value), which either evaluates fine or throws if bar is poisoned for some reason besides being broken or unfree */ }`

will evaluate to the first set, unless bar is also poisoned for some other reason we don't want to ignore. The specifics of the poisons attribute I haven't yet worked out, but can be made evaluation-order independent.

Alternative: Instead of the diagnosePoison builtin, we can just have different eval modes, where nix-build says "throw on poison" and hydra says "don't throw on poison" and poisoned drvs can be queried directly for poison status.

@edolstra I will implement this if it is an acceptable solution.

See also #1000 and NixOS/nixpkgs#7830

@shlevy

This comment has been minimized.

Copy link
Member Author

shlevy commented Sep 2, 2016

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Sep 2, 2016

This is exactly what I've wanted!!!!! Honestly I'd even go with no exception at all, as relying on the metadata of broken packages doesn't seem that bad. (Maybe I'm naive here.)

@shlevy

This comment has been minimized.

Copy link
Member Author

shlevy commented Sep 2, 2016

@Ericson2314 So what would nix-build -A bar do? inspect the top-level drv for poison in C++? I guess that works but it does end up doing more work than needed.

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Sep 2, 2016

Just work, I was thinking.

@shlevy

This comment has been minimized.

Copy link
Member Author

shlevy commented Sep 2, 2016

The whole point is we don't want to build broken packages by default

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Sep 2, 2016

Err I misread the example and thought bar always only depended on the metadata of foo.

@copumpkin

This comment has been minimized.

Copy link
Member

copumpkin commented Sep 6, 2016

First off, general 👍 on this direction.

As I was trying to think this through, I thought about how I might model dependencies in Haskell: I'd probably make a simple Applicative that let me depend on things, along the lines of this post. I could then reconstruct your notion of "poison" by enumerating the dependencies and accumulating poison up the dependency tree.

That then got me wondering if a simpler solution would be to just expose derivation dependencies through some sort of magic meta attribute or builtin. If I can enumerate which derivations someone depends on (not just buildInputs, but all the random crap people use in string contexts and the like) then I can reconstruct your notion of poison in Nix itself.

Does that sound more difficult to implement than poison? Not sure if I'm missing something that'll kill my proposal. It seems like ultimately your poison builtin would need to decide where to propagate its poison to, and that's just what I'd call a "dependency" above. So in some sense it's just a generalization, but maybe it generalizes too far?

@shlevy

This comment has been minimized.

Copy link
Member Author

shlevy commented Sep 6, 2016

Yeah, I was actually going to first try to implement this by replacing the normal string context, where strings can refer to paths, with arbitrary serializable values, then add nix primops to manipulate the context. This would require either special-casing paths in context as magic in nix (because derivationStrict needs to know which values in context to care about and which not to) or implementing most of derivationStrict in nix itself and just having a primop that takes the full struct derivation (i.e. you have to pass in inputDrvs and inputSrcs in nix, rather than them being inferred from context).

This would be useful for a lot more than just poison and would move much of the infrastructure to nixpkgs, but it may make things significantly slower. Waiting on @edolstra's approval for the overall project before I dig in.

@Ericson2314

This comment has been minimized.

Copy link
Member

Ericson2314 commented Sep 6, 2016

I'm reminded of NixOS/nixpkgs#11094. If we do go the poisoned derivation route, reflecting on poisoning would just be a matter of doing `"${poison}" == "${myDerivation}"``.

implementing most of derivationStrict in nix itself and just having a primop that takes the full struct derivation (i.e. you have to pass in inputDrvs and inputSrcs in nix, rather than them being inferred from context).

That is very interesting.

@shlevy

This comment has been minimized.

Copy link
Member Author

shlevy commented Feb 22, 2017

@edolstra Still would be interested in doing this

@copumpkin

This comment has been minimized.

Copy link
Member

copumpkin commented Feb 23, 2017

This would also be great for tracking licenses in a fully reliable manner. I want to be able to enumerate licenses used by a package I care about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.