Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion failed in derivation add with output from derivation show #8747

Open
iFreilicht opened this issue Jul 26, 2023 · 1 comment
Open
Labels

Comments

@iFreilicht
Copy link
Contributor

iFreilicht commented Jul 26, 2023

Describe the bug

Trying to import any json derivation that was created with nix derivation show by using nix derivation add will fail with an assertion error in nlohmann:

Assertion failed: (it != m_value.object->end()), function operator[], file /nix/store/8h9pxgq1776ns6qi5arx08ifgnhmgl22-nlohmann_json-3.11.2/include/nlohmann/json.hpp, line 2135.

Steps To Reproduce

Short version: nix derivation show nixpkgs#halp | nix derivation add

Long version:

  1. Export derivation: nix derivation show nixpkgs#halp > halp.json
  2. Remove it from the store: nix store delete $(cat halp.json | jq -r 'keys[0]')
  3. $ cat halp.json | nix derivation add
    Assertion failed: (it != m_value.object->end()), function operator[], file /nix/store/8h9pxgq1776ns6qi5arx08ifgnhmgl22-nlohmann_json-3.11.2/include/nlohmann/json.hpp, line 2135.
    zsh: done       cat halp.json | 
    zsh: abort      result/bin/nix derivation add
    

Step 2 can also be skipped, as this already fails in the function Derivation::fromJSON, I just wanted to make clear this has nothing to do with the derivation already being present in the store.

This doesn't seem to depend on the package at all, I tried eight different ones and they all show the exact same error.

Expected behavior

I would expect this to work. A simple roundtrip conversion from a derivation to json and back should be no issue.

I understand now (after consulting the tests) that technically this already works, you just have to extract the actual derivation, as it's wrapped in another object:

$ nix derivation show nixpkgs#halp | jq '.[]' | nix derivation add
/nix/store/88gcds8vqir0sb1zzx0f34j9s8cz8sl6-halp-0.1.7.drv

I guess this makes some sense, as derivation show can output multiple derivations if passed the -r flag, but passing that to derivation add is a nightmare anyway.

Maybe a good way to solve this would be to detect if the input JSON is an object with keys ending in ".drvs" and if so, iterate over their values and ingest those with Derivation::fromJSON? This would be a non-breaking change.

This behavior is also critically underdocumented, there is not a single example in derivation show or derivation add that explains that you can't just feed the output of the first into the latter without transforming it first.

nix-env --version output
nix (Nix) 2.15.1
nix (Nix) 2.16.1
nix (Nix) 2.18.0pre20230725_2d1d811 (current master at the time of this writing)

Additional context

@Ericson2314 I'm tagging you as it seems you created derivation add in 59e0728. I'm willing to submit a PR for this myself.

Priorities

Add 👍 to issues you find important.

@iFreilicht iFreilicht added the bug label Jul 26, 2023
@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/compile-nixos-derivations-into-a-single-file-for-a-single-host/30920/14

iFreilicht added a commit to iFreilicht/nix that referenced this issue Jul 28, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Jul 31, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 4, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 5, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 6, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 6, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 6, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 7, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 7, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 7, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 7, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 7, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 17, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
iFreilicht added a commit to iFreilicht/nix that referenced this issue Aug 17, 2023
`derivation add` could not ingest output from `derivation show`
directly, it had to be transformed in between.
Documentation for `derivation add` was also very sparse, and tests
didn't cover the `derivation show --recursive` case.

Fixes NixOS#8747
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants