Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixops depends on nix-channel #736

Open
bobvanderlinden opened this issue Sep 30, 2017 · 13 comments

Comments

@bobvanderlinden
Copy link

@bobvanderlinden bobvanderlinden commented Sep 30, 2017

I use NixOS on my laptop and use that laptop to deploy machine configurations to my servers. For experimentation purposes I used nixpkgs unstable channel for my laptop. Later I did a deploy in nixops and it 'upgraded' all machines to unstable versions. As you can imagine, that wasn't my intention.

There needs to be a way to get nixops to use packages independent of the system it is running on. Basically I want 'nixops deploy' to be reproducible: it shouldn't depend on system-wide repositories/options like nix-channels. Ideally, I think, this should be the default.

To override nix-channels I tried to use NIX_PATH=nixpkgs=https://nixos.org/channels/nixos-17.09/nixexprs.tar.xz nixops deploy but that resulted in:

NIX_PATH=nixpkgs=https://nixos.org/channels/nixos-17.09/nixexprs.tar.xz nixops deploy
error: hash mismatch importing path ‘/nix/store/7y6zp51ahqgn03yb7zr2pwzrjdc3274w-nixexprs.tar.xz’; expected hash ‘sha256:19b8596gq7nnzrhny7p39k52w84jdcf6jv9ddali8spb0hr7mhjz’, got ‘sha256:194z7axwwxxpxs5lmxj1ff7094jdn72qmbanqryrwac98lx0kfgl’

Another alternative, suggested by infinisil on IRC, is to use a git submodule of nixpkgs in the configuration repository where the machine configuration of nixops resides. In addition use NIX_PATH=nixpkgs=$PWD/nixpkgs nixops deploy to make sure nixops is using the right version of nixpkgs.

Even when using the URL in nixpkgs would've worked, these solutions both seem like workarounds. What is the recommended way to have nixops work independently of the local system configuration?

@aycanirican

This comment has been minimized.

Copy link
Member

@aycanirican aycanirican commented Oct 1, 2017

@AmineChikhaoui

This comment has been minimized.

Copy link
Member

@AmineChikhaoui AmineChikhaoui commented Oct 2, 2017

@bobvanderlinden

This comment has been minimized.

Copy link
Author

@bobvanderlinden bobvanderlinden commented Oct 2, 2017

Yes, on IRC it was suggested to add a git submodule in the repo where my nixops configuration resides. It indeed seems like an alright solution.

My main concern is that this is something that should be default. It's very dangerous that nixops deploy depends on system-wide nixpkgs without that being very explicit in the command-line or documentation. I ran into this by accident updating my server to nixpkgs-unstable and I can imagine others doing the same thing. In addition, the server only being able to update when the system is updated.

It 'feels' dirty. It is doing the exact thing NixOS is trying to avoid: depend on system-wide and implicit settings. Nixops should have its own channel/update procedure.

Maybe this ties into NixOS/nix#779 ? Which suggests removing nix-channel altogether and relying on URLs in NIX_PATH.

EDIT: I do like @aycanirican suggestion of using nix.nixPath. Combining that with an URL does seem to resolve the issue I was having:

    nix.nixPath = [
      "nixpkgs=https://nixos.org/channels/nixos-17.09/nixexprs.tar.xz"
    ];

Should this be documented in the nixops manual?

@aycanirican

This comment has been minimized.

Copy link
Member

@aycanirican aycanirican commented Oct 3, 2017

@bobvanderlinden

This comment has been minimized.

Copy link
Author

@bobvanderlinden bobvanderlinden commented Oct 4, 2017

The previously mentioned option nix.nixPath = [ ... ] does not work as intended. It does not seem to be used while building the machine configuration. (does not seem to download anything when using a nix.nixPath = [ "nixpkgs=URL" ].

Using nix-shell is indeed a good option. I gave it a go, altered your example a bit and it seems be workable: https://gist.github.com/bobvanderlinden/e9ef4c338882f1ef31424e4743be15fc

That being said, should this be documented for nixops? The current documentation doesn't state anything on nixops depending on the running systems configuration/channels. In hindsight I see this could have been obvious, but it seems dangerous when nixops is being used in production.

@eqyiel

This comment has been minimized.

Copy link

@eqyiel eqyiel commented Oct 4, 2017

Another thing you can do is use something like direnv and set a default NIX_PATH in your deployments repo: https://github.com/eqyiel/deployments/blob/master/.envrc#L4

@bobvanderlinden

This comment has been minimized.

Copy link
Author

@bobvanderlinden bobvanderlinden commented Oct 5, 2017

@eqyiel That's a fine solution when working by myself. I'm more interested in a documented/standard/recommended workflow that can work for teams as well.

When I'm showing people nixops it doesn't motivate them to get started when they see the 'extra baggage' that is needed to get a reproducible environment. Whether that 'extra baggage' is direnv, nix-shell, NIX_PATH does not really matter. I do have to say that nixops has a more stable environment than most other deploytools have, but reproducability is Nix's main selling point and nixops doesn't really adhere to that point without the 'extra baggage'. Especially when that 'extra baggage' isn't really documented.

@eqyiel

This comment has been minimized.

Copy link

@eqyiel eqyiel commented Oct 5, 2017

@bobvanderlinden FWIW, that is what I do with my team in our private repo and they are able to understand that they need to install direnv from a "getting started" note in the README.

NIX_PATH is not extra baggage, that's just how you tell Nix where to find nixpkgs (and expressions for nixops) if you don't want to use -I nixpkgs=/path/to/nixpkgs everywhere. It's documented in the Nix manual: https://nixos.org/nix/manual/#sec-common-env

How do you suggest this be made clearer?

I think another problem here is that nix-channel is confusing, even for experienced users:

NixOS/nixpkgs#7113
NixOS/nix#813
NixOS/nix#1548

@bobvanderlinden

This comment has been minimized.

Copy link
Author

@bobvanderlinden bobvanderlinden commented Oct 6, 2017

Yes, I absolutely agree nix-channel is confusing. It might also be the root of this issue. From what I understand nix-channel will be deprecated sooner or later (probably later ;)).

That said, it's still confusing that the documentation doesn't warn anyone using nixops deploy to make sure the same nixpkgs is used every time it is executed.

If I were to use nixops in a team I would make sure to don't tell anyone to use nixops by themselves, but create a shell script that uses any of the above suggestions to run nixops deploy.

The problem is that if anyone runs nixops deploy without any of the above suggestions it would deploy a very different version to a whole cluster!

Having nix_path defined (and used) in the machine/cluster configuration seems like the best solution. Currently it isn't picked up by nixops it seems.

@teto

This comment has been minimized.

Copy link
Contributor

@teto teto commented Oct 17, 2017

[temporary hijack since it seems related] Is it possible that nixops affect nixos-rebuild in some way ? like retaining references to some package that make this package reused by the system instead of a newer version. While updating the strongswan package I've sometimes noticed network manager reusing the old strongswan after a nixos-rebuild even though it had built the new one.

@bobvanderlinden

This comment has been minimized.

Copy link
Author

@bobvanderlinden bobvanderlinden commented Oct 19, 2017

I don't know how nixops itself could affect nixos-rebuild. I can imagine doing a nix-channel update, because you want to do an update through nixops, will also affect the next run of nixos-rebuild.

@wmertens

This comment has been minimized.

Copy link

@wmertens wmertens commented Jul 30, 2019

the nix.nixPath = [ "nixpkgs=https://nixos.org/channels/nixos-19.03/nixexprs.tar.xz" ]; thing works, but you also need it on your deploy host.

@asymmetric

This comment has been minimized.

Copy link
Contributor

@asymmetric asymmetric commented Aug 3, 2019

On the deploy host I use a nix shell to pin NIX_PATH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.