Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 2.0 old tracking issue #1242

Open
16 tasks done
grahamc opened this issue Mar 3, 2020 · 29 comments
Open
16 tasks done

Release 2.0 old tracking issue #1242

grahamc opened this issue Mar 3, 2020 · 29 comments
Milestone

Comments

@grahamc
Copy link
Member

grahamc commented Mar 3, 2020

features to delete or move to plugins:

  • encryptedLinksTo support
  • autocrypt?
  • backups?
  • storeKeysOnMachine
@grahamc grahamc changed the title Release 1.8 Release ~1.8~ 2.0 Mar 26, 2020
@grahamc grahamc changed the title Release ~1.8~ 2.0 Release 2.0 Mar 26, 2020
This was referenced Mar 26, 2020
@jonringer
Copy link

Once NixOS/nixpkgs#104726 gets merged and back-ported, people will have to explicitly add:

     {
       permittedInsecurePackages = [
         "python2.7-cryptography-2.9.2"
       ];
     }

for nixops to work.

@adisbladis
Copy link
Member

adisbladis commented Nov 24, 2020

Hello
Are there any plans to officially release 2.0 soon? ^^
Thanks,

The main blockers at the moment are:

  • State backends Example NixOps State Backends #1264
    This PR restricts what paths NixOps checks for deployment expressions to only network.nix or flake.nix and I'd really like to see some more input on that implication before we go ahead and merge.

  • Testing tests: Add functional tests using NixOS in Podman (Docker) #1326
    I'm not happy at all with this testing approach. The only reason for the design is that it works in Github Actions. Preferably we'd use the VM testing framework but that doesn't have acceleration in CI.

@tewfik-ghariani @grahamc (and also anyone else interested): Would you be up for a review call either this week or the next and try to push NixOps 2.0 across the finish line? There really isn't much blocking us at the moment and it's a shame that those PRs have gone unmerged for so long.

@tewfik-ghariani
Copy link
Contributor

@jonringer I don't think that would be an issue when using nixops 2.0 as that package has been upgraded in nixops-gcp :
https://github.com/nix-community/nixops-gce/blob/bc94673f9aa6abd9949ecbba86dbbec05f5f7242/pyproject.toml#L12

@adisbladis Thanks for summary! Is there a chance we can postpone the 'State Backends' changes a bit? Let's say move it to 2.1 release? I believe it's better to not rush it and test it properly and the nixops 2.0 contains many ground breaking changes that the community would like to take advantage of the soonest possible

Regarding the "Testing" PR, I agree that scheduling a review call would be the best approach to unblock the merge and the release of 2.0

cc @AmineChikhaoui

@jhillyerd
Copy link

Given 1.7 is somewhat broken, I'd also like to see 2.0 sooner. I expect a number of people try 1.7, realize it's out of date, come to this repo to try and use 2.0, and give up.

@jonringer
Copy link

jonringer commented Nov 24, 2020

@jonringer I don't think that would be an issue when using nixops 2.0 as that package has been upgraded in nixops-gcp :
https://github.com/nix-community/nixops-gce/blob/bc94673f9aa6abd9949ecbba86dbbec05f5f7242/pyproject.toml#L12

Yes, but all versions of nixops in nixpkgs are based on the 1.7 python2 build. I'm aware the the current dev branch of nixops uses python3, but that means that people will be required to package this outside of the repo for them not to be broken (or inconvenienced and vulnerable)

EDIT:
Since you don't think this will be an issue, I will move foward with marking the package as vulnerable on nixpkgs

@mkg20001
Copy link
Member

In light of the package being vulnerable and outdated I feel like NixOS/nixpkgs#83548 should get finished up and merged, so people wanting to try the version 2.0 can at least do so without further fuzz (the package could also be mentioned on the readme of this repo)

@calbrecht
Copy link
Member

There is more to do for the nixops-hetzner plugin than just merging NixOS/nixops-hetzner#9 to be compatible with nixops 2.0. I managed to get the subcommands info ssh and check running in calbrecht/nixops-hetzner#master. I guess deploy will work too, but would not verify yet. Also i managed to get two plugins work together within a single nixops derivation on the latest nixpkgs master in calbrecht/nixops-overlay#main/default.nix, fyi.

@dotlambda
Copy link
Member

dotlambda commented Feb 2, 2021

Once NixOS/nixpkgs#104726 gets merged and back-ported, people will have to explicitly add:

     {
       permittedInsecurePackages = [
         "python2.7-cryptography-2.9.2"
       ];
     }

for nixops to work.

A similar thing is the case with NixOS/nixpkgs#111322.

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-7/11552/1

@Mic92
Copy link
Member

Mic92 commented Mar 4, 2021

nix-community is switched to nixopsUnstable now due to this: nix-community/infra#57

@deepfire
Copy link
Contributor

This gets excitingly close!

@devhell
Copy link

devhell commented May 2, 2021

@deepfire Right? I'm really excited too!

@roberth
Copy link
Member

roberth commented Aug 26, 2021

Some important issues around the new state handling and locking. Fixing these results in breaking changes, so must be part of this major release.

  • How do -d and choice of flake attrs interact? Create multiple networks from the same flake #1451
  • Should we have a -d-like parameter to pick a remote state dynamically? (again, -d, as implemented, is not suitable for remote state, because it locks every deployment instead of just one)

Making remote state and locking usable (non-breaking, but quite needed)

@roberth
Copy link
Member

roberth commented Sep 9, 2021

The generated option docs are missing. They have been for a long time. Might these be a victim of the plugin architecture and repo split? This is definitely a blocker.

@JanCVanB
Copy link

JanCVanB commented Nov 5, 2021

Will NixOps 2.0 support macOS?

@roberth roberth changed the title Release 2.0 Release 2.0 old tracking issue Nov 18, 2021
@roberth
Copy link
Member

roberth commented Nov 18, 2021

Project board

https://github.com/NixOS/nixops/projects/4

Please add (or mention here so we can add):

  • Issues that impede usability
  • Improvements to documentation
  • Important changes that require changes to user expressions

@talyz
Copy link
Contributor

talyz commented Nov 23, 2021

The state backends and the general workflow still needs work done and decisions need to be made on how they should work. Quoting my comment on #1264:

Yeah, the legacy backend was broken by this, since it now ignores the nixAttrs state attribute. I've solved this by using the require attribute to import the deployment files from the network file. Specifying the network file is however cumbersome, since the --network flag currently expects a directory, not a file. To make it easier, I've opened #1480.

With that in place, I think we could essentially drop deployments altogether without losing granularity: instead of running nixops deploy -d my_deployment you'd get the same result by running nixops deploy -n my_deployment.nix, as long as my_deployment.nix requires the appropriate files and specifies the backend to use. Note however that this strategy doesn't work with the current legacy backend, since it doesn't allow choosing where the state is stored. A simple json file backend based on import/export, like @roberth is suggesting, would solve this.

Dropping deployments would mean people would have to split their deployments into separate networks, but I think it would reduce complexity and confusion immensely.

@dzmitry-lahoda
Copy link

dzmitry-lahoda commented Jan 30, 2023

I tried nixos-generators + terranix combo, looked great, so complicated (created/commented some issues on terranix to improve). I could be interesting if nixops to abandon to write is own Python provision layer and just go with Terraform plugins (run them). So NixOps would concentrate doing nix magic of wiring things (I wire generate image as variable into terraform and deploy new VM after change of nixos config - sure I would optimize later just use Cachix Deploy or SSH to switch config only, but that is later - may be I will mount config via terraform into VM :) ).

Also would be greate if NixOps abstracts away pull vs push deployment somehow. And use (like home manager, nixos-generator, nixos itself full end to end module system (if not yet). As of now I see it only partially so.

Oh, we used NixOps from master for a while. So it harder to handle then combo above as i feel.

UPDATE:

just made end to end example fully in nix, from nixos custom amazon images, to domain name system to vm insances and then nixos-rebuild nixos updates of nixified server nodes releases ggxchain/ggxnode#45 .

from pain points which are different from nixops, these so can be fixed. how to connect output of terraform to nix input. so that commands which run nixos-rebuild know what to call (actually generating nixos configuration using terraform output). part of issues can be solved running nix as hooks under terraform nix -> terraform -> nix https://developer.hashicorp.com/terraform/language/resources/provisioners/syntax

another missing peace is reading terraform lock file and getting all plugins mounted (plugins can be mounted via https://github.com/nix-community/nixpkgs-terraform-providers-bin, but need reader).

and sure what is lacking, evaluating nix code druing terraform build. HCL is like 2 languages, those that evaluates into JSON spec(that what nix works well), and another one which runs on resource/data attributes to produce attributes to be used for other resources. when terraform binary starts, it has no all data, so cannot evaluate on start. so some plugs of terraform for loops and maps are used as strings. so in theory can be solved by having converter of nix to hcl functions during derivation.

> resources.dns_zone is created before domain name
resources.domain.nameservers = [for n in resources.dns_zone.ns : name => n )]

in terranix

resources.domain.nameservers = "\${[for n in resources.dns_zone.ns : name => n )]";

so this is not type checked and is injection of foreign language.

does nixops allows to evaluate nix expression based on output response from cloud?

basically, can nix have https://developer.hashicorp.com/terraform/language/expressions/references#values-not-yet-known unkown values system like?

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/unknown-values-propagation-in-nix-like-in-hcl/26743/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests