Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubenix refactoring #9

Open
offlinehacker opened this Issue Feb 10, 2019 · 15 comments

Comments

Projects
None yet
3 participants
@offlinehacker
Copy link
Contributor

offlinehacker commented Feb 10, 2019

Description

Kubenix is in stage of refactoring due all the things we discovered at @GateHubNet while we were using kubenix for complex production deployments. We made kubenix public even before, so someone could reuse ideas, but it was still highly specialized tool for our needs. This refactoring will make kubenix generally available, but will also introduce breaking changes.

For work in progress branch please look into: https://github.com/xtruder/kubenix/tree/kubenix-2.0

New features

  • Improved and reimplemented submodule system with versioning support
  • Pre-generated kubernetes module definitions with explicit API versioning support
  • Pre-generated istio module definitions
  • Support for k8s 1.11,1.12,1.13
  • Validation for io.k8s.apimachinery.pkg.api.resource.Quantity and similar types
  • Helm integration
  • Initial testing framework
  • Improved unit tests
  • Improved e2e tests
  • CI integration
  • More examples
  • Usage documentation
  • Submodule documentation
  • Generated documentation for all kubernetes options
@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Feb 11, 2019

Today i added initial istio pre-generated modules. Generator is really ugly, but gets a job done.

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Feb 12, 2019

Implemented a simple testing framework and initial tests.

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Feb 20, 2019

Initial support for e2e testing has been implemented.

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Feb 28, 2019

Helm has been integrated in ba1144a thanks to @matejc who implemented: https://github.com/matejc/helm2json example usage is here: ba1144a#diff-b025248827de015784990d5577d75c8eR32

@gilligan

This comment has been minimized.

Copy link

gilligan commented Mar 7, 2019

I'm trying to the 2.0 branch for a little test ride but i'm having some difficulties figuring out what the API has changed to now. In my code I currently have something like the following to generate different configurations:

{
  buildConfig = t: kubenix.buildResources { configuration = import ./configuration.nix { type = t; }; };
}

Looking at the code I tried something like..

  buildConfig = t: builtins.toJSON (kubenix.buildResources { 
    module = import ./configuration.nix { type = t; }; 
    modules = [ kubenix.k8s ];
  });

but that doesn't quite do the right thing and I have the feeling i am doing something very wrong here 😅

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 7, 2019

@gilligan

This comment has been minimized.

Copy link

gilligan commented Mar 7, 2019

@offlinehacker thank you so much :-)

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 7, 2019

I did some refactoring on codebase, fixed a bug and added e2e test for nginx-deployment example (test.nix file). Will add more tests tomorrow, including for helm and kubenix submodules, stay tuned.

@kalbasit

This comment has been minimized.

Copy link

kalbasit commented Mar 11, 2019

  • CI integration

@offlinehacker What's the plan with regards to CI integration? I'm very much interested in replacing Helm in a GitOps managed environment with KubeNix. Is that feasible?

cc @lzaldivarkt

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 13, 2019

@kalbasit You are probably taking about e2e tests for kubenix 2.0, it's already implemented. It's spawns kubernetes cluster in qemu using nixos testing framework and nixos kubernetes module. Take a look here for example: https://github.com/xtruder/kubenix/blob/kubenix-2.0/examples/nginx-deployment/test.nix#L8 and README in same folder to see how you can run test. Also you can find bunch of unit and e2e tests for kubenix 2.0 here: https://github.com/xtruder/kubenix/tree/kubenix-2.0/tests

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 13, 2019

@kalbasit also implementation of testing is defined in this kubenix module: https://github.com/xtruder/kubenix/blob/kubenix-2.0/modules/testing.nix

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 13, 2019

As from implementation point of view kubenix 2.0 is getting more stable and i have implemented most if not all features we need, i will work now on documentation, more examples, and on refactoring of https://github.com/xtruder/kubenix-modules for kubenix 2.0

@kalbasit

This comment has been minimized.

Copy link

kalbasit commented Mar 13, 2019

@kalbasit You are probably taking about e2e tests for kubenix 2.0, it's already implemented. It's spawns kubernetes cluster in qemu using nixos testing framework and nixos kubernetes module. Take a look here for example: /examples/nginx-deployment/test.nix@kubenix-2.0#L8 and README in same folder to see how you can run test. Also you can find bunch of unit and e2e tests for kubenix 2.0 here: /tests@kubenix-2.0

This is truly awesome!!

However, I was actually talking about JenkinsX and Prow for managing and testing pull requests against an application project on GitHub. JenkinX offers deployment of applications to multiple environments (dev, staging, prod) via GitOps, which are really just Git repositories where each repositories describe the state of the world on k8s using Helm. See https://jenkins-x.io/about/concepts/#architecture and https://www.weave.works/technologies/gitops/ for more information on how this works.

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 22, 2019

This is work in progress refactoring of kubenix-submodules, i'm currently working on to make reusable set of submodules for kubenix 2.0: xtruder/kubenix-modules#88

@offlinehacker

This comment has been minimized.

Copy link
Contributor Author

offlinehacker commented Mar 22, 2019

@kalbasit We have similar setup, i will let you know how we integrate kubenix 2.0 and make an example of such setup

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.