Discover kustomize + skaffold + nix-shell

[blaggacao]

This issue is about knowledge sharing. It is not any actionable suggestion or a specific request, other than the invitation to ponder about it.

In the recent weeks I've been tweaking a git template to manage different development environments easily throughout the team.

Goals:

• 100% reproducible k8s-dev environment
• all relevant tooling packaged in the environment (k3d, skaffold, etc.) "no need to setup minikube by hand"
• best practice base folder structure to manage different k8s environments (base, staging, dev, prod)

Please find attached the tar of the current state of the (private) repo.

chore.k8s-master.tar.gz

To test it:

1. unpack
2. have nix installed curl -L https://nixos.org/nix/install | sh
   • if you already have nix installed, make sure to update the channel nix-channel --update, kubespy is only on recent unstable channel, after it was merged a few days ago.
3. cd chore.k8s
4. nix-shell -> after building some patched command versions, you'll be dropped into a shell with the following extra tools:

 104   │ pkgs.mkShell {
 105   │   buildInputs = [
 106   │     commitizenPython
 107   │     container-structure-test
 108   │     kube-score
 109   │     # pkgs.delta - not packaged (yet)
 110   │     pkgs.dive
 111   │     pkgs.docker
 112   │     pkgs.docker-compose
 113   │     pkgs.docker-slim
 114   │     pkgs.drone-cli
 115   │     pkgs.fd
 116   │     pkgs.fluxctl
 117   │     pkgs.git
 118   │     pkgs.git-town
 119   │     pkgs.just
 120   │     pkgs.kompose
 121   │     # pkgs.kube3d
 122   │     kube3d # v3.0.0
 123   │     # pkgs.kubebox - not yet merged
 124   │     pkgs.kubectl
 125   │     pkgs.kubernetes-helm
 126   │     pkgs.kubespy
 127   │     pkgs.kubeval
 128   │     pkgs.kustomize
 129   │     pkgs.lazydocker
 130   │     pkgs.ripgrep
 131   │     pkgs.scc
 132   │     # pkgs.skaffold
 133   │     skaffold # patched fork for k3d v3.0.0
 134   │   ];

Just to highlight a few:

• dive -> insights into docker images (for auditing & optimization)
• fluxctl -> cli for fluxcd, gitops for k8s
• git-town -> better git flows for teams
• just -> a better make
• k3d "kube3d" -> launch a quick k3s cluster in docker containers (cave: no pv support!)
• kubespy -> see resources evolving on the command line
• kubebox -> k8s TUI
• skaffold -> goole's framework for manageing k8s projects (from dev to deployment)

The ./k8s folder structure is modelled after the official kustomize/kubectl docs on best practices on application management through gitops.

Feel free to ask if you have any questions.

Disclaimer: I have certain interest in this: I'd like to reach out to the communities from which we adopt software and share my findings about best practices, so that this and similar devops use cases are transparent and discoverable. Over time: mind share.

[blaggacao]

PS: please close, since it has no further purpose.

[revant]

Nice! Make it into a wiki if you wish.

[blaggacao]

Make it into a wiki if you wish.

That is not a bad idea!

I'm still fine tuning, but I will be packaging erpnext at some time in the coming weeks. My main interest is to have it work based on skaffold and fluxcd.

If I can feed my learnings back in a useful way, that's long term profit for everybody!