DEMO: Local development with k3d, skaffold, kustomize

An example local k3s development environment using kustomize, skaffold and k3d.

Table of Contents

• Features
• Prerequisites
• Usage
• Kustomize configuration

Features

• Bootstraps k3s cluster in Docker using k3d
• Skaffold loads docker images directly into the k3s cluster
• Skaffold uses kustomize for building and deploying k8s manifests using local-dev overlay.
• An example node.js app will be bootstrapped with File sync and Port forward enabled

Prerequisites

NB. The setup is tested on MacOS & Linux with brew installed.

The first prerequisite is to install go-task in order to make the setup a bit easier:

### MacOS
brew install go-task/tap/go-task

### Linux
$ sudo sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin

The following prerequisites are used in order to create and manage the local K3s cluster:

• Docker or Docker Desktop
• Kubectl
• Kustomize
• Skaffold
• K3d
• Go Task (Note: Go Task as a more modern iteration of the Makefile utility)

If you don't have them installed yet you can install them using install-prerequisites task:

task install-prerequisites

Usage

Create k3s cluster:

NB If you want to change the amount of k3s agents argument e.g. k3d:create-cluster -- <number_of_agents>

$ task k3d:create-cluster

Make sure your KUBECONFIG points to k3s cluster context (if not already):

$ kubectl get nodes

Start the local development environment:

$ task skaffold:dev

An example node.js app is available at:

localhost:3000

$ curl localhost:3000
Hello World!

Make some changes to src/index.js and they will be synchronized to the pod(s) running the app.

Delete the k3s cluster:

task k3d:delete-cluster

Delete images that are built by Skaffold and stored on the local Docker daemon:

task docker:rmi

Kustomize configuration

Kustomize configuration is based on Directory Structure Based Layout in order to use multiple environments with different configuration. In order to use different clusters remember to specify the corresponding context before applying changes using Skaffold.

├── base
│   ├── deployment.yaml
│   ├── hpa.yaml
│   ├── kustomization.yaml
│   └── service.yaml
└── overlays
    ├── local-dev
    │   ├── deployment-patch.yaml
    │   ├── hpa-patch.yaml
    │   ├── kustomization.yaml
    ├── prod
    │   ├── deployment-patch.yaml
    │   ├── hpa-patch.yaml
    │   └── kustomization.yaml
    └── staging
        ├── deployment-patch.yaml
        ├── hpa-patch.yaml
        └── kustomization.yaml        

        

Note: kustomize for prod and staging, need changes of skaffold.yaml file -> kubeContext & (OPTIONAL) portForward: resourceName: if using kustomize EXAMPLE 2 , kustomization files, ETC.