Skip to content

rurumimic/dockerless

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
ca-trust
ca-trust
 
 
k8s
k8s
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
Vagrantfile
Vagrantfile
 
 
hello.png
hello.png
 
 

Repository files navigation

dockerless

VM

  • Ubuntu Server 20.04 LTS
  • CPU: 2
  • Memory: 4GB
  • IP: 192.168.33.100
  • Kubernetes: 1.19.4

(Option) Add a enterprise CA as a trusted certificate authority

Add *.cer, *.crt, *.pem to .gitignore.

.cer to .crt:

  • openssl x509 -inform DER -in before.cer -noout text
  • openssl x509 -inform DER -in before.cer -out after.crt

And copy certificates to ca-trust directory:

cp /path/to/enterprise.crt ./ca-trust

Vagrant

vagrant box add ubuntu/focal64 # --insecure
vagrant up
vagrant ssh

CRI-O

Install

OS=xUbuntu_20.04
VERSION=1.19
cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /
EOF
cat <<EOF | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list
deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /
EOF
curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key add -
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add -

sudo apt-get update
sudo apt-get install -y cri-o cri-o-runc

Start

sudo systemctl daemon-reload
sudo systemctl enable crio
sudo systemctl start crio

Kubernetes

Setup

sudo swapoff -a

sudo modprobe br_netfilter # lsmod | grep br_netfilter

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system | grep k8s

cat <<EOF | sudo tee /proc/sys/net/ipv4/ip_forward
1
EOF

kubelet, kubeadm, kubectl

sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet=1.19.4-00 kubeadm=1.19.4-00 kubectl=1.19.4-00
sudo apt-mark hold kubelet kubeadm kubectl

Configure kubelet

cat <<EOF | sudo tee /etc/default/kubelet
KUBELET_EXTRA_ARGS=--feature-gates="AllAlpha=false,RunAsGroup=true" --container-runtime=remote --cgroup-driver=systemd --container-runtime-endpoint='unix:///var/run/crio/crio.sock' --runtime-request-timeout=5m
EOF
sudo systemctl daemon-reload
sudo systemctl restart kubelet

Init a cluster

CIDR=192.168.0.0/16 # Calico CNI
VM_IP=192.168.33.100
sudo kubeadm init --pod-network-cidr=$CIDR --control-plane-endpoint=$VM_IP --apiserver-advertise-address=$VM_IP

Result:

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.33.100:6443 --token wyq927.gaw8fuj1u8be3tm1 \
    --discovery-token-ca-cert-hash sha256:e8a7a22886a9b066861996cc1d19e656516308aceb3c2e560d5bc77df613d29b

Setup kubectl

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install Calico

kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml
watch kubectl get po -A

Remove the taints on the master

kubectl describe node ubuntu-focal | grep Taints

Taints:             node-role.kubernetes.io/master:NoSchedule

Remove the taints:

kubectl taint nodes --all node-role.kubernetes.io/master-

node/ubuntu-focal untainted

Get nodes

kubectl get nodes -o wide

NAME           STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
ubuntu-focal   Ready    master   15m   v1.19.4   10.0.2.15     <none>        Ubuntu 20.04.1 LTS   5.4.0-56-generic   cri-o://1.19.0

Container Images

Docker Registry Secret

Create a Secret by providing credentials on the command line

kubectl create secret docker-registry regcred \
--docker-server=<your-registry-server> \ 
--docker-username=<your-name> \
--docker-password=<your-pword> \
--docker-email=<your-email>
kubectl get secret regcred --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode

Start Kaniko

Node.js App Dockerfile: /k8s/app/Dockerfile

KANIKO_PATH="/k8s/app"
DOCKERHUB_REPO="<user-name>/<repo>"

Run kaniko:

cat /k8s/kaniko/volume.yaml | sed "s#{{path}}#$KANIKO_PATH#g" | kubectl apply -f -
kubectl apply -f /k8s/kaniko/volume-claim.yaml
cat /k8s/kaniko/pod.yaml | sed "s#{{repo}}#$DOCKERHUB_REPO#g" | kubectl apply -f -

Check whether the build complete and show the build logs:

kubectl logs kaniko -f
kubectl get pods

NAME     READY   STATUS      RESTARTS   AGE
kaniko   0/1     Completed   0          42s

Go to: https://hub.docker.com/repository/docker/<user-name>/<repo>

Deploy a service

kubectl apply -f /k8s/nginx/configmap.yaml -f /k8s/deployment.yaml

Check:

watch kubectl get pods

NAME                     READY   STATUS      RESTARTS   AGE
hello-8595777499-6np5n   2/2     Running     0          30s

Test

On the host machine: http://192.168.33.100:30080

Delete a service

kubectl delete -f /k8s/nginx/configmap.yaml -f /k8s/deployment.yaml

Git server

Create git user

sudo adduser --shell /usr/bin/git-shell --disabled-password --gecos "" git
echo git:12345 | sudo chpasswd
sudo -u git mkdir -m 700 /home/git/.ssh /home/git/repos /home/git/git-shell-commands

Set git-shell-commands

cat <<EOF | sudo -u git tee /home/git/git-shell-commands/no-interactive-login
#!/bin/sh
printf '%s\n' "Hi! You've successfully authenticated, but I do not"
printf '%s\n' "provide interactive shell access."
exit 128
EOF
sudo chmod +x /home/git/git-shell-commands/no-interactive-login

Generate a pair keys

ssh-keygen -t ed25519 -C "your_email@example.com" -f $HOME/.ssh/id_ed25519 -q -N ""
ls -1 $HOME/.ssh/id_*

/home/vagrant/.ssh/id_ed25519
/home/vagrant/.ssh/id_ed25519.pub

SSH add:

eval $(ssh-agent)
ssh-add ~/.ssh/id_ed25519

Copy keys:

cat $HOME/.ssh/id_ed25519.pub | sudo tee -a /home/git/.ssh/authorized_keys

Create a new repo

Copy a node.js project

mkdir $HOME/workspace
cp -r /k8s/app $HOME/workspace/app

Init a git repository

cd $HOME/workspace/app
git init --shared=true
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git add .
git commit -m "first commit"

Create a bare repository

cd $HOME/workspace
git clone --bare app app.git

Copy repositories

sudo cp -r $HOME/workspace/app.git /home/git/repos
sudo chown -R git:git /home/git/repos
sudo chmod -R ug+rwX /home/git/repos
sudo find /home/git/repos -type d -exec chmod g+s '{}' +

Test clone a repository

git clone ssh://git@192.168.33.100/~/repos/app.git /tmp/app
Cloning into '/tmp/app'...
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 8 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (8/8), 7.39 KiB | 7.39 MiB/s, done.

Jenkins

Create a namespace

kubectl create namespace jenkins

Create a service account

kubectl apply -f /k8s/jenkins/sa.yaml

Deploy Jenkins

kubectl apply -f /k8s/jenkins/deployment.yaml

Get Jenkins credentials

kubectl exec -n jenkins $(kubectl get pods -n jenkins -l app=jenkins --no-headers -o custom-columns=":metadata.name") \
-- cat /var/jenkins_home/secrets/initialAdminPassword

# 3d355c615186450291f868c4f4155c7e

Access Jenkins dashboard

http://192.168.33.100:32080

  1. Unlocking Jenkins
  2. Install suggested plugins
  3. Create first admin user: admin
  4. Instance configuration
    • Jenkins URL: http://192.168.33.100:32080/

(Skip) Install plugins

Installed plugin list: Already installed the Kubernetes plugin.

How to install plugins:

  1. Go to http://192.168.33.100:32080/pluginManager/available.
  2. Search kubernetes, git, pipeline, Shared workspace.
  3. Install plugins and restart Jenkins.
    • Kubernetes, Kubernetes :: Pipeline :: DevOps Steps
    • Git
    • Pipeline
    • Shared workspace

Shared Workspace

  1. System Configuration
  2. Workspace Sharing
    1. Workspaces Name: my_project
    2. URL: ssh://git@192.168.33.100/~/repos/app.git
  3. Save

Kubernetes Cloud configuration

  1. Jenkins: Configure Clouds
  2. Add a new cloud: kubernetes
    • Kubernetes URL: https://192.168.33.100:6443 or https://kubernetes.default.svc.cluster.local
    • Kubernetes Namespace: jenkins
    • Credentials
      • Kind: Kubernetes Service Account
    • Direct Connection: enabled
    • Jenkins URL: http://192.168.33.100:32080
    • Jenkins tunnel: http://192.168.33.100:32081
  3. System Configuration → Jenkins Location → URL: http://192.168.33.100:32080
  4. Configure Global Security → Agents: TCP port for inbound agents 32081

Create a Pipeline

Git Hook: post-receive

cat <<EOF | sudo -u git tee /home/git/repos/app.git/hooks/post-receive
#!/bin/sh
curl http://192.168.33.100:32080/git/notifyCommit?url=ssh://git@192.168.33.100/~/repos/app.git
EOF
sudo chmod 775 /home/git/repos/app.git/hooks/post-receive

Creat a Folder

  1. Create a job
    1. Enter an item name: My Pipelines
    2. Select Folder
  2. Save

Create a Job 1

  1. Go to My Pipelines → Creat a Job
    1. Enter an item name: Git Hook
    2. Select Freestyle project
  2. General → Shared Workspace: my_project (ssh://git@192.168.33.100/~/repos/app.git)
  3. Source Code Management: Git
    • Repository URL: ssh://git@192.168.33.100/~/repos/app.git
    • Credentials: make git
      1. Add
      2. Kind: SSH Username with private key
      3. Username: git
      4. Private Key: enter id_ed25519.
  4. Build Triggers: Poll SCM enabled (No polling schedule is required.)
  5. Save

Creat a Job 2

  1. Go to My Pipelines → Creat a Job
    1. Enter an item name: Build a image
    2. Select Pipeline
  2. Build Triggers → Build after other projects are built → git hook
  3. Add Pipeline and Save

Script:

pipeline {
  agent {
    kubernetes {
      defaultContainer 'kaniko'
      yaml """
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    imagePullPolicy: Always
    args: ["--dockerfile=/workspace/Dockerfile", "--destination=rurumimic/hellothere"]
    tty: true
    volumeMounts:
      - name: jenkins-docker-cfg
        mountPath: /kaniko/.docker
      - name: dockerfile-storage
        mountPath: /workspace
  volumes:
  - name: jenkins-docker-cfg
    projected:
      sources:
      - secret:
          name: regcred
          items:
            - key: .dockerconfigjson
              path: config.json
  - name: dockerfile-storage
    hostPath:
      path: /var/jenkins_home/sharedspace/my_project
"""
    }
  }
  stages {
    stage('Build with Kaniko') {
      steps {
        sh 'ls -al /workspace'
      }
    }
  }
}
podTemplate(yaml: """
kind: Pod
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    imagePullPolicy: Always
    command:
    - /busybox/cat
    tty: true
    volumeMounts:
      - name: kaniko-secret
        mountPath: /kaniko/.docker
      - name: dockerfile-storage
        mountPath: /workspace
  volumes:
  - name: kaniko-secret
    secret:
      secretNname: regcred
      items:
        - key: .dockerconfigjson
          path: config.json
  - name: dockerfile-storage
    hostPath:
      path: /var/jenkins_home/sharedspace/my_project
"""
  ) {

  node(POD_LABEL) {
    stage('Build with Kaniko') {
      container('kaniko') {
        sh '/kaniko/executor -f /workspace/Dockerfile -c /workspace --cache=true --destination=rurumimic/hellothere'
      }
    }
  }
}

Commit and Deploy

Create a secret for jenkins branch

kubectl create secret docker-registry regcred -n jenkins \
--docker-server=https://index.docker.io/v1/ \ 
--docker-username=<your-name> \
--docker-password=<your-pword> \
--docker-email=<your-email>

Clone a repository

git clone ssh://git@192.168.33.100/~/repos/app.git /tmp/app

Edit

cd /tmp/app
sed -i 's/Hello/Hi/' app.js

Commit and Push

git add .
git commit -m "Update"
git push

Test

Go to Jenkins: my-project

And your Docker Hub repository.

Clean up VM

vagrant halt
vagrant destroy -f

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

3 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages