Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pod-delete): Adding pod-delete experiment in go #1

Merged
merged 2 commits into from
May 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
build/_output
23 changes: 23 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
sudo: required
dist: xenial

services:
- docker
language: go
go:
- 1.13.1

addons:
apt:
update: true

script:
# Installing and configuring dependencies
- make deps
# Building go binaries
- make go-build
# Build
- make build

after_success:
- make push
59 changes: 59 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Makefile for building Litmus and its tools
# Reference Guide - https://www.gnu.org/software/make/manual/make.html

#
# Internal variables or constants.
# NOTE - These will be executed when any make target is invoked.
#
IS_DOCKER_INSTALLED = $(shell which docker >> /dev/null 2>&1; echo $$?)

.PHONY: all
all: deps go-build build push

.PHONY: help
help:
@echo ""
@echo "Usage:-"
@echo "\tmake all -- [default] builds the litmus containers"
@echo ""

.PHONY: deps
deps: _build_check_docker

_build_check_docker:
@echo "------------------"
@echo "--> Check the Docker deps"
@echo "------------------"
@if [ $(IS_DOCKER_INSTALLED) -eq 1 ]; \
then echo "" \
&& echo "ERROR:\tdocker is not installed. Please install it before build." \
&& echo "" \
&& exit 1; \
fi;

PHONY: go-build
go-build: experiment-go-binary

experiment-go-binary:
@echo "------------------"
@echo "--> Build experiment go binary"
@echo "------------------"
@sh build/generate_go_binary

.PHONY: build
build: litmus-go-build

litmus-go-build:
@echo "------------------"
@echo "--> Build litmus-go image"
@echo "------------------"
sudo docker build . -f build/litmus-go/Dockerfile -t litmuschaos/litmus-go:ci

.PHONY: push
push: litmus-go-push

litmus-go-push:
@echo "------------------"
@echo "--> litmus-go image"
@echo "------------------"
REPONAME="litmuschaos" IMGNAME="litmus-go" IMGTAG="ci" ./hack/push
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# litmus-python:
# litmus-go:

This repo consists of Litmus Chaos Experiments written in golang. The examples in this repo are good indicators
of how to construct the experiments in golang: complete with steady state checks, chaosresult generation, chaos injectionm,
of how to construct the experiments in golang: complete with steady state checks, chaosresult generation, chaos injection etc..,
post chaos checks, create events and reports for observability and configure sinks for these.

**NOTE**
Expand Down
2 changes: 2 additions & 0 deletions build/generate_go_binary
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Building go binaries for pod_delete experiment
go build -o build/_output/pod-delete ./experiments/generic/pod-delete
9 changes: 9 additions & 0 deletions build/litmus-go/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM ubuntu:bionic
Copy link
Member

@ksatchit ksatchit May 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can use a lighter image as well (post 1.4)


LABEL maintainer="LitmusChaos"

#Installing necessary ubuntu packages
RUN apt-get update && apt-get install -y curl

#Copying Necessary Files
COPY ./build/_output ./experiments
37 changes: 37 additions & 0 deletions build/push
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/bin/bash
set -e

if [ -z "${REPONAME}" ]
then
REPONAME="litmuschaos"
fi

if [ -z "${IMGNAME}" ] || [ -z "${IMGTAG}" ];
then
echo "Image details are missing. Nothing to push.";
exit 1
fi

IMAGEID=$( sudo docker images -q ${REPONAME}/${IMGNAME}:${IMGTAG} )

if [ ! -z "${DNAME}" ] && [ ! -z "${DPASS}" ];
then
sudo docker login -u "${DNAME}" -p "${DPASS}";
# Push image to docker hub
echo "Pushing ${REPONAME}/${IMGNAME}:${IMGTAG} ...";
sudo docker push ${REPONAME}/${IMGNAME}:${IMGTAG} ;
if [ ! -z "${TRAVIS_TAG}" ] ;
then
# Push with different tags if tagged as a release
# When github is tagged with a release, then Travis will
# set the release tag in env TRAVIS_TAG
echo "Pushing ${REPONAME}/${IMGNAME}:${TRAVIS_TAG} ...";
sudo docker tag ${IMAGEID} ${REPONAME}/${IMGNAME}:${TRAVIS_TAG}
sudo docker push ${REPONAME}/${IMGNAME}:${TRAVIS_TAG};
echo "Pushing ${REPONAME}/${IMGNAME}:latest ...";
sudo docker tag ${IMAGEID} ${REPONAME}/${IMGNAME}:latest
sudo docker push ${REPONAME}/${IMGNAME}:latest;
fi;
else
echo "No docker credentials provided. Skip uploading ${REPONAME}/${IMGNAME}:${IMGTAG} to docker hub";
fi;
144 changes: 144 additions & 0 deletions chaoslib/litmus/pod_delete/pod-delete.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
package pod_delete

import (
"k8s.io/klog"
"time"

"github.com/litmuschaos/litmus-go/pkg/types"
"github.com/litmuschaos/litmus-go/pkg/status"
"github.com/litmuschaos/litmus-go/pkg/environment"
"k8s.io/apimachinery/pkg/apis/meta/v1"
)

//PodDeleteChaos deletes the random single/multiple pods
func PodDeleteChaos(experimentsDetails *types.ExperimentDetails, clients environment.ClientSets, Iterations int, resultDetails *types.ResultDetails) error {

//ChaosStartTimeStamp contains the start timestamp, when the chaos injection begin
ChaosStartTimeStamp := time.Now().Unix()

for x := 0; x < Iterations; x++ {

//Getting the list of all the target pod for deletion
targetPodList, err := PreparePodList(experimentsDetails, clients,resultDetails)
if err != nil {
return err
}
klog.V(0).Infof("Killing %v pods", targetPodList)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
klog.V(0).Infof("Killing %v pods", targetPodList)
klog.Infof("Killing %v pods", targetPodList)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like you are not using other Verbosity Levels, so you might not need to set it also.


//Deleting the application pod
for _, pods := range targetPodList {
err := clients.KubeClient.CoreV1().Pods(experimentsDetails.AppNS).Delete(pods, &v1.DeleteOptions{})
if err != nil {
resultDetails.FailStep = "Deleting the application pod"
return err
}
}

//ChaosCurrentTimeStamp contains the current timestamp
ChaosCurrentTimeStamp := time.Now().Unix()

//ChaosDiffTimeStamp contains the difference of current timestamp and start timestamp
//It will helpful to track the total chaos duration
chaosDiffTimeStamp := ChaosCurrentTimeStamp - ChaosStartTimeStamp

if int(chaosDiffTimeStamp) >= experimentsDetails.ChaosDuration {
break
}

//Waiting for the chaos interval after chaos injection
if experimentsDetails.ChaosInterval != 0 {
klog.V(0).Infof("[Wait]: Wait for the chaos interval %vs", experimentsDetails.ChaosInterval)
waitForChaosInterval(experimentsDetails)
}
//Verify the status of pod after the chaos injection
klog.V(0).Infof("[Status]: Verification for the recreation of application pod")
err = status.CheckApplicationStatus(experimentsDetails.AppNS, experimentsDetails.AppLabel, clients)
if err != nil {
resultDetails.FailStep = "Verification for the recreation of application pod"
return err
}
}

return nil
}

//PreparePodDelete contains the steps for prepration before chaos
func PreparePodDelete(experimentsDetails *types.ExperimentDetails, clients environment.ClientSets, resultDetails *types.ResultDetails) error {

//It will get the total iteration for the pod-delete
Iterations := GetIterations(experimentsDetails)
//Waiting for the ramp time before chaos injection
if experimentsDetails.RampTime != 0 {
klog.V(0).Infof("[Ramp]: Waiting for the %vs ramp time before injecting chaos", experimentsDetails.RampTime)
waitForRampTime(experimentsDetails)
}
//Deleting for the application pod
err := PodDeleteChaos(experimentsDetails, clients, Iterations, resultDetails)
if err != nil{
return err
}
//Waiting for the ramp time after chaos injection
if experimentsDetails.RampTime != 0 {
klog.V(0).Infof("[Ramp]: Waiting for the %vs ramp time after injecting chaos", experimentsDetails.RampTime)
waitForRampTime(experimentsDetails)
}
return nil
}

//GetIterations derive the iterations value from given parameters
func GetIterations(experimentsDetails *types.ExperimentDetails) int {

Iterations := experimentsDetails.ChaosDuration / experimentsDetails.ChaosInterval
return Maximum(Iterations, 1)

}

//waitForRampTime waits for the given ramp time duration (in seconds)
func waitForRampTime(experimentsDetails *types.ExperimentDetails) {
time.Sleep(time.Duration(experimentsDetails.RampTime) * time.Second)
}

//waitForChaosInterval waits for the given ramp time duration (in seconds)
func waitForChaosInterval(experimentsDetails *types.ExperimentDetails) {
time.Sleep(time.Duration(experimentsDetails.ChaosInterval) * time.Second)
}

// Maximum calculates the maximum value among two integers
func Maximum(a int, b int) int {
if a > b {
return a
}
return b
}

//Minimum calculates the minimum value among two integers
func Minimum(a int, b int) int {
if a > b {
return b
}
return a
}

//PreparePodList derive the list of target pod for deletion
//It is based on the KillCount value
func PreparePodList(experimentsDetails *types.ExperimentDetails, clients environment.ClientSets, resultDetails *types.ResultDetails) ([]string, error) {

var targetPodList []string
//Getting the list of pods with the given labels and namespaces
pods, err := clients.KubeClient.CoreV1().Pods(experimentsDetails.AppNS).List(v1.ListOptions{LabelSelector: experimentsDetails.AppLabel})
if err != nil {
resultDetails.FailStep = "Getting the list of pods with the given labels and namespaces"
return targetPodList, err
}

//Adding the first pod only, if KillCount is not set or 0
//Otherwise derive the min(KIllCount,len(pod_list)) pod
if experimentsDetails.KillCount == 0 {
targetPodList = append(targetPodList, pods.Items[0].Name)
} else {
for i := 0; i < Minimum(experimentsDetails.KillCount, len(pods.Items)); i++ {
targetPodList = append(targetPodList, pods.Items[i].Name)
}
}
return targetPodList, err
}
63 changes: 63 additions & 0 deletions experiments/generic/pod-delete/pod-delete-k8s-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
apiVersion: batch/v1
kind: Job
metadata:
generateName: pod-delete-
spec:
template:
metadata:
labels:
experiment: pod-delete
spec:
# Placeholder that is updated by the executor for automated runs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unable to get the context of this comment

# Provide appropriate SA (with desired permissions) if executed manually
serviceAccountName: litmus
restartPolicy: Never
containers:
- name: gotest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use appropriate container name? something like go-exp

image: litmuschaos/litmus-go:ci
imagePullPolicy: Always
env:
- name: ANSIBLE_STDOUT_CALLBACK
value: 'default'

# provide application namespace
- name: APP_NAMESPACE
value: ''

# provide application labels
- name: APP_LABEL
value: ''

# provide application kind
- name: APP_KIND
value: ''

- name: TOTAL_CHAOS_DURATION
value: '30'

- name: CHAOS_INTERVAL
value: '10'

- name: CHAOS_NAMESPACE
value: ''

## Period to wait before/after injection of chaos
- name: RAMP_TIME
value: ''

## env var that describes the library used to execute the chaos
## Only supported litmus
- name: LIB
value: 'litmus'

- name: CHAOSENGINE
value: ''

- name: CHAOS_SERVICE_ACCOUNT
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName

command: ["/bin/bash"]
args: ["-c", "./experiments/pod-delete"]
Loading