Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support running tests in kind #1540

Merged
merged 1 commit into from
Apr 19, 2021
Merged

Support running tests in kind #1540

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 19 additions & 1 deletion tests/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,29 @@ NODE_IP =
TAG = latest
PREFIX = test-runner
KUBE_CONFIG_FOLDER = $${HOME}/.kube
KIND_KUBE_CONFIG_FOLDER = $${HOME}/.kube/kind
SHOW_IC_LOGS = no
PYTEST_ARGS =

.PHONY: build
build:
docker build -t $(PREFIX):$(TAG) -f docker/Dockerfile ..

.PHONY: run-tests
run-tests:
docker run --rm -v $(KUBE_CONFIG_FOLDER):/root/.kube $(PREFIX):$(TAG) --context=$(CONTEXT) --image=$(BUILD_IMAGE) --image-pull-policy=$(PULL_POLICY) --deployment-type=$(DEPLOYMENT_TYPE) --ic-type=$(IC_TYPE) --service=$(SERVICE) --node-ip=$(NODE_IP) --show-ic-logs=$(SHOW_IC_LOGS) $(PYTEST_ARGS)
docker run --rm -v $(KUBE_CONFIG_FOLDER):/root/.kube $(PREFIX):$(TAG) --context=$(CONTEXT) --image=$(BUILD_IMAGE) --image-pull-policy=$(PULL_POLICY) --deployment-type=$(DEPLOYMENT_TYPE) --ic-type=$(IC_TYPE) --service=$(SERVICE) --node-ip=$(NODE_IP) --show-ic-logs=$(SHOW_IC_LOGS) $(PYTEST_ARGS)

.PHONY: run-tests-in-kind
run-tests-in-kind:
$(eval KIND_CLUSTER_IP=$(shell docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' kind-control-plane))
sed -i "" "s|server:.*|server: https://$(KIND_CLUSTER_IP):6443|" $(KIND_KUBE_CONFIG_FOLDER)/config
docker run --network=kind --rm -v $(KIND_KUBE_CONFIG_FOLDER):/root/.kube $(PREFIX):$(TAG) --context=kind-kind --image=$(BUILD_IMAGE) --image-pull-policy=$(PULL_POLICY) --deployment-type=$(DEPLOYMENT_TYPE) --ic-type=$(IC_TYPE) --service=nodeport --node-ip=$(KIND_CLUSTER_IP) --show-ic-logs=$(SHOW_IC_LOGS) $(PYTEST_ARGS)

.PHONY: create-kind-cluster
create-kind-cluster:
kind create cluster
kind export kubeconfig --kubeconfig $(KIND_KUBE_CONFIG_FOLDER)/config

.PHONY: delete-kind-cluster
delete-kind-cluster:
kind delete cluster
35 changes: 30 additions & 5 deletions tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The project includes automated tests for testing the Ingress Controller in a Kubernetes cluster. The tests are written in Python3 and use the pytest framework.

Below you will find the instructions on how to run the tests against a Minikube cluster. However, you are not limited to Minikube and can use other types of Kubernetes clusters. See the [Configuring the Tests](#configuring-the-tests) section to find out about various configuration options.
Below you will find the instructions on how to run the tests against a Minikube and kind clusters. However, you are not limited to those options and can use other types of Kubernetes clusters. See the [Configuring the Tests](#configuring-the-tests) section to find out about various configuration options.

## Running Tests in Minikube

Expand Down Expand Up @@ -36,22 +36,47 @@ Run the tests:
```
The tests will use the Ingress Controller for NGINX with the default *nginx/nginx-ingress:edge* image. See the section below to learn how to configure the tests including the image and the type of NGINX -- NGINX or NGINX Plus.

## Running Tests in Kind

### Prerequisites:

* [Kind](https://kind.sigs.k8s.io/).
* Docker.

#### Step 1 - Create a Kind Cluster

```bash
$ make create-kind-cluster
```

#### Step 2 - Run the Tests

**Note**: if you have the Ingress Controller deployed in the cluster, please uninstall it first, making sure to remove its namespace and RBAC resources.

Run the tests in Docker:
```bash
$ cd tests
$ make build
$ make run-tests-in-kind
```
The tests will use the Ingress Controller for NGINX with the default *nginx/nginx-ingress:edge* image. See the section below to learn how to configure the tests including the image and the type of NGINX -- NGINX or NGINX Plus.

## Configuring the Tests

The table below shows various configuration options for the tests. If you use Python3 to run the tests, use the command-line arguments. If you use Docker, use the [Makefile](Makefile) variables.


| Command-line Argument | Makefile Variable | Description | Default |
| :----------------------- | :------------ | :------------ | :----------------------- |
| `--context` | `CONTEXT` | The context to use in the kubeconfig file. | `""` |
| `--context` | `CONTEXT`, not supported by `run-tests-in-kind` target. | The context to use in the kubeconfig file. | `""` |
| `--image` | `BUILD_IMAGE` | The Ingress Controller image. | `nginx/nginx-ingress:edge` |
| `--image-pull-policy` | `PULL_POLICY` | The pull policy of the Ingress Controller image. | `IfNotPresent` |
| `--deployment-type` | `DEPLOYMENT_TYPE` | The type of the IC deployment: deployment or daemon-set. | `deployment` |
| `--ic-type` | `IC_TYPE` | The type of the Ingress Controller: nginx-ingress or nginx-ingress-plus. | `nginx-ingress` |
| `--service` | `SERVICE` | The type of the Ingress Controller service: nodeport or loadbalancer. | `nodeport` |
| `--node-ip` | `NODE_IP` | The public IP of a cluster node. Not required if you use the loadbalancer service (see --service argument). | `""` |
| `--service` | `SERVICE`, not supported by `run-tests-in-kind` target. | The type of the Ingress Controller service: nodeport or loadbalancer. | `nodeport` |
| `--node-ip` | `NODE_IP`, not supported by `run-tests-in-kind` target. | The public IP of a cluster node. Not required if you use the loadbalancer service (see --service argument). | `""` |
| `--kubeconfig` | `N/A` | An absolute path to a kubeconfig file. | `~/.kube/config` or the value of the `KUBECONFIG` env variable |
| `N/A` | `KUBE_CONFIG_FOLDER` | A path to a folder with a kubeconfig file. | `~/.kube/` |
| `N/A` | `KUBE_CONFIG_FOLDER`, not supported by `run-tests-in-kind` target. | A path to a folder with a kubeconfig file. | `~/.kube/` |
| `--show-ic-logs` | `SHOW_IC_LOGS` | A flag to control accumulating IC logs in stdout. | `no` |
| `N/A` | `PYTEST_ARGS` | Any additional pytest command-line arguments (i.e `-m "smoke"`) | `""` |

Expand Down