etcdproxy-proof-of-concept

The etcdproxy-proof-of-concept is a proof of concept made for my GSoC project to demonstrate using etcd Namespaces exposed by etcd-gRPC server with Kubernetes Aggregated API servers.

This project is no longer maintained. Instead, check out xmudrii/etcdproxy-controller.

If you want to learn more about my GSoC project, check out the following resources:

• GSoC project page
• Submitted proposal (PDF)
• Proposal draft (public Google Document)
• SIG-API-Machinery mailing list post

You can follow the project's progress by following:

• Project's GitHub Tracker repository
• Project's Trello
• Daily updated Google Document

Prerequisites

• etcd version 3.2 or newer
• etcdctl is recommended if you want to test is the proxy working as expected
• If you want to test this project with aggregated API server, you need:
  • minimal Kubernetes cluster (Minikube or local-up-cluster.sh)
  • sample-apiserver
• Certificates generated needed for the proxy server. Certificates can be generated using the following commands:

openssl genrsa -out server.key 2048
openssl ecparam -genkey -name secp384r1 -out server.key
openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650

etcd

If you're developing for Kubernetes, the easiest way to install etcd is to use the provided install-etcd.sh script. After running it, the script will output instructions needed to complete installation.

Other ways include running etcd using Docker or running standalone, which are covered in the Getting Started section of etcd's README.

Kubernetes

The sample-apiserver requires kubeconfig and a minimal Kubernetes cluster. The easiest way to run an cluster is to use the local-up-cluster.sh script located in the Kubernetes repository. You can also use Minikube.

sample-apiserver

The sample-apiserver is a minimal aggregated API server and a great foundation for building your own API servers. It works with etcd, so we can use it to test does etcdproxy-proof-of-concept works as expected. To do so, you need to point API server to use the etcdproxy-proof-of-concept as its etcd server, by using the --etcd-servers flag. Before running the API server, make sure the etcdproxy is running as well.

Instructions for running sample-apiserver will be added to the project's README.md file. In meanwhile, you can check out the kubernetes/kubernetes#55476 PR for instructions.

Installing etcdproxy-proof-of-concept

In order to install and compile the etcdproxy-proof-of-conecpt project, you need configured Go environment.

Installing the project can be done using the Go's toolchain:

go get -u github.com/xmudrii/etcdproxy-proof-of-concept

You can also use precompiled releases, but keep in mind that this is a fast-moving project, so you could be missing the latest features.

Running etcdproxy-proof-of-concept

Once installed, you can run the project by using the etcdproxy-proof-of-concept command, which runs etcd server on the port 23790, by proxying the namespace called default. By default it uses etcd server running on the port 2379.

The etcdproxy-proof-of-concept command has the following flags you can use to change default values:

• --etcdAddresses - address of the etcd instance (formatted as http://127.0.0.1:2379,http://<ip>:<port>,...)
• --namespace - namespace to use
• --bindAddress - bind etcdproxy to the provided address and port (formatted as <address>:<port>)

Testing proxy using sample-apiserver

To test this project using sample-apiserver you need to point sample-apiserver to use etcdproxy-proof-of-concept as its etcd server, by using the --etcd-servers flag.

The sample-apiserver comes with a Flunder resource, which you can use to create a simple resource, need to test writing to etcd.

In the artifacts/flunders/flunder.yml file you can find the manifest for a simple Flunder. You can use API server's API to apply it, such as:

http --verify no -j --cert-key client.key --cert client.crt https://localhost:8443/apis/wardle.k8s.io/v1alpha1/namespaces/default/flunders < <(python -c 'import sys, yaml, json; json.dump(yaml.load(sys.stdin), sys.stdout, indent=4)' < artifacts/flunders/flunder.yml)

The latest version of curl doesn't work very well with unsigned certificates, so it's recommended to use httpie instead.

The above command uses Python to convert the YAML manifest to the JSON formatted request, so API can parse it. The response contains information about your request.

You can use etcdctl to make sure the Flunder is defined in etcd:

ETCDCTL_API=3 etcdctl get / --prefix --keys-only --endpoints 127.0.0.1:23790 --cert="server.crt" --key="server.key" --insecure-skip-tls-verify=true | grep wardle

This should return output such as:

/registry/wardle.kubernetes.io/wardle.k8s.io/flunders/default/my-first-flunder

Now, point etcdctl to the main etcd server. This time, the key should have /default (or namespace's name) as a prefix.

ETCDCTL_API=3 etcdctl get / --prefix --keys-only | grep wardle

The output should look like:

/default/registry/wardle.kubernetes.io/wardle.k8s.io/flunders/default/my-first-flunder