Skip to content
egressip-ipam-operator
Go Makefile Shell Other
Branch: master
Clone or download

Latest commit

raffaelespazzoli minor fixes to the API (#7)
* minor fixes to the API

Signed-off-by: Raffaele Spazzoli <raffaele.spazzoli@gmail.com>

* more doc fixes

Signed-off-by: Raffaele Spazzoli <raffaele.spazzoli@gmail.com>
Latest commit 4e1a0c7 Mar 15, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
build initial commit Mar 3, 2020
charts/egressip-ipam-operator minor fixes to the API (#7) Mar 15, 2020
cmd/manager added deploy key (#2) Mar 8, 2020
deploy
pkg minor fixes to the API (#7) Mar 15, 2020
scripts/build initial commit Mar 3, 2020
test minor fixes to the API (#7) Mar 15, 2020
version initial commit Mar 3, 2020
.gitignore initial commit Mar 3, 2020
.travis.yml added deploy key (#2) Mar 8, 2020
Makefile
github_deploy_key.enc added deploy key (#2) Mar 8, 2020
go.mod Aws 2 (#4) Mar 13, 2020
go.sum Aws 2 (#4) Mar 13, 2020
readme.md minor fixes to the API (#7) Mar 15, 2020
tools.go initial commit Mar 3, 2020

readme.md

EgressIP IPAM Operator

Build Status Docker Repository on Quay

This operator automates the assignment of egressIPs to namespaces. Namespaces can opt in to receiving one or more egressIPs with the following annotation egressip-ipam-operator.redhat-cop.io/egressipam:<egressIPAM>, where egressIPAM is the CRD that controls how egressIPs are assigned. IPs assigned to the namespace can be looked up in the following annotation: egressip-ipam-operator.redhat-cop.io/egressips.

EgressIP assignments is managed by the EgressIPAM CRD. Here is an example of it:

apiVersion: redhatcop.redhat.io/v1alpha1
kind: EgressIPAM
metadata:
  name: example-egressipam
spec:
  cidrAssignments:
    - labelValue: "true"
      CIDR: 192.169.0.0/24
  topologyLabel: egressGateway
  nodeSelector:
    node-role.kubernetes.io/worker: ""

This EgressIPAM specifies that all nodes that comply with the specified node selector and that also have labels egressGateway: "true" will be assigned egressIP from the specified CIDR.

Note that the cidrAssigments field is an array and therefore, multiple groups of nodes can be identified with the labelValue and different CIDRs can be assigned to them. This is usually not necessary on a bare metal deployment.

In the bare metal scenario, when this egressCRD is created, all the hostsubnet relative to the nodes selected by this EgressIPAM will be update to have the EgressCIDRs field equal to the specified CIDR (see below for cloud provider scenarios).

When a namespace is created with the opt-in annotation: egressip-ipam-operator.redhat-cop.io/egressipam=<egressIPAM>, an available egressIP is selected from the CIDR and assigned to the namespace.The netnamespace associated with this namespace is updated to use that egressIP.

Passing EgressIPs as input

The normal mode of operatios of this operator is to pick a random IP from the configured CIDR. However it also support a scenario where egressIP are picked by an external process and passed as input.

In this case IPs must me passed as an annotation to the namespace, like this: egressip-ipam-operator.redhat-cop.io/egressips=IP1,IP2.... The value of the annotation is a comma separated array of ip with no spaces.

There must be exactly one IP per CIDR defined in the referenced egressIPAM. Moreover each IP must belong to the corresponding CIDR. It also also a responsibility of the progress picking the IPs to ensure that those IPs are actually available.

Assumptions

The following assumptions apply when using this operator:

  1. If multiple EgressIPAMs are defined, their selected nodes MUST NOT overlap.
  2. When using a cloud provider the topology label MUST be topology.kubernetes.io/zone.

Support for AWS

In AWS as well as other cloud providers, one cannot freely assign IPs to machines. Additional steps need to be performed in this case. Considering this EgressIPAM

apiVersion: redhatcop.redhat.io/v1alpha1
kind: EgressIPAM
metadata:
  name: egressipam-aws
spec:
  cidrAssignments:
    - labelValue: "eu-central-1a"
      CIDR: 10.0.128.0/20
    - labelValue: "eu-central-1b"
      CIDR: 10.0.144.0/20
    - labelValue: "eu-central-1c"
      CIDR: 10.0.160.0/20
  topologyLabel: topology.kubernetes.io/zone
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker: ""

When a namespace with the opt-in annotation is created, the following happens:

  1. for each of the CIDRs, an available IP is selected and assigned to the namespace
  2. the relative netnamespace is update to reflect the assignment (multiple IPs will be assigned in this case).
  3. one node per zone is selected to carry the egressIP
  4. the relative aws machines are assigned the additional IP on the main interface (support for secondary interfaces in not available)
  5. the relative hostsubnets are updated to reflect the assigned IP, the egressIP field is updated.

Deploying the Operator

This is a cluster-level operator that you can deploy in any namespace, egressip-ipam-operator is recommended.

You can either deploy it using Helm or creating the manifests directly.

Deploying with Helm

Here are the instructions to install the latest release with Helm.

oc new-project egressip-ipam-operator

helm repo add egressip-ipam-operator https://redhat-cop.github.io/egressip-ipam-operator
helm repo update
export egressip_ipam_operator_chart_version=$(helm search repo egressip-ipam-operator/egressip-ipam-operator | grep egressip-ipam-operator/egressip-ipam-operator | awk '{print $2}')

helm fetch egressip-ipam-operator/egressip-ipam-operator --version ${egressip_ipam_operator_chart_version}
helm template egressip-ipam-operator-${egressip_ipam_operator_chart_version}.tgz --namespace egressip-ipam-operator | oc apply -f - -n egressip-ipam-operator

rm egressip-ipam-operator-${egressip_ipam_operator_chart_version}.tgz

Deploying directly with manifests

Here are the instructions to install the latest release creating the manifest directly in OCP.

git clone git@github.com:redhat-cop/egressip-ipam-operator.git; cd egressip-ipam-operator
oc apply -f deploy/crds/redhatcop.redhat.io_egressipams_crd.yaml
oc new-project egressip-ipam-operator
oc -n egressip-ipam-operator apply -f deploy

Local Development

Execute the following steps to develop the functionality locally. It is recommended that development be done using a cluster with cluster-admin permissions.

go mod download

optionally:

go mod vendor

Using the operator-sdk, run the operator locally:

oc apply -f deploy/crds/redhatcop.redhat.io_egressipams_crd.yaml
oc new-project egressip-ipam-operator
oc apply -f deploy/service_account.yaml -n egressip-ipam-operator
oc apply -f deploy/role.yaml -n egressip-ipam-operator
oc apply -f deploy/role_binding.yaml -n egressip-ipam-operator
export token=$(oc serviceaccounts get-token 'egressip-ipam-operator' -n egressip-ipam-operator)
oc login --token=${token}
OPERATOR_NAME='egressip-ipam-operator' NAMESPACE='egressip-ipam-operator' operator-sdk --verbose up local --namespace ""

Testing

Baremetal test

oc apply -f test/egressIPAM-baremetal.yaml
oc apply -f test/namespace-baremetal.yaml

AWS test

based on the output of the below command, configure your egressIPAM for AWS.

for vmidurl in $(oc get nodes -l node-role.kubernetes.io/worker="" -o json | jq -r .items[].spec.providerID); do
  vmid=${vmidurl##*/}
  subnetid=$(aws ec2 --region eu-central-1 describe-instances --instance-ids ${vmid} | jq -r .Reservations[0].Instances[0].NetworkInterfaces[0].SubnetId)
  echo $(aws ec2 --region eu-central-1 describe-subnets --subnet-ids ${subnetid} | jq -r '.Subnets[0] | .CidrBlock + " " + .AvailabilityZone')
done
oc apply -f test/egressIPAM-AWS.yaml
oc apply -f test/namespace-AWS.yaml

Release Process

To release execute the following:

git tag -a "<version>" -m "release <version>"
git push upstream <version>

use this version format: vM.m.z

You can’t perform that action at this time.