diff --git a/docs/tutorials/gke.md b/docs/tutorials/gke.md index 28419ad05e..902d4ceeb6 100644 --- a/docs/tutorials/gke.md +++ b/docs/tutorials/gke.md @@ -1,414 +1,295 @@ # Setting up ExternalDNS on Google Kubernetes Engine -This tutorial describes how to setup ExternalDNS for usage within a GKE cluster. Make sure to use **>=0.4** version of ExternalDNS for this tutorial +This tutorial describes how to setup ExternalDNS for usage within a [GKE](https://cloud.google.com/kubernetes-engine) ([Google Kuberentes Engine](https://cloud.google.com/kubernetes-engine)) cluster. Make sure to use **>=0.11.0** version of ExternalDNS for this tutorial -## Set up your environment +## Single project test scenario using access scopes -Setup your environment to work with Google Cloud Platform. Fill in your values as needed, e.g. target project. +*If you prefer to try-out ExternalDNS in one of the existing environments you can skip this step* -```console -$ gcloud config set project "zalando-external-dns-test" -$ gcloud config set compute/region "europe-west1" -$ gcloud config set compute/zone "europe-west1-d" -``` +The following instructions use [access scopes](https://cloud.google.com/compute/docs/access/service-accounts#accesscopesiam) to provide ExternalDNS with the permissions it needs to manage DNS records within a single [project](https://cloud.google.com/docs/overview#projects), the organizing entity to allocate resources. -## GKE Node Scopes +Note that since these permissions are associated with the instance, all pods in the cluster will also have these permissions. As such, this approach is not suitable for anything but testing environments. -*If you prefer to try-out ExternalDNS in one of the existing environments you can skip this step* +This solution will only work when both CloudDNS and GKE are provisioned in the same project. If the CloudDNS zone is in a different project, this solution will not work. -The following instructions use instance scopes to provide ExternalDNS with the -permissions it needs to manage DNS records. Note that since these permissions -are associated with the instance, all pods in the cluster will also have these -permissions. As such, this approach is not suitable for anything but testing -environments. +### Configure Project Environment -Create a GKE cluster. +Setup your environment to work with Google Cloud Platform. Fill in your variables as needed, e.g. target project. -```console -$ gcloud container clusters create "external-dns" \ - --num-nodes 1 \ - --scopes "https://www.googleapis.com/auth/ndev.clouddns.readwrite" +```bash +# set variables to the appropriate desired values +PROJECT_ID="my-external-dns-test" +REGION="europe-west1" +ZONE="europe-west1-d" +ClOUD_BILLING_ACCOUNT="" +# set default settings for project +gcloud config set project $PROJECT_ID +gcloud config set compute/region $REGION +gcloud config set compute/zone $ZONE +# enable billing and APIs if not done already +gcloud beta billing projects link $PROJECT_ID \ + --billing-account $BILLING_ACCOUNT +gcloud services enable "dns.googleapis.com" +gcloud services enable "container.googleapis.com" +``` + +### Create GKE Cluster + +```bash +gcloud container clusters create $GKE_CLUSTER_NAME \ + --num-nodes 1 \ + --scopes "https://www.googleapis.com/auth/ndev.clouddns.readwrite" ``` -Create a DNS zone which will contain the managed DNS records. +**WARNING**: Note that this cluster will use the default [compute engine GSA](https://cloud.google.com/compute/docs/access/service-accounts#default_service_account) that contians the overly permissive project editor (`roles/editor`) role. So essentially, anything on the cluster could potentially grant escalated privileges. Also, as mentioned earlier, the access scope `ndev.clouddns.readwrite` will allow anything running on the cluster to have read/write permissions on all Cloud DNS zones within the same project. -```console -$ gcloud dns managed-zones create "external-dns-test-gcp-zalan-do" \ - --dns-name "external-dns-test.gcp.zalan.do." \ - --description "Automatically managed zone by kubernetes.io/external-dns" -``` +### Cloud DNS Zone -Make a note of the nameservers that were assigned to your new zone. +Create a DNS zone which will contain the managed DNS records. If using your own domain that was registered with a third-party domain registrar, you should point your domain's name servers to the values under the `nameServers` key. Please consult your registrar's documentation on how to do that. This tutorial will use example domain of `example.com`. -```console -$ gcloud dns record-sets list \ - --zone "external-dns-test-gcp-zalan-do" \ - --name "external-dns-test.gcp.zalan.do." \ - --type NS -NAME TYPE TTL DATA -external-dns-test.gcp.zalan.do. NS 21600 ns-cloud-e1.googledomains.com.,ns-cloud-e2.googledomains.com.,ns-cloud-e3.googledomains.com.,ns-cloud-e4.googledomains.com. +```bash +gcloud dns managed-zones create "example-com" --dns-name "example.com." \ + --description "Automatically managed zone by kubernetes.io/external-dns" ``` -In this case it's `ns-cloud-{e1-e4}.googledomains.com.` but your's could slightly differ, e.g. `{a1-a4}`, `{b1-b4}` etc. - -Tell the parent zone where to find the DNS records for this zone by adding the corresponding NS records there. Assuming the parent zone is "gcp-zalan-do" and the domain is "gcp.zalan.do" and that it's also hosted at Google we would do the following. +Make a note of the nameservers that were assigned to your new zone. -```console -$ gcloud dns record-sets transaction start --zone "gcp-zalan-do" -$ gcloud dns record-sets transaction add ns-cloud-e{1..4}.googledomains.com. \ - --name "external-dns-test.gcp.zalan.do." --ttl 300 --type NS --zone "gcp-zalan-do" -$ gcloud dns record-sets transaction execute --zone "gcp-zalan-do" +```bash +gcloud dns record-sets list \ + --zone "example-com" --name "example.com." --type NS ``` -### Deploy ExternalDNS +Outputs: -Then apply the following manifests file to deploy ExternalDNS. - -```yaml -apiVersion: v1 -kind: ServiceAccount -metadata: - name: external-dns ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: external-dns -rules: -- apiGroups: [""] - resources: ["services","endpoints","pods"] - verbs: ["get","watch","list"] -- apiGroups: ["extensions","networking.k8s.io"] - resources: ["ingresses"] - verbs: ["get","watch","list"] -- apiGroups: [""] - resources: ["nodes"] - verbs: ["get", "watch", "list"] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: external-dns-viewer -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: external-dns -subjects: -- kind: ServiceAccount - name: external-dns - namespace: default ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: external-dns -spec: - strategy: - type: Recreate - selector: - matchLabels: - app: external-dns - template: - metadata: - labels: - app: external-dns - spec: - serviceAccountName: external-dns - containers: - - name: external-dns - image: k8s.gcr.io/external-dns/external-dns:v0.8.0 - args: - - --source=service - - --source=ingress - - --domain-filter=external-dns-test.gcp.zalan.do # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones - - --provider=google -# - --google-project=zalando-external-dns-test # Use this to specify a project different from the one external-dns is running inside - - --google-zone-visibility=private # Use this to filter to only zones with this visibility. Set to either 'public' or 'private'. Omitting will match public and private zones - - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization - - --registry=txt - - --txt-owner-id=my-identifier +``` +NAME TYPE TTL DATA +example.com. NS 21600 ns-cloud-e1.googledomains.com.,ns-cloud-e2.googledomains.com.,ns-cloud-e3.googledomains.com.,ns-cloud-e4.googledomains.com. ``` -Use `--dry-run` if you want to be extra careful on the first run. Note, that you will not see any records created when you are running in dry-run mode. You can, however, inspect the logs and watch what would have been done. +In this case it's `ns-cloud-{e1-e4}.googledomains.com.` but your's could slightly differ, e.g. `{a1-a4}`, `{b1-b4}` etc. -### Verify ExternalDNS works +## Cross project access scenario using Google Service Account -Create the following sample application to test that ExternalDNS works. +More often, following best practices in regards to security and operations, Cloud DNS zones will be managed in a separate project from the Kubernetes cluster. This section shows how setup ExternalDNS to access Cloud DNS from a different project. These steps will also work for single project scenarios as well. -```yaml -apiVersion: v1 -kind: Service -metadata: - name: nginx - annotations: - external-dns.alpha.kubernetes.io/hostname: nginx.external-dns-test.gcp.zalan.do. -spec: - type: LoadBalancer - ports: - - port: 80 - targetPort: 80 - selector: - app: nginx +ExternalDNS will need permissions to make changes to the Cloud DNS zone. There are three ways to configure the access needed: ---- +* [Worker Node Service Account](#worker-node-service-account) +* [Static Credentials](#static-credentials) +* [Work Load Identity](#work-load-identity) -apiVersion: apps/v1 -kind: Deployment -metadata: - name: nginx -spec: - selector: - matchLabels: - app: nginx - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - name: nginx - ports: - - containerPort: 80 -``` +### Setup Cloud DNS and GKE -After roughly two minutes check that a corresponding DNS record for your service was created. - -```console -$ gcloud dns record-sets list \ - --zone "external-dns-test-gcp-zalan-do" \ - --name "nginx.external-dns-test.gcp.zalan.do." +Below are examples on how you can configure Cloud DNS and GKE in separate projects, and then use one of the three methods to grant access to ExternalDNS. Replace the environment variables to values that make sense in your environment. -NAME TYPE TTL DATA -nginx.external-dns-test.gcp.zalan.do. A 300 104.155.60.49 -nginx.external-dns-test.gcp.zalan.do. TXT 300 "heritage=external-dns,external-dns/owner=my-identifier" -``` +#### Configure Projects -Note created TXT record alongside A record. TXT record signifies that the corresponding A record is managed by ExternalDNS. This makes ExternalDNS safe for running in environments where there are other records managed via other means. +For this process, create projects with the appropriate APIs enabled. -Let's check that we can resolve this DNS name. We'll ask the nameservers assigned to your zone first. +```bash +# set variables to appropriate desired values +GKE_PROJECT_ID="my-workload-project" +DNS_PROJECT_ID="my-cloud-dns-project" +ClOUD_BILLING_ACCOUNT="" +# enable billing and APIs for DNS project if not done already +gcloud config set project $DNS_PROJECT_ID +gcloud beta billing projects link $CLOUD_DNS_PROJECT \ + --billing-account $ClOUD_BILLING_ACCOUNT +gcloud services enable "dns.googleapis.com" +# enable billing and APIs for GKE project if not done already +gcloud config set project $GKE_PROJECT_ID +gcloud beta billing projects link $CLOUD_DNS_PROJECT \ + --billing-account $ClOUD_BILLING_ACCOUNT +gcloud services enable "container.googleapis.com" +``` + +#### Provisioning Cloud DNS + +Create a Cloud DNS zone in the designated DNS project. -```console -$ dig +short @ns-cloud-e1.googledomains.com. nginx.external-dns-test.gcp.zalan.do. -104.155.60.49 +```bash +gcloud dns managed-zones create "example-com" --project $DNS_PROJECT_ID \ + --description "example.com" --dns-name="example.com." --visibility=public ``` -Given you hooked up your DNS zone with its parent zone you can use `curl` to access your site. +If using your own domain that was registered with a third-party domain registrar, you should point your domain's name servers to the values under the `nameServers` key. Please consult your registrar's documentation on how to do that. The example domain of `example.com` will be used for this tutorial. -```console -$ curl nginx.external-dns-test.gcp.zalan.do - - - -Welcome to nginx! -... - - -... - - -``` +#### Provisioning a GKE cluster for cross project access -Let's check that Ingress works as well. Create the following Ingress. +Create a GSA (Google Service Account) and grant it the [minimal set of privileges required](https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster#use_least_privilege_sa) for GKE nodes: -```yaml -apiVersion: networking.k8s.io/v1 -kind: Ingress -metadata: - name: nginx -spec: - rules: - - host: via-ingress.external-dns-test.gcp.zalan.do - http: - paths: - - backend: - service: - name: nginx - port: - number: 80 - pathType: Prefix -``` +```bash +GKE_CLUSTER_NAME="my-external-dns-cluster" +GKE_REGION="us-central1" +GKE_SA_NAME="worker-nodes-sa" +GKE_SA_EMAIL="$GKE_SA_NAME@${GKE_PROJECT_ID}.iam.gserviceaccount.com" -Again, after roughly two minutes check that a corresponding DNS record for your Ingress was created. +ROLES=( + roles/logging.logWriter + roles/monitoring.metricWriter + roles/monitoring.viewer + roles/stackdriver.resourceMetadata.writer +) -```console -$ gcloud dns record-sets list \ - --zone "external-dns-test-gcp-zalan-do" \ - --name "via-ingress.external-dns-test.gcp.zalan.do." \ +gcloud iam service-accounts create $GKE_SA_NAME \ + --display-name $GKE_SA_NAME --project $GKE_PROJECT_ID -NAME TYPE TTL DATA -via-ingress.external-dns-test.gcp.zalan.do. A 300 130.211.46.224 -via-ingress.external-dns-test.gcp.zalan.do. TXT 300 "heritage=external-dns,external-dns/owner=my-identifier" +# assign google service account to roles in GKE project +for ROLE in ${ROLES[*]}; do + gcloud projects add-iam-policy-binding $GKE_PROJECT_ID \ + --member "serviceAccount:$GKE_SA_EMAIL" \ + --role $ROLE +done ``` -Let's check that we can resolve this DNS name as well. +Create a cluster using this service account and enable [workload identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity): -```console -dig +short @ns-cloud-e1.googledomains.com. via-ingress.external-dns-test.gcp.zalan.do. -130.211.46.224 +```bash +gcloud container clusters create $GKE_CLUSTER_NAME \ + --project $GKE_PROJECT_ID --region $GKE_REGION --num-nodes 1 \ + --service-account "$GKE_SA_EMAIL" \ + --workload-pool "$GKE_PROJECT_ID.svc.id.goog" ``` -Try with `curl` as well. +### Worker Node Service Account method -```console -$ curl via-ingress.external-dns-test.gcp.zalan.do - - - -Welcome to nginx! -... - - -... - - -``` +In this method, the GSA (Google Service Account) that is associated with GKE worker nodes will be configured to have access to Cloud DNS. -### Clean up +**WARNING**: This will grant access to modify the Cloud DNS zone records for all containers running on cluster, not just ExternalDNS, so use this option with caution. This is not recommended for production environments. -Make sure to delete all Service and Ingress objects before terminating the cluster so all load balancers get cleaned up correctly. +```bash +GKE_SA_EMAIL="$GKE_SA_NAME@${GKE_PROJECT_ID}.iam.gserviceaccount.com" -```console -$ kubectl delete service nginx -$ kubectl delete ingress nginx +# assign google service account to dns.admin role in the cloud dns project +gcloud projects add-iam-policy-binding $DNS_PROJECT_ID \ + --member serviceAccount:$GKE_SA_EMAIL \ + --role roles/dns.admin ``` -Give ExternalDNS some time to clean up the DNS records for you. Then delete the managed zone and cluster. +After this, follow the steps in [Deploy ExternalDNS](#deploy-externaldns). Make sure to set the `--google-project` flag to match the Cloud DNS project name. -```console -$ gcloud dns managed-zones delete "external-dns-test-gcp-zalan-do" -$ gcloud container clusters delete "external-dns" -``` +### Static Credentials -Also delete the NS records for your removed zone from the parent zone. +In this scenario, a new GSA (Google Service Account) is created that has access to the CloudDNS zone. The credentials for this GSA are saved and installed as a Kubernetes secret that will be used by ExternalDNS. -```console -$ gcloud dns record-sets transaction start --zone "gcp-zalan-do" -$ gcloud dns record-sets transaction remove ns-cloud-e{1..4}.googledomains.com. \ - --name "external-dns-test.gcp.zalan.do." --ttl 300 --type NS --zone "gcp-zalan-do" -$ gcloud dns record-sets transaction execute --zone "gcp-zalan-do" -``` +This allows only containers that have access to the secret, such as ExternalDNS to update records on the Cloud DNS Zone. -## GKE with Workload Identity +#### Create GSA for use with static credentials -The following instructions use [GKE workload -identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) -to provide ExternalDNS with the permissions it needs to manage DNS records. -Workload identity is the Google-recommended way to provide GKE workloads access -to GCP APIs. +```bash +DNS_SA_NAME="external-dns-sa" +DNS_SA_EMAIL="$DNS_SA_NAME@${GKE_PROJECT_ID}.iam.gserviceaccount.com" -Create a GKE cluster with workload identity enabled. +# create GSA used to access the Cloud DNS zone +gcloud iam service-accounts create $DNS_SA_NAME --display-name $DNS_SA_NAME -```console -$ gcloud container clusters create external-dns \ - --workload-metadata-from-node=GKE_METADATA_SERVER \ - --identity-namespace=zalando-external-dns-test.svc.id.goog +# assign google service account to dns.admin role in cloud-dns project +gcloud projects add-iam-policy-binding $DNS_PROJECT_ID \ + --member serviceAccount:$DNS_SA_EMAIL --role "roles/dns.admin" ``` -Create a GCP service account (GSA) for ExternalDNS and save its email address. +#### Create Kubernetes secret using static credentials -```console -$ sa_name="Kubernetes external-dns" -$ gcloud iam service-accounts create sa-edns --display-name="$sa_name" -$ sa_email=$(gcloud iam service-accounts list --format='value(email)' \ - --filter="displayName:$sa_name") +Generate static credentials from the ExternalDNS GSA. + +```bash +# download static credentials +gcloud iam service-accounts keys create /local/path/to/credentials.json \ + --iam-account $DNS_SA_EMAIL ``` -Bind the ExternalDNS GSA to the DNS admin role. +Create a Kubernetes secret with the credentials in the same namespace of ExternalDNS. -```console -$ gcloud projects add-iam-policy-binding zalando-external-dns-test \ - --member="serviceAccount:$sa_email" --role=roles/dns.admin +```bash +kubectl create secret generic "external-dns" --namespace ${EXTERNALDNS_NS:-"default"} \ + --from-file /local/path/to/credentials.json ``` -Link the ExternalDNS GSA to the Kubernetes service account (KSA) that -external-dns will run under, i.e., the external-dns KSA in the external-dns -namespaces. +After this, follow the steps in [Deploy ExternalDNS](#deploy-externaldns). Make sure to set the `--google-project` flag to match Cloud DNS project name. Make sure to uncomment out the section that mounts the secret to the ExternalDNS pods. +### Workload Identity -```console -$ gcloud iam service-accounts add-iam-policy-binding "$sa_email" \ - --member="serviceAccount:zalando-external-dns-test.svc.id.goog[external-dns/external-dns]" \ - --role=roles/iam.workloadIdentityUser -``` +[Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) allows workloads in your GKE cluster to impersonate GSA (Google Service Accounts) using KSA (Kubernetes Service Accounts) configured during deployemnt. These are the steps to use this feature with ExternalDNS. + +#### Create GSA for use with Workload Identity -Create a DNS zone which will contain the managed DNS records. +```bash +DNS_SA_NAME="external-dns-sa" +DNS_SA_EMAIL="$DNS_SA_NAME@${GKE_PROJECT_ID}.iam.gserviceaccount.com" -```console -$ gcloud dns managed-zones create external-dns-test-gcp-zalan-do \ - --dns-name=external-dns-test.gcp.zalan.do. \ - --description="Automatically managed zone by ExternalDNS" +gcloud iam service-accounts create $DNS_SA_NAME --display-name $DNS_SA_NAME +gcloud projects add-iam-policy-binding $DNS_PROJECT_ID \ + --member serviceAccount:$DNS_SA_EMAIL --role "roles/dns.admin" ``` -Make a note of the nameservers that were assigned to your new zone. +#### Link KSA to GSA -```console -$ gcloud dns record-sets list \ - --zone=external-dns-test-gcp-zalan-do \ - --name=external-dns-test.gcp.zalan.do. \ - --type NS -NAME TYPE TTL DATA -external-dns-test.gcp.zalan.do. NS 21600 ns-cloud-e1.googledomains.com.,ns-cloud-e2.googledomains.com.,ns-cloud-e3.googledomains.com.,ns-cloud-e4.googledomains.com. +Add an IAM policy binding bewtween the workload identity GSA and ExternalDNS GSA. This will link the ExternalDNS KSA to ExternalDNS GSA. + +```bash +gcloud iam service-accounts add-iam-policy-binding $DNS_SA_EMAIL \ + --role "roles/iam.workloadIdentityUser" \ + --member "serviceAccount:$GKE_PROJECT_ID.svc.id.goog[${EXTERNALDNS_NS:-"default"}/external-dns]" ``` -In this case it's `ns-cloud-{e1-e4}.googledomains.com.` but your's could -slightly differ, e.g. `{a1-a4}`, `{b1-b4}` etc. +#### Deploy External DNS + +Deploy ExternalDNS with the following steps below, documented under [Deploy ExternalDNS](#deploy-externaldns). Set the `--google-project` flag to the Cloud DNS project name. -Tell the parent zone where to find the DNS records for this zone by adding the -corresponding NS records there. Assuming the parent zone is "gcp-zalan-do" and -the domain is "gcp.zalan.do" and that it's also hosted at Google we would do the -following. +#### Link KSA to GSA in Kubernetes -```console -$ gcloud dns record-sets transaction start --zone=gcp-zalan-do -$ gcloud dns record-sets transaction add ns-cloud-e{1..4}.googledomains.com. \ - --name=external-dns-test.gcp.zalan.do. --ttl 300 --type NS --zone=gcp-zalan-do -$ gcloud dns record-sets transaction execute --zone=gcp-zalan-do +Add the proper workload identity annotation to the ExternalDNS KSA. + +```bash +kubectl annotate serviceaccount "external-dns" \ + --namespace ${EXTERNALDNS_NS:-"default"} \ + "iam.gke.io/gcp-service-account=$DNS_SA_EMAIL" ``` -Connect your `kubectl` client to the cluster you just created and bind your GCP -user to the cluster admin role in Kubernetes. +#### Update ExternalDNS pods + +Update the Pod spec to schedule the workloads on nodes that use Workload Identity and to use the annotated Kubernetes service account. -```console -$ gcloud container clusters get-credentials external-dns -$ kubectl create clusterrolebinding cluster-admin-me \ - --clusterrole=cluster-admin --user="$(gcloud config get-value account)" +```bash +kubectl patch deployment "external-dns" \ + --namespace ${EXTERNALDNS_NS:-"default"} \ + --patch \ + '{"spec": {"template": {"spec": {"nodeSelector": {"iam.gke.io/gke-metadata-server-enabled": "true"}}}}}' ``` -### Deploy ExternalDNS +After all of these steps you may see several messages with `googleapi: Error 403: Forbidden, forbidden`. After several minutes when the token is refreshed, these error messages will go away, and you should see info messages, such as: `All records are already up to date`. + +## Deploy ExternalDNS -Apply the following manifest file to deploy external-dns. +Then apply the following manifests file to deploy ExternalDNS. ```yaml apiVersion: v1 -kind: Namespace -metadata: - name: external-dns ---- -apiVersion: v1 kind: ServiceAccount metadata: name: external-dns - namespace: external-dns + labels: + app.kubernetes.io/name: external-dns --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: external-dns + labels: + app.kubernetes.io/name: external-dns rules: - apiGroups: [""] - resources: ["services", "endpoints", "pods"] - verbs: ["get", "watch", "list"] - - apiGroups: ["extensions", "networking.k8s.io"] + resources: ["services","endpoints","pods","nodes"] + verbs: ["get","watch","list"] + - apiGroups: ["extensions","networking.k8s.io"] resources: ["ingresses"] - verbs: ["get", "watch", "list"] - - apiGroups: [""] - resources: ["nodes"] - verbs: ["list"] + verbs: ["get","watch","list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: external-dns-viewer + labels: + app.kubernetes.io/name: external-dns roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole @@ -416,83 +297,81 @@ roleRef: subjects: - kind: ServiceAccount name: external-dns - namespace: external-dns + namespace: default # change if namespace is not 'default' --- apiVersion: apps/v1 kind: Deployment metadata: name: external-dns - namespace: external-dns + labels: + app.kubernetes.io/name: external-dns spec: strategy: type: Recreate selector: matchLabels: - app: external-dns + app.kubernetes.io/name: external-dns template: metadata: labels: - app: external-dns + app.kubernetes.io/name: external-dns spec: + serviceAccountName: external-dns containers: - - args: - - --source=ingress + - name: external-dns + image: k8s.gcr.io/external-dns/external-dns:v0.11.0 + args: - --source=service - - --domain-filter=external-dns-test.gcp.zalan.do + - --source=ingress + - --domain-filter=example.com # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones - --provider=google - - --google-project=zalando-external-dns-test + # - --google-project=my-cloud-dns-project # Use this to specify a project different from the one external-dns is running inside + - --google-zone-visibility=public # Use this to filter to only zones with this visibility. Set to either 'public' or 'private'. Omitting will match public and private zones + - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization - --registry=txt - --txt-owner-id=my-identifier - image: k8s.gcr.io/external-dns/external-dns:v0.8.0 - name: external-dns - securityContext: - fsGroup: 65534 - runAsUser: 65534 - serviceAccountName: external-dns + # # uncomment below if static credentials are used + # env: + # - name: GOOGLE_APPLICATION_CREDENTIALS + # value: /etc/secrets/service-account/credentials.json + # volumeMounts: + # - name: google-service-account + # mountPath: /etc/secrets/service-account/ + # volumes: + # - name: google-service-account + # secret: + # secretName: external-dns ``` -Then add the proper workload identity annotation to the cert-manager service -account. +Create the deployment for ExternalDNS: ```bash -$ kubectl annotate serviceaccount --namespace=external-dns external-dns \ - "iam.gke.io/gcp-service-account=$sa_email" +kubectl create --namespace "default" --filename externaldns.yaml ``` -### Deploy a sample application +## Verify ExternalDNS works + +The following will deploy a small nginx server that will be used to demonstrate that ExternalDNS is working. -Create the following sample application to test that ExternalDNS works. +### Verify using an external load balancer + +Create the following sample application to test that ExternalDNS works. This example will provision a L4 load balancer. ```yaml -apiVersion: networking.k8s.io/v1 -kind: Ingress -metadata: - name: nginx -spec: - rules: - - host: via-ingress.external-dns-test.gcp.zalan.do - http: - paths: - - backend: - service: - name: nginx - port: - number: 80 - pathType: Prefix ---- apiVersion: v1 kind: Service metadata: - annotations: - external-dns.alpha.kubernetes.io/hostname: nginx.external-dns-test.gcp.zalan.do. name: nginx + annotations: + # change nginx.example.com to match an appropriate value + external-dns.alpha.kubernetes.io/hostname: nginx.example.com spec: + type: LoadBalancer ports: - - port: 80 - targetPort: 80 + - port: 80 + targetPort: 80 selector: app: nginx - type: LoadBalancer --- apiVersion: apps/v1 kind: Deployment @@ -508,78 +387,118 @@ spec: app: nginx spec: containers: - - image: nginx - name: nginx - ports: - - containerPort: 80 + - image: nginx + name: nginx + ports: + - containerPort: 80 ``` -After roughly two minutes check that a corresponding DNS records for your -service and ingress were created. +Create the deployment and service objects: -```console -$ gcloud dns record-sets list \ - --zone "external-dns-test-gcp-zalan-do" \ - --name "via-ingress.external-dns-test.gcp.zalan.do." \ - --type A -NAME TYPE TTL DATA -nginx.external-dns-test.gcp.zalan.do. A 300 104.155.60.49 -nginx.external-dns-test.gcp.zalan.do. TXT 300 "heritage=external-dns,external-dns/owner=my-identifier" -via-ingress.external-dns-test.gcp.zalan.do. TXT 300 "heritage=external-dns,external-dns/owner=my-identifier" -via-ingress.external-dns-test.gcp.zalan.do. A 300 35.187.1.246 +```bash +kubectl create --namespace "default" --filename nginx.yaml ``` -Let's check that we can resolve this DNS name as well. +After roughly two minutes check that a corresponding DNS record for your service was created. -```console -$ dig +short @ns-cloud-e1.googledomains.com. via-ingress.external-dns-test.gcp.zalan.do. -35.187.1.246 +```bash +gcloud dns record-sets list --zone "example-com" --name "nginx.example.com." ``` -Try with `curl` as well. +Example output: -```console -$ curl via-ingress.external-dns-test.gcp.zalan.do - - - -Welcome to nginx! -... - - -... - - +``` +NAME TYPE TTL DATA +nginx.example.com. A 300 104.155.60.49 +nginx.example.com. TXT 300 "heritage=external-dns,external-dns/owner=my-identifier" ``` -### Clean up +Note created `TXT` record alongside `A` record. `TXT` record signifies that the corresponding `A` record is managed by ExternalDNS. This makes ExternalDNS safe for running in environments where there are other records managed via other means. -Make sure to delete all service and ingress objects before terminating the -cluster so all load balancers and DNS entries get cleaned up correctly. +Let's check that we can resolve this DNS name. We'll ask the nameservers assigned to your zone first. -```console -$ kubectl delete ingress nginx -$ kubectl delete service nginx +```bash +dig +short @ns-cloud-e1.googledomains.com. nginx.example.com. +104.155.60.49 ``` -Give ExternalDNS some time to clean up the DNS records for you. Then delete the -managed zone and cluster. +Given you hooked up your DNS zone with its parent zone you can use `curl` to access your site. + +```bash +curl nginx.example.com +``` + +### Verify using an ingress + +Let's check that Ingress works as well. Create the following Ingress. -```console -$ gcloud dns managed-zones delete external-dns-test-gcp-zalan-do -$ gcloud container clusters delete external-dns +```yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: nginx +spec: + rules: + - host: server.example.com + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: nginx-svc + port: + number: 80 ``` -Also delete the NS records for your removed zone from the parent zone. +Create the ingress objects with: -```console -$ gcloud dns record-sets transaction start --zone gcp-zalan-do -$ gcloud dns record-sets transaction remove ns-cloud-e{1..4}.googledomains.com. \ - --name=external-dns-test.gcp.zalan.do. --ttl 300 --type NS --zone=gcp-zalan-do -$ gcloud dns record-sets transaction execute --zone=gcp-zalan-do +```bash +kubectl create --namespace "default" --filename ingress.yaml ``` -## User Demo How-To Blogs and Examples +Note that this will ingress object will use the default ingress controller that comes with GKE to create a L7 load balancer in addition to the L4 load balancer previously with the service object. To use only the L7 load balancer, update the service manafest to change the Service type to `NodePort` and remove the ExternalDNS annotation. + +After roughly two minutes check that a corresponding DNS record for your Ingress was created. + +```bash +gcloud dns record-sets list \ + --zone "example-com" \ + --name "server.example.com." \ +``` +Output: -* A full demo on GKE Kubernetes + CloudDNS + SA-Permissions [How-to Kubernetes with DNS management (ssl-manager pre-req)](https://medium.com/@jpantjsoha/how-to-kubernetes-with-dns-management-for-gitops-31239ea75d8d) -* Run external-dns on GKE with workload identity. See [Kubernetes, ingress-nginx, cert-manager & external-dns](https://blog.atomist.com/kubernetes-ingress-nginx-cert-manager-external-dns/) +``` +NAME TYPE TTL DATA +server.example.com. A 300 130.211.46.224 +server.example.com. TXT 300 "heritage=external-dns,external-dns/owner=my-identifier" +``` + +Let's check that we can resolve this DNS name as well. + +```bash +dig +short @ns-cloud-e1.googledomains.com. server.example.com. +130.211.46.224 +``` + +Try with `curl` as well. + +```bash +curl server.example.com +``` + +### Clean up + +Make sure to delete all Service and Ingress objects before terminating the cluster so all load balancers get cleaned up correctly. + +```bash +kubectl delete service nginx +kubectl delete ingress nginx +``` + +Give ExternalDNS some time to clean up the DNS records for you. Then delete the managed zone and cluster. + +```bash +gcloud dns managed-zones delete "example-com" +gcloud container clusters delete "external-dns" +```