DNS entries managed by BPKR (on AKS) aren't refreshed

[sameersbn]

I created two AKS clusters in the same Resource Group and followed the private beta guide to deploy BKPR and wordpress in each of these clusters. The two clusters were setup correctly and I was able to access the logging and monitoring dashboards as well as the wordpress app.

However after a few hours I noticed that the DNS records for the kibana, prometheus and wordpress subdomains of the first cluster vanished from the DNS zone.

Looking at the logs of the external-dns pod in the two clusters it appears each is one is deleting the entries made by the other which explains why the records are missing from the zone. Note the two clusters were setup to use different subdomains of the same top-level domain.

Logs of the external-dns container for the first cluster

time="2018-07-05T04:55:52Z" level=info msg="Deleting A record named 'blog' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T04:55:53Z" level=info msg="Deleting A record named 'kibana' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T04:55:53Z" level=info msg="Deleting A record named 'prometheus' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T04:55:53Z" level=info msg="Deleting TXT record named 'blog' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T04:55:53Z" level=info msg="Deleting TXT record named 'kibana' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T04:55:53Z" level=info msg="Deleting TXT record named 'prometheus' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T04:55:54Z" level=info msg="Updating A record named 'blog' to '40.121.61.194' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T04:55:54Z" level=info msg="Updating A record named 'kibana' to '40.121.61.194' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T04:55:54Z" level=info msg="Updating A record named 'prometheus' to '40.121.61.194' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T04:55:54Z" level=info msg="Updating TXT record named 'blog' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/blog.one.my-domain.com-blog\"' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T04:55:55Z" level=info msg="Updating TXT record named 'kibana' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/kube-system/kibana-logging\"' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T04:55:55Z" level=info msg="Updating TXT record named 'prometheus' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/kube-system/prometheus\"' for Azure DNS zone 'one.my-domain.com'."

Logs of the external-dns container for the second cluster

time="2018-07-05T05:00:10Z" level=info msg="Deleting A record named 'blog' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T05:00:10Z" level=info msg="Deleting A record named 'kibana' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T05:00:11Z" level=info msg="Deleting A record named 'prometheus' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T05:00:11Z" level=info msg="Deleting TXT record named 'blog' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T05:00:11Z" level=info msg="Deleting TXT record named 'kibana' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T05:00:11Z" level=info msg="Deleting TXT record named 'prometheus' for Azure DNS zone 'one.my-domain.com'."
time="2018-07-05T05:00:12Z" level=info msg="Updating A record named 'prometheus' to '40.76.205.98' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T05:00:12Z" level=info msg="Updating A record named 'blog' to '40.76.205.98' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T05:00:12Z" level=info msg="Updating A record named 'kibana' to '40.76.205.98' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T05:00:12Z" level=info msg="Updating TXT record named 'prometheus' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/kube-system/prometheus\"' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T05:00:13Z" level=info msg="Updating TXT record named 'blog' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/default/blog.two.my-domain.com-blog\"' for Azure DNS zone 'two.my-domain.com'."
time="2018-07-05T05:00:13Z" level=info msg="Updating TXT record named 'kibana' to '\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/kube-system/kibana-logging\"' for Azure DNS zone 'two.my-domain.com'."

[anguslees]

Thanks for the confirmation @sameersbn. I believe the solution is to add --txt-owner-id $unique_string to external-dns flags (which affects the "owner" value in the TXT records).