Ingress with IPv4 and IPv6 IPs results in failure to add records

[samip5]

What happened:
The external-dns reported an error and it's unable to create records because it's trying to create A records only with IPv4 and IPv6 addresses.

What you expected to happen:
I expected it to work properly

How to reproduce it (as minimally and precisely as possible):

1. Deploy cluster in dual-stack mode.
2. Deploy nginx-ingress controller
3. Use calico CNI
4. Set nginx externalIPs to have v4 and v6
5. Try to use the pdns provider

Anything else we need to know?:

time="2021-09-16T01:48:32Z" level=debug msg="UPDATE-NEW: traefik.<snip>.fi 0 IN A  185.132.53.251;2001:67c:1104:fc00::4443 []"
time="2021-09-16T01:48:32Z" level=debug msg="UPDATE-NEW: traefik.<snip>.fi 0 IN TXT  \"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/networking/traefik-dashboard\" []"
time="2021-09-16T01:48:32Z" level=debug msg="Zone List generated from Endpoints: [{Id:<snip>.fi. Name:<snip>.fi. Type_: Url:/api/v1/servers/localhost/zones/<snip>.fi. Kind:Master Rrsets:[{Name:traefik.<snip>.fi. Type_:A Ttl:300 Changetype:REPLACE Records:[{Content:185.132.53.251 Disabled:false SetPtr:false} {Content:2001:67c:1104:fc00::4443 Disabled:false SetPtr:false}] Comments:[]} {Name:traefik.<snip>.fi. Type_:TXT Ttl:300 Changetype:REPLACE Records:[{Content:\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/networking/traefik-dashboard\" Disabled:false SetPtr:false}] Comments:[]}] Serial:2021091601 NotifiedSerial:0 Masters:[] Dnssec:false Nsec3param: Nsec3narrow:false Presigned:false SoaEdit: SoaEditApi: ApiRectify:false Zone: Account: Nameservers:[]}]"
time="2021-09-16T01:48:32Z" level=debug msg="Struct for PatchZone:\n{\"id\":\"<snip>.fi.\",\"name\":\"<snip>.fi.\",\"url\":\"/api/v1/servers/localhost/zones/<snip>.fi.\",\"kind\":\"Master\",\"rrsets\":[{\"name\":\"traefik.<snip>.fi.\",\"type\":\"A\",\"ttl\":300,\"changetype\":\"REPLACE\",\"records\":[{\"content\":\"185.132.53.251\",\"disabled\":false},{\"content\":\"2001:67c:1104:fc00::4443\",\"disabled\":false}]},{\"name\":\"traefik.<snip>.fi.\",\"type\":\"TXT\",\"ttl\":300,\"changetype\":\"REPLACE\",\"records\":[{\"content\":\"\\\"heritage=external-dns,external-dns/owner=default,external-dns/resource=ingress/networking/traefik-dashboard\\\"\",\"disabled\":false}]}],\"serial\":2021091601}"
time="2021-09-16T01:48:32Z" level=debug msg="Unable to patch zone Status: 422 Unprocessable Entity, Body: {\"error\": \"Record traefik.<snip>.fi./A '2001:67c:1104:fc00::4443': Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address\"}"
time="2021-09-16T01:48:32Z" level=debug msg="Retrying PatchZone() ... 0"
time="2021-09-16T01:48:32Z" level=debug msg="Unable to patch zone Status: 422 Unprocessable Entity, Body: {\"error\": \"Record traefik.<snip>.fi./A '2001:67c:1104:fc00::4443': Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address\"}"
time="2021-09-16T01:48:32Z" level=debug msg="Retrying PatchZone() ... 1"
time="2021-09-16T01:48:33Z" level=debug msg="Unable to patch zone Status: 422 Unprocessable Entity, Body: {\"error\": \"Record traefik.<snip>.fi./A '2001:67c:1104:fc00::4443': Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address\"}"
time="2021-09-16T01:48:33Z" level=debug msg="Retrying PatchZone() ... 2"
time="2021-09-16T01:48:34Z" level=error msg="Unable to patch zone. Status: 422 Unprocessable Entity, Body: {\"error\": \"Record traefik.<snip>.fi./A '2001:67c:1104:fc00::4443': Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address\"}"
time="2021-09-16T01:48:34Z" level=debug msg="PDNS API response: "
time="2021-09-16T01:48:34Z" level=error msg="Status: 422 Unprocessable Entity, Body: {\"error\": \"Record traefik.<snip>.fi./A '2001:67c:1104:fc00::4443': Parsing record content (try 'pdnsutil check-zone'): unable to parse IP address\"}"

Environment:

• External-DNS version (use external-dns --version): v0.9.0
• DNS provider: pdns
• Others: Dual-stack k3s cluster

[samip5]

Hi @ffledgling, it seems that it's your code that has an assumption that there will ever be one target in endpoints which I have troubleshooted to be the culprit for the error as targets in my case has two, one IPv4 and one IPv6 address.

external-dns/provider/pdns/pdns.go

Lines 309 to 319 in 8cc9522

	if dnsname == zone.Name || strings.HasSuffix(dnsname, "."+zone.Name) {
	// The assumption here is that there will only ever be one target
	// per (ep.DNSName, ep.RecordType) tuple, which holds true for
	// external-dns v5.0.0-alpha onwards
	records := []pgo.Record{}
	for _, t := range ep.Targets {
	if ep.RecordType == "CNAME" {
	t = provider.EnsureTrailingDot(t)
	}
	records = append(records, pgo.Record{Content: t})

[ffledgling]

@samip5 I haven't touched this code in a very long time, but skimming through your PR, I'm a little confused? Why are you trying to force an IPv6 address into an A record? That should be in a AAAA record instead.

The assumption is that there will only ever be one target per unique pair of (DNSName, recordType).

It should still work with both IPv4 and IPv6 because that should be (<your_fqdn>, A) for IPv4 and (<your_fqdn>, AAAA) for your IPv6 address.

I smell something fishy with the way ngnix is setting this up. Can you pastebin your CRD Objects?

[samip5]

@samip5 I haven't touched this code in a very long time, but skimming through your PR, I'm a little confused? Why are you trying to force an IPv6 address into an A record? That should be in a AAAA record instead.

Please look though the PR without skimming it.

It should still work with both IPv4 and IPv6 because that should be (<your_fqdn>, A) for IPv4 and (<your_fqdn>, AAAA) for your IPv6 address.

Current code without my PR changes will not work because it sees an IP and always expects it to be compatible with A records, which IPv6 addresses is not.

I smell something fishy with the way ngnix is setting this up. Can you pastebin your CRD Objects?

There is nothing fishy. Nginx provide a two endpoints/targets which the external-dns code assumes to be suitable for A record when it sees an IP address, even if it's IPv6 address.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[samip5]

/remove-lifecycle stale

[akharlamov]

I have created a PR that allows a temporary resolution of the issue. If --suppress-ipv6 is passed in the command line, IPv6 addresses are filtered out and do not go through to any providers. That workarounds IPv6 incompatible providers for now.

[samip5]

I have created a PR that allows a temporary resolution of the issue. If --suppress-ipv6 is passed in the command line, IPv6 addresses are filtered out and do not go through to any providers. That workarounds IPv6 incompatible providers for now.

I'm sorry but that's not a solution to the issue. Disabling IPv6 is what that effectly does which shouldn't be done.

There's a proper IPv6 support PR #2461

[akharlamov]

Unfortunately, this is not quite correct. I have a pure IPv4 cluster that uses Hetzner Load Balancer. The LBs always have IPv4 and IPv6 addresses. That seems like quite a good option for IPv6 support for me...

Unfortunately, that means the Ingress service has IPv4 and IPv6 external addresses, while the cluster is in IPv4 mode only. The current public release fails to update AWS Route 53 zones in such a situation.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[samip5]

/lifecycle frozen

[jamesharr]

Is this SIG accepting PRs for issues like this? Issue is important to me and it seems like a lot of the PRs are stale / ignored