when primary upstream goes down all queries return REFUSED

[scarby]

I have installed k8s_gateway into my k8s cluster using the helm chart.

In the happy times everything is good, however if for some reason my upstream DNS (which is also in cluster) dies (in my case it had a failure pulling the image) the service responds with REFUSED and does not recover.

this is despite having the following in extraZonePlugins:

  cache 30
  rewrite edns0 subnet set 32 64
  forward . 192.168.88.51 8.8.8.8 8.8.4.4 1.1.1.1 {
    policy sequential
    max_fails 1
    health_check 1s domain google.com
  }

my understanding is that this should cause queries to be passed to 8.8.8.8 should 192.168.88.51 be down? Can anyone offer some insight here?

[networkop]

my understanding is that this should cause queries to be passed to 8.8.8.8 should 192.168.88.51 be down?

yes, this is my understand as well.
AFAIK, there should be no interaction between the k8s_gateway and forward plugins. Is the behaviour any different with the upstream coredns plugin? e.g. what happens if you have the same config in your internal coredns?