Fallback to legacy discovery on a wider range of conditions

[Jefftree]

/kind bug

What this PR does / why we need it:

Fallback to legacy discovery if Aggregated Discovery query was unsuccessful. This includes other response codes than 406, and missing/incorrect content-type headers.

Which issue(s) this PR fixes:

Fixes #119662

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Fixed a regression in default 1.27 configurations in kube-apiserver: fixed the AggregatedDiscoveryEndpoint feature (beta in 1.27+) to successfully fetch discovery information from aggregated API servers that do not check `Accept` headers when serving the `/apis` endpoint

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

Please note that we're already in Test Freeze for the release-1.28 branch. This means every merged PR will be automatically fast-forwarded via the periodic ci-fast-forward job to the release branch of the upcoming v1.28.0 release.

Fast forwards are scheduled to happen every 6 hours, whereas the most recent run was: Wed Aug 9 16:32:55 UTC 2023.

[Jefftree]

/assign @liggitt
/cc @apelisse @alexzielenski

[jiahuif]

/triage accepted

[alexzielenski]

mostly lgtm

[liggitt]

I think a switch might still be more readable here

switch {
case writer.respCode == http.StatusNotModified:
  // ...cache case

case writer.respCode == http.StatusServiceUnavailable:
  // ...server unavailable

case writer.respCode == http.StatusOK && discovery.ContentTypeIsGVK(writer.Header().Get("Content-Type"), ...):
  // ...aggregated

default:
  // ...fallback to legacy
}

[Jefftree]

Since discovery.ContentTypeIsGVK(...) returns two values, the condition would need to be checked outside the case statement. Eg:

isGVK, err := discovery.ContentTypeIsGVK(...)
switch {
case ...
case writer.respCode == http.StatusOK && isGVK && err == nil:
// ...aggregated
...
}

Wouldn't an if statement be slightly cleaner in this regard?

[liggitt]

heh, true

[Jefftree]

Changed to a case statement with the GVK check outside of it, lmk if this is readable.

[liggitt]

/priority important-soon

[liggitt]

/kind regression

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Jefftree, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• staging/src/k8s.io/client-go/OWNERS [liggitt]
• staging/src/k8s.io/kube-aggregator/OWNERS [liggitt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment