Allow different ingress controllers to coexist

[bprashanth]

We need to be able to run cloud and non-cloud ingress controllers simultaneously, in the same cluster. Current they will fight it out. It's also difficult to turn off the cluster addon ingress controller and deploy one of your own on something like GKE, even though the api encourages extensibility.

A couple of options:

1. User create a claim which translates 1:1 to an lb, Ingresses join claims. This was outlined in L7 Loadbalancing  #12827 but is significantly more work than other options.
2. Use annotations for mutual exclusion. To support the case of a user supplied controller, existing controllers must ignore ingresses without their annotation.
3. Don't run any controller as a cluster addon, force them to deploy exactly one.

This is a usability problem because it keeps confusing people, and makes things like automatic certificate acquisition harder.

[bprashanth]

We currentld do 2 through the badly named ingress.class annotation: https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md#disabling-glbc
There's a proposal for 1: #30151