Ingress route rule compatibility

[kyessenov]

Tracking some issues with mismatch between ingress and route rule:

• ingress consists of many routes; do we map ingress to a list of rules or make route rule a list?
• ingress points to a port name on the backend service; do we put port names into the core model?
• defaulting is different for ingress proxy; it always replies with 404 and doesn't use sidecar default route rules.

[zcahana]

Regarding 1st item: Ingresses in kubernetes do not define any relation (i.e. precedence) between multiple routes/virtual hosts specified within the same ingress resources. Therefore, we don't "lose" anything if mapping a single ingress into an unordered set of route rules. This is the approach I took so far in the ingress controller work.

[rshriram]

And I don't think we need to worry about the third one. Even with a sidecar, we would be responding with a 404 when no routes match. So the semantics are same between ingress and sidecar. Unless you are thinking of default routes to an existing service, e.g. by default go to v1. Since ingress only allows for a single backend per route, that's a bigger problem. @zcahana, does the current controller have the ability to parse annotations ?

On Thu, Feb 16, 2017 at 8:15 AM Zvi Cahana ***@***.***> wrote: Regarding 1st item: Ingresses in kubernetes do not define any relation (i.e. precedence) between multiple routes/virtual hosts specified within the same ingress resources. Therefore, we don't "lose" anything if mapping a single ingress into an unordered set of route rules. This is the approach I took so far in the ingress controller work. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#151 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0qd_p7YOPD4lsEt05wccJ3BIGbESLuks5rdEv-gaJpZM4MCK1C> .

-- ~shriram

[zcahana]

Regarding 2nd item: adding port (numbers and/or names) into the model (e.g., under WeightedDestination seems to be the only viable way to capture user intent expressed in ingress resources. Aside from ingress rules, this can be used for rerouting traffic transparently between services (i.e., client addresses service-a:80, which is rerouted to service-b:8080 by the proxy).

[zcahana]

Regarding 3rd item: ingresses allows to specify a default backend for requests that doesn't match any route. An ingress controller needs to honor that specification. If not specified, we can do anything we dim fit (reply with 404, use sidecar default route rules [what are these anyway], ...). My thinking is that 404 is the proper response for ingress traffic which do not meet any explicit user-provided routes.

[kyessenov]

I think we need to reconsider if a route rule spec is sufficient for ingress controller. It is not clear how to associate TLS certs, which are bound to specific hostnames, with a route rule or a destination policy. The hostname in ingress spec does not match the destination fields in route rules and policies. Moreover, the destination field in the ingress backend (service name) could benefit from applying route rules to it, to apply client side traffic shifting at the ingress proxy instead of standing up a separate proxy.

Maybe we should make ingress an explicit model element, like the route rule?
Do we lose any features that we get for free if we treat ingress rule like other route rules?

@zcahana @rshriram WDYT?

[rshriram]

Why do we need to associate tls certs with routes or policies?

On Mon, Mar 20, 2017 at 5:38 PM Kuat ***@***.***> wrote: I think we need to reconsider if route rule is sufficient for ingress controller. It is not clear how to associate TLS certs, which are bound to specific hostnames, with a route rule or a destination policy. The hostname in ingress spec does not match the destination fields in route rules and policies. Moreover, the destination field in the ingress backend (service name) could benefit from applying route rules to it, to apply client side traffic shifting at the ingress proxy instead of standing up a separate proxy. Maybe we should make ingress an explicit model element, like the route rule? Do we lose any features that we get for free if we treat ingress rule like other route rules? @zcahana <https://github.com/zcahana> @rshriram <https://github.com/rshriram> WDYT? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#151 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0qdw1W47FNq0OWdNMD0B3Rg45JcMPrks5rnxwXgaJpZM4MCK1C> .

-- ~shriram

[ijsnellf]

@rshriram: The Kubernetes ingress spec allows different TLS certs to be used based on host: https://github.com/kubernetes/kubernetes/blob/9497139cb62015905ba5b3d11836f2b0c117ff80/pkg/apis/extensions/types.go#L604

Currently, we map Kubernetes Ingress rules to Istio rules and we need some way to convey that information.

[zcahana]

The hostname in ingress spec does not match the destination fields in route rules and policies.

Don't think it's a problem. We use the route rule's headers matching map to match on the host from ingress spec.

The destination field in the ingress backend (service name) could benefit from applying route rules to it

Definitely (#217). Does separating ingress rules and route rules will get us closer to that? I was thinking this could be somehow achieved by the ingressWatcher processing both kind of rule kinds.

One incompatibility that we do have today is that we need to pass the target service port from the ingress spec to the ingress watcher. Today we do that uncleanly via the destination tags.

As for TLS, won't we need some way to specify certs location in the model, regardless of ingress?
Maybe we need to extend the model with some CertificateStore.

[kyessenov]

For beta, we should define a unified ingress rule definition that covers k8s/extended ingress controllers/Istio use cases. It's clear now that we need a unified model to combine various sources of ingress definitions (k8s, deployment annotations, internal istio) that specifically covers edge traffic concerns (TLS, url/host rewriting) and that plays well with in-cluster routing.

@rshriram @zcahana

[rshriram]

The ingress is going nowhere and its a PITA. We should get rid of ingress and create our own front proxy using LB ports. Besides, @kyessenov has managed to combine ingress spec with route rules for most part. Close?