Create just one loadbalancer for cross-namespace ingresses

[sheerun]

Hello,

In my setup I create per-namespace apps and I have ingress for each of these (Azure, ACS), then I had single traefik ingress controller in default namespace handling all of these.

Now, when I install Voyager, it creates a loadbalancer service in each namespace. This is not optimal as it allocates new IP per-application while I want to share one IP for all applications. Doing something described in cross-namespace traffic routing doesn't help as it still creates ingress controller per-application, but in default namespace:

voyager-app-2226907767-j5b3t          1/1       Running   0          4h
voyager-test-3401873822-w08qn         1/1       Running   0          21m
voyager-pigeon-2702276977-rkd7l       1/1       Running   0          21m
voyager-segmenter-1243905025-n821d    1/1       Running   0          21m
voyager-summarizer-2048425227-rsbxv   1/1       Running   0          21m

Also it's not perfect as I'd like to create ingress resources in the namespaces of applications, not in the default namespace.

Is it possible to configure voyager so it creates just one ingress service in default namespace and handles all ingresses through it? I don't want to specify loadbalancer ip address upfront.

[tamalsaha]

Voyager creates a separate LB service (IP) for each Ingress object by design. We believe this gives users more control over the ingress decision. Please see prior discussions here; #331

[sheerun]

Please understand that each service creates new ip address for each of my apps. IP addresses are scarce and I don't have money for one loadbalancer per microservice :) What if I scale to 30 microservices? I don't want to allocate 30 IPs

[sheerun]

I could get around this by creating just one ingress and referencing all apps there, but I wanted to ship each app in isolation. i.e. ship ingress.yaml next to deployment.yaml for each app. Also with this approach I'm unable to target specific rules with voyager annotations as they apply to the whole ingress and all rules in it (and in my case I'd have just one ingress).

In short, I prefer many ingress resources (so I can apply RBAC on them), but an annotation that would specify which service voyager should connect it to (and optionally create) e.g:

annotations:
  ingress.appscode.com/service: voyager.default

So the other way around than you currently recommend.

The default could be to create separate service per-ingress, no problem

[tamalsaha]

Currently your only option is to add them in the same YAML and then point to service in different namespace. Feature like #600 will help ease this issue.

[zentron]

There have been a bunch of closed tickets around managing multiple services in other ways other than a single massive ingress file.

We are planning on running multiple services in our cluster, where new service could be added at any time as new customers come/go. Maintaining a single ingress file causes some concerns around things like concurrency (if we add multiple customers at once, we need to ensure there is some sort of lock so that only one update is made at a time since we would need to retrieve the "current state" from somewhere, insert the new rules and update).

Annotations would be a really nice approach but it was closed as a duplicate of this ticket. Does this mean annotations are still on the table? Combining ingress objects or templates would both be other options. What direction are you thinking you will take?

There are a few other providers out there which provide a more dynamic approach to configuring these rules and i'd be great if voyager had some of this flexibility.

[giovannicandido]

One important use case is on automated deployments. Gitlab CI, can create ingress configs and this is per application deployment (or per namespace). A single point of traffic (or multiple HA but static) is preferable because of the points already mentioned (cost, and maintenance).

Also in this use case is not practical to maintain one ingress file.

[oliver-ni]

Has there been any updates since? I don't want to put everything into a single file either, and I'd rather not pay for multiple load balancers if I do not need.