[nginx] Support for multiple namespaces in Kubernetes Clusters with RBAC

[sbueringer]

Hi,

related to #732 (sorry for the new issue, I wasn't able to reopen the old one)

Currently it is possible to use Ingresses either from all namespaces or from one individual namespace (--force-namespace-isolation). In our environment we only have the rights to access all namespaces except kube-system, so it's not possible to retrieve Ingresses cluster-wide.

It would really help if it would be possible to specify a list of namespaces from which the Nginx Ingress Controller retrieves Ingress. The only alternative for us, would be to deploy one Ingress Controller per namespace.

The traefik guys just implemented this feature 1895. They actually create informers per namespace on startup. I'm aware that this is the solution declined in #732 .

I would propose the following solution:

The nginx ingress controller currently has the following flags:

watch-namespace
force-namespace-isolation
Change watch-namespace to an array (or add an additional flag to set multiple namespaces)

If force-namespace-isolation is not set:

Create cluster-wide informers
Filter the results locally if watch-namespace(s) is set
If force-namespace-isolation is set:

Create namespace-wide informers for every namespace set per watch-namespace(s)
So the user can actually choose if he wants the performance benefits of cluster-wide informers, although cluster-wide informers should receive way more events than namespace-wide informers. But if the user deploys to a kubernetes cluster with several tenants, he can also choose to just create informers for namespaces he has access to.

I think this is a common use case, if you deploy in a kubernetes cluster with several tenants, where not every tenant has cluster-wide access to objects like secrets. Moreover, it doesn't really scale in this scenarios to create an own ingress controller vor every namespace.

P.S. If there is a way forward with the proposed solution, I'm obviously willing to contribute.

[aledbf]

The traefik guys just implemented this feature 1895. They actually create informers per namespace on startup. I'm aware that this is the solution declined in #732 .

Please check my comments in 732. Also please check the comment traefik/traefik#1895 (comment)
This makes debugging harder than should be.
Please try to see this from a maintainer perspective.

The only alternative for us, would be to deploy one Ingress Controller per namespace.

This is the correct way to handle this scenario.

[redbaron]

@aledbf , in #732 (comment) you gave reason for not having multiple namespaces because it would be inefficient in consuming resources from APIServer. Here you advice to run nginx-ingress controller per namespace.

Adding them all together I can't see how these 2 pieces of advice match: running ingress controller per namespace still consumes extra resources from APIserver, as each runs Watcher. In fact it is making it even worse, although total number of watchers stay the same, they are spread across more ingress pods, which are not free.

With new namespaces shared informers in go-client 6.0.0 available, would you reconsider your decision not to support multiple namespaces natively?

[aledbf]

With new namespaces shared informers in go-client 6.0.0 available, would you reconsider your decision not to support multiple namespaces natively?

I need to see exactly what's supported. If the new shared informer provides support for multiple namespaces off course I am open to add that feature.

[redbaron]

No, it just removes cruft and makes it easier to fire informer watching a single namespace. So you'd still need to have an informer per namespace.

My point is, not supporting this feature doesn't make it any more efficient on API server, users running ingress controller per namespace consume create number of watchers as if feature was supported natively.

[gpanagiotidis]

@redbaron Would it be possible to iterate over different namespaces and create a new ListWatch, informer and thus a controller per namespace, running in parallel?