Cluster/context selection and multi-file KUBECONFIG

[thomas-riccardi]

I have access to multiple k8s clusters, they are defined in multiple config files, all pointed-to by the KUBECONFIG env var, separated with ::

$ echo $KUBECONFIG
/home/foo/.kube/config:/home/foo/.minikube/kubeconfig

kubent doesn't support context selection, it seems to use the default context on the first config file, and if it points to a context defined in another file it fails: ERR Failed to initialize collector: <nil> error="invalid configuration: [context was not found for specified context: bar-context, cluster has no server defined]"

Currently I workaround that by setting the current context and edit KUBECONFIG, that's not ideal.

Ideally:

• kubent supports multi-file KUBECONFIG
• kubent supports context selection (technically as an indirect selector for cluster) with --context (alas the standard -c is already used for something else)

[stepanstipl]

Hi @thomas-riccardi, thanks for opening this issue. I assume the "cross-file" context selection is something that works out of the box with kubectl, correct? I will have to check how that is implemented, but the idea is to behave in the same way, so it sounds like something we can look into.

Context selection makes sense. I think kubectl supports this via --context, but also --cluster and --user, flags. So probably something we can add (or at least context for a start).

[thomas-riccardi]

Hi @stepanstipl

I assume the "cross-file" context selection is something that works out of the box with kubectl, correct?

Yes; that's indeed a little bit surprising for k8s tools developers but it works transparently on kubectl.
It probably merges all yaml files into one config object (concat all arrays (clusters, contexts, users) (with dedup? error on conflict?) except current-context, which seems to be taken from the first file), then work from that as usual.
kubectl config view shows the parsed/merged config.

Context selection makes sense. I think kubectl supports this via --context, but also --cluster and --user, flags. So probably something we can add (or at least context for a start).

Personally I only use --context as --cluster probably also requires --user: slower to type. So starting with --context would be perfect for me :), and hopefully not just me.

[ryan-dyer-sp]

Just stumbled across this tool as something to investigate, but we also have the same issue; multiple kubeconfig files.

[stepanstipl]

@thomas-riccardi @ryan-dyer-sp Support for both multiple kubeconfig files and selecting context has now been added and is part of the latest 0.5.0 release. Please give it a go and let me know if there are any issues. Cheers 👍

[ryan-dyer-sp]

@stepanstipl Can confirm it works. Thanks!

[thomas-riccardi]

@stepanstipl I confirm this works in our setup, great with kubectx and explicit --context option, thx!

(though 0.5.0 broke --additional-kind: #204)