Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flux CLI tool doesn't respect context namespace #3453

Open
1 task done
sharkymcdongles opened this issue Dec 30, 2022 · 9 comments
Open
1 task done

Flux CLI tool doesn't respect context namespace #3453

sharkymcdongles opened this issue Dec 30, 2022 · 9 comments

Comments

@sharkymcdongles
Copy link

Describe the bug

Say a Kubeconfig has the following block:

contexts:
- context:
    cluster: cluster
    namespace: services
    user: user
  name: cluster
current-context: cluster

When you run flux commands it will not target the namespace your context is currently using. It always targets flux-system unless the namespace flag is specified. For continuity sake between all kubernetes cli tools e.g. kubectl and helm the cli should use namespace context for CLI calls.

Steps to reproduce

  1. Set your kubecontext to a namespace containing a helmrelease e.g.
namespace: services
helmrelease: service
  1. Run
    flux reconcile hr service

  2. See this error
    ✗ helmreleases.helm.toolkit.fluxcd.io "service" not found

  3. Run
    flux reconcile hr service --namespace service

It works

Expected behavior

  1. Set your kubecontext to a namespace containing a helmrelease e.g.
namespace: services
helmrelease: service
  1. Run
    flux reconcile hr service
    it works

Screenshots and recordings

No response

OS / Distro

Fedora

Flux version

v0.38.2

Flux check

N/A

Git provider

No response

Container Registry provider

No response

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@kingdonb
Copy link
Member

kingdonb commented Jan 1, 2023

I agree with this report in principle, however I'd like to note as a point of information that this is not a regression. This is how flux CLI has always worked. You can use the FLUX_SYSTEM_NAMESPACE environment variable to set flux CLI to work in a different namespace, but, generally, it is anticipated that on most clusters, most or all Flux resources will remain in the flux-system namespace, or at least that Flux resources will likely be centralized within a single namespace (eg. dev-team) for any given tenant.

You can use targetNamespace to target resources for apply into a different namespace. Otherwise, the flux cli only switches namespace when you pass it an -n – this diverges from the kubectl behavior in that we do not honor current-context in any way in the flux CLI. Maybe it should not diverge this way?

I think it would be very inconvenient for most users if the default flux-system namespace pin were removed, given understanding this likely constraint on a single-tenant system. It's hard to say whether most Flux installations will be single or multi-tenant installations at large.

@sharkymcdongles
Copy link
Author

@kingdonb I can better understand the way most people use flux from your perspective, and what you said makes perfect sense.

Perhaps there could be an environment variable for following the current context or not as a compromise? We do keep most resources within flux-system e.g. helmcharts, helmrepos, kustomizations, gitrepos etc., but having helmreleases in the intended namespace makes the most sense for how we interact with flux because if a helmrelease is having issues it's nice to be able to see the helm installation via helm commands to fix it. This is why having context follow would be nice since most of our flux interactions involve working with helmreleases.

@kingdonb
Copy link
Member

Thanks for that added context,

an environment variable for following the current context or not as a compromise

That sounds like a good compromise! Only, flux already follows the context, it's just namespace that isn't followed.

If FLUX_FOLLOW_CURRENT_NAMESPACE is true, then flux get ... will always use the namespace from kubectl's context – does that sound like it would help resolve the issue? (Does FOLLOW sound like the right verb here, any other feedback about this as a variable name?)

@sharkymcdongles
Copy link
Author

@kingdonb I think FOLLOW or INHERIT make sense. Either that or maybe FLUX_INHERIT_CONTEXT_NAMESPACE. Maybe say CONTEXT instead of CURRENT because the namespace comes from current context.

@kingdonb
Copy link
Member

kingdonb commented Jan 11, 2023

The rubyist in me wants every name to be as close to a complete and fully expressive sentence as possible, now it's clear the concept is a bit obscure; I don't think inherit quite captures what we're saying, (and I'm afraid of using big/overloaded words like inherit that somebody may incorrectly infer that this has something to do with object oriented inheritance.)

FLUX_NAMESPACE_FOLLOWS_KUBE_CONTEXT=true
or how about FLUX_NS_FOLLOW_KUBECONTEXT maybe?

Not to put the cart in front of the horse either, let's figure out who can contribute the PR for this and whether it could merge, or if there's an even better option I haven't considered, before I spend too much time on what color to paint the bikeshed 🙃

@sharkymcdongles
Copy link
Author

@kingdonb honestly I don't really mind what it is called so long as it is documented and I can use it.

I was just giving some ideas. I think either of your suggestions works for me. <3

@makkes
Copy link
Member

makkes commented Aug 8, 2023

I took a stab at implementing this today as this feature might be especially helpful in multi-tenant scenarios where each tenant has their Flux resources in a dedicated Namespace, different from the default of "flux-system".

What makes this incredibly hard to do is that the upstream library we use has no (easy) way to determine if the user explicitly requested a certain Namespace or not where in the latter case we would default to "flux-system". There is the Namespace() method on ClientConfig but that always returns "default" in case no Namespace is explicitly requested so we can't know if we should override that with "flux-system" or not. 🤔 At the same time the library doesn't provide enough access to its struct fields in which case we could re-implement parts of the Namespace() method ourselves.

Desired order of precedence:

flux-system (default setting) < kubeconfig context < FLUX_SYSTEM_NAMESPACE environment variable < --namespace flag

@makkes
Copy link
Member

makkes commented Aug 8, 2023

Ok, I found a way but it's much more involved than what I have time for right now. If anyone wants to pick my current progress up, here's the commit.

@makkes
Copy link
Member

makkes commented Aug 8, 2023

P.S.: Global variables are a PITA!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants