proposal: Applications outside argocd namespace #6409

jannfis · 2021-06-05T13:45:45Z

This is a proposal to enable Application resources to exist outside the Argo CD control plane's namespace.

Should be considered an alternative to #6405

Related issues:

application resource can be applied in any namespace #3474

Signed-off-by: jannfis jann@mistrust.net

Checklist:

Signed-off-by: jannfis <jann@mistrust.net>

codecov · 2021-06-05T13:56:55Z

Codecov Report

Merging #6409 (a50f20d) into master (c0f16fb) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6409   +/-   ##
=======================================
  Coverage   45.98%   45.98%           
=======================================
  Files         226      226           
  Lines       27413    27413           
=======================================
  Hits        12605    12605           
  Misses      13094    13094           
  Partials     1714     1714

Impacted Files	Coverage Δ
util/settings/settings.go	`51.42% <0.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Signed-off-by: jannfis <jann@mistrust.net>

sbose78 · 2021-06-17T13:54:06Z

docs/proposals/003-applications-outside-argocd-namespace.md

+## Open Questions [optional]
+
+* The major open question is, how to name `Application`s in a scenario where
+  the K8s resource's name isn't unique anymore.


K8s resource's name isn't unique anymore.

Is this the resource that the user is deploying from her git repo?

This means the name of the Application resource, whether created from Git or in another way. Where as before uniqueness was ensured by Kubernetes because no two Applications could share the same name in the same namespace, now possibly an application named foo could exist in namespace ns-foo, and another application named foo could exist in namespace ns-bar.

sbose78 · 2021-06-17T16:33:21Z

I've gone through the whole proposal and it looks like a fairly simple enhancement to the existing paradigm!

jannfis · 2021-09-02T10:56:50Z

@jessesuen @alexmt Can we please revisit this topic? I think it'd be an important feature to have.

tiagomeireles · 2021-09-15T18:39:52Z

FWIW I really look forward to seeing this ship.

bloodsper · 2021-11-03T20:21:36Z

looking for this feature

it makes "apps of apps" such a mess if want to have tenant namespaces but having the application definition elsewhere from the actual deployments

m-yosefpor · 2021-11-04T23:38:57Z

Thanks. This is indeed a great feature which we are looking for. Right now we are trying to do a similar thing with a ton of hacks:

We have written a k8s controller to watch for Application CRs in all namespaces except for argocd ns, and create a new corresponding app CR in argocd ns (using parts of uid for avoiding duplication). Also we have written a validation webhook to enforce and reject App CRs created in other ns, by checking and validating the .spec.project of apps based on a a label on namespaces which identifies the argo project, e.g. ns1 with the label argo.snappcloud.io/project: proj1 will cause creating any Application CR with .spec.project != proj1 in ns1 namespace to be rejected.

m-yosefpor · 2021-11-05T00:00:13Z

docs/proposals/003-applications-outside-argocd-namespace.md

+  field to associate themselves to the `AppProject` named `some-project`.
+  In the above example, the Application `some-app` would be allowed to associate
+  to the AppProject `some-project`, but the Application `other-app` would be
+  invalid.


One thing that is not clear in this section is that how do you define invalid? In the implementation details you have mentioned aborting reconcile, but it is not clear to users and is only discoverable in the controller logs. Should it be rejecting via a validation webhook? or letting the creation to happen, and introduce a new status for Application as Invalid and maybe also have some events?

I think the former might be more clear and less error prone for users, especially as rejection can happen with a clear message, while the latter requires users to notice that the app status is invalid and search for the reason in the events or in .status.description.

Also it is unclear in this proposal how updates to appProj is handled (e.g. when we revoke acccess with removing some soureNamespaces)? I think we should immediately recheck all affected applications (all apps created in sourceNamespaces section of updated appproj) and update their status to invalid if required.

The downside of validationwebhook is that, when an app is created in a ns which initially was allowed in sourceNamespaces, we can not notify users that it is no longer valid, and the reconcile will only fail and might confuse them.

One thing that is not clear in this section is that how do you define invalid? In the implementation details you have mentioned aborting reconcile, but it is not clear to users and is only discoverable in the controller logs.

This would be reflected in the Application's .status field, similar to how a failed sync is reflected right now. Since it's the application controller who is updating the .status of an Application, it's also the application controller who will be enforcing whether an application will be reconciled or not. This check is happening constantly before reconciling any app, so when you remove a source namespace from the AppProject, it would not be allowed to sync anymore.

m-yosefpor · 2021-11-05T00:04:50Z

docs/proposals/003-applications-outside-argocd-namespace.md

+  ```
+
+* `Applications` created imperatively (e.g. through Argo CD API via UI or
+  CLI) will keep being created in the control plane namespace, and Argo CD


So what if users define apps in new ns, in this case they can create with kubectl -f file.yaml and they are not able to create the same manifest with argocd app create -f file.yaml right? wouldn't it be confusing?

I think the corresponding command would then be argocd app create -n <namespace> -f file.yaml (or specfiying the target namespace in the YAML). The API would then show this application as <namespace>/<appname> instead of just <appname>.

m-yosefpor · 2021-11-05T00:09:19Z

docs/proposals/003-applications-outside-argocd-namespace.md

+of the second application would become `barns/some-app`. This would have the
+advantage to instantly have a clue about _where_ the application is coming from,
+without having to resort to the Kubernetes API to inspect `.spec.namespace`
+field of the app. The notation may also use a hyphen (`-`) or underscore (`_`)


a - should not be used as it is allowed in k8s names. See the two following conflicting examples:

name: hello namespace: hi-all --- name: hello-hi namespace: all

both will be hello-hi-all if separator be a dash.

Good catch. Thinking about this for a while :) Will come back to it later.

jannfis · 2021-11-12T09:53:16Z

Thanks @m-yosefpor for reviewing and picking up the discussion.

alexmt

LGTM

We've been going back and forth on this proposal for many months. Recently all maintainers agreed this should be implemented.

I think we should merge proposal,start implementing it and handle implementation details as we go.

@jessesuen , @jannfis , @wtam2018 let me know if you have any objections please.

m-yosefpor · 2022-03-10T23:31:13Z

LGTM

We've been going back and forth on this proposal for many months. Recently all maintainers agreed this should be implemented.

I think we should merge proposal,start implementing it and handle implementation details as we go.

@jessesuen , @jannfis , @wtam2018 let me know if you have any objections please.

I have commented some notes on this proposal. @jannfis has replied them with valid answers. However I think it would be great if those answers are somehow mentioned in the proposal as well.

Also I think the hyphen separator mentioned in #6409 (comment) should be deleted due to described duplication error.

mkaiserincomm · 2022-04-07T16:05:02Z

Will RBAC be addressed? When I create an ArgoCD Application via the UI, I am constrained by the RBAC rules effecting my user. If I create an ArgoCD application by creating a YAML for an Application object and applying, it will not know my ArgoCD user and I suspect will not enforce RBAC. My use case is that multiple teams share a cluster. I want teams to be able to use the App of Apps pattern, but I want them to be restricted into only creating applications within the projects their RBAC allows and only deploying to clusters which their RBAC allows.

Avni-Sharma · 2022-06-17T19:44:08Z

Hi all, Was just curious and wanted to know where are we on this?

Paulo-B · 2022-07-12T15:55:08Z

Hello, any news when this feature will be released?

crenshaw-dev · 2022-07-13T02:04:27Z

@Paulo-B it's expected for 2.5 in August. The PR is under review.

FireDrunk · 2022-07-28T09:22:54Z

@jannfis I very much agree with the concerns of @mkaiserincomm. There is a significant risk involved if people can create Applications linking to any project in their own namespace.
This would grant them permissions by-proxy that they should not have according to the RBAC rules.

I think that some explicit configuration is required to allow this.
One idea would be to only allow Applications in namespaces that are explicitly linked to Projects that explicitly contain configuration to read Application objects from that namespace.

For example:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: cicd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  description: Project CICD owned by Team CICD

  destinations:
    - namespace: 'cicd'
      allowApplications: true
      server: 'https://kubernetes.default.svc'
    - namespace: 'cicd-*'
      allowApplications: false #default
      server: 'https://kubernetes.default.svc'

  sourceRepos:
    - '*'

This configuration would trigger ArgoCD to read any Application CRD's from the cicd namespace, and explicitly 'parent' them into the CICD project regardless of the project property in the Application Resource.

jannfis · 2022-07-28T10:40:39Z

@FireDrunk This problem is solved with the new sourceNamespaces property in the AppProject. This defines a list of namespaces where Applications are allowed to exist in order to associate to the given project.

For example, with an AppProject foobar that has

spec:
  sourceNamespaces:
  - ns-foo
  - ns-bar

Then an Application created in either namespace ns-foo or ns-bar may set their own .spec.project to foobar, thus associating to the project. If an Application is created in namespace ns-baz and has .spec.project set to foobar, the application controller will not reconcile this application.

FireDrunk · 2022-07-28T12:25:35Z

@FireDrunk This problem is solved with the new sourceNamespaces property in the AppProject. This defines a list of namespaces where Applications are allowed to exist in order to associate to the given project.

For example, with an AppProject foobar that has
spec:
  sourceNamespaces:
  - ns-foo
  - ns-bar
Then an Application created in either namespace ns-foo or ns-bar may set their own .spec.project to foobar, thus associating to the project. If an Application is created in namespace ns-baz and has .spec.project set to foobar, the application controller will not reconcile this application.

That looks like a perfect alternative! I would be happy to test this functionality, since we are anxiously awaiting these kinds of improvements!

Is there any kind of 'event' or notification stream or any kind of monitoring solution to make it easy to find these kinds of mishaps?

crenshaw-dev · 2022-07-28T13:31:10Z

(The PR for testing if you want @FireDrunk :-) #9755)

Signed-off-by: jannfis <jann@mistrust.net>

…plications-namespaces

jannfis · 2022-08-08T16:40:44Z

The PR #9755 implementing this proposal will be merged soon eventually.

* proposal: Applications outside argocd namespace Signed-off-by: jannfis <jann@mistrust.net> * Typos and add another use-case Signed-off-by: jannfis <jann@mistrust.net> * Update 003-applications-outside-argocd-namespace.md Signed-off-by: jannfis <jann@mistrust.net>

proposal: Applications outside argocd namespace

0617385

Signed-off-by: jannfis <jann@mistrust.net>

jannfis requested review from alexmt, jessesuen, jgwest, kshamajain99, sbose78 and mayzhang2000 June 5, 2021 13:46

Typos and add another use-case

eb0eec9

Signed-off-by: jannfis <jann@mistrust.net>

sbose78 reviewed Jun 17, 2021

View reviewed changes

This was referenced Jun 21, 2021

application resource can be applied in any namespace #3474

Closed

feat: (PoC) Applications outside argocd namespace #6537

Closed

jannfis mentioned this pull request Oct 4, 2021

docs: update v2.3+ roadmap #7353

Merged

lebenitza mentioned this pull request Oct 26, 2021

ArgoCD deleted all the namespaces/apps which it manages #6046

Closed

m-yosefpor reviewed Nov 5, 2021

View reviewed changes

jannfis mentioned this pull request Nov 12, 2021

Impersonation of destination ServiceAccounts, Users, Groups #7689

Open

jemag mentioned this pull request Dec 21, 2021

Locking "App Of Apps" to a single project and namespace #2785

Closed

alexmt approved these changes Mar 10, 2022

View reviewed changes

jannfis mentioned this pull request Jun 22, 2022

feat: Applications in any namespace #9755

Merged

6 tasks

jannfis added 2 commits August 8, 2022 08:48

Update 003-applications-outside-argocd-namespace.md

6f1e391

Signed-off-by: jannfis <jann@mistrust.net>

Merge branch 'master' of github.com:argoproj/argo-cd into proposal/ap…

a50f20d

…plications-namespaces

jannfis force-pushed the proposal/applications-namespaces branch from 3014451 to a50f20d Compare August 8, 2022 08:50

jannfis merged commit ca08f4d into argoproj:master Aug 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

proposal: Applications outside argocd namespace #6409

proposal: Applications outside argocd namespace #6409

jannfis commented Jun 5, 2021 •

edited

codecov bot commented Jun 5, 2021 •

edited

sbose78 Jun 17, 2021

jannfis Jun 17, 2021 •

edited

sbose78 commented Jun 17, 2021

jannfis commented Sep 2, 2021

tiagomeireles commented Sep 15, 2021

bloodsper commented Nov 3, 2021

m-yosefpor commented Nov 4, 2021

m-yosefpor Nov 5, 2021 •

edited

jannfis Nov 12, 2021

m-yosefpor Nov 5, 2021

jannfis Nov 12, 2021

m-yosefpor Nov 5, 2021

jannfis Nov 12, 2021

jannfis commented Nov 12, 2021

alexmt left a comment

m-yosefpor commented Mar 10, 2022

mkaiserincomm commented Apr 7, 2022

Avni-Sharma commented Jun 17, 2022

Paulo-B commented Jul 12, 2022 •

edited

crenshaw-dev commented Jul 13, 2022 •

edited

FireDrunk commented Jul 28, 2022

jannfis commented Jul 28, 2022

FireDrunk commented Jul 28, 2022

crenshaw-dev commented Jul 28, 2022

jannfis commented Aug 8, 2022

proposal: Applications outside argocd namespace #6409

proposal: Applications outside argocd namespace #6409

Conversation

jannfis commented Jun 5, 2021 • edited

codecov bot commented Jun 5, 2021 • edited

Codecov Report

sbose78 Jun 17, 2021

Choose a reason for hiding this comment

jannfis Jun 17, 2021 • edited

Choose a reason for hiding this comment

sbose78 commented Jun 17, 2021

jannfis commented Sep 2, 2021

tiagomeireles commented Sep 15, 2021

bloodsper commented Nov 3, 2021

m-yosefpor commented Nov 4, 2021

m-yosefpor Nov 5, 2021 • edited

Choose a reason for hiding this comment

jannfis Nov 12, 2021

Choose a reason for hiding this comment

m-yosefpor Nov 5, 2021

Choose a reason for hiding this comment

jannfis Nov 12, 2021

Choose a reason for hiding this comment

m-yosefpor Nov 5, 2021

Choose a reason for hiding this comment

jannfis Nov 12, 2021

Choose a reason for hiding this comment

jannfis commented Nov 12, 2021

alexmt left a comment

Choose a reason for hiding this comment

m-yosefpor commented Mar 10, 2022

mkaiserincomm commented Apr 7, 2022

Avni-Sharma commented Jun 17, 2022

Paulo-B commented Jul 12, 2022 • edited

crenshaw-dev commented Jul 13, 2022 • edited

FireDrunk commented Jul 28, 2022

jannfis commented Jul 28, 2022

FireDrunk commented Jul 28, 2022

crenshaw-dev commented Jul 28, 2022

jannfis commented Aug 8, 2022

jannfis commented Jun 5, 2021 •

edited

codecov bot commented Jun 5, 2021 •

edited

jannfis Jun 17, 2021 •

edited

m-yosefpor Nov 5, 2021 •

edited

Paulo-B commented Jul 12, 2022 •

edited

crenshaw-dev commented Jul 13, 2022 •

edited