docs: proposal for sharding application controller

[ishitasequeira]

Signed-off-by: ishitasequeira isequeir@redhat.com

Created proposal doc for sharding application controller to start discussions. I would update the proposal document based on the discussions/comments on this PR.

Note on DCO:

If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• Optional. My organization is added to USERS.md.
• I have signed off all my commits as required by DCO
• I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
• My build is green (troubleshooting builds).

[codecov]

Codecov Report

Merging #8340 (b83baa0) into master (d2b5697) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8340   +/-   ##
=======================================
  Coverage   41.91%   41.91%           
=======================================
  Files         178      178           
  Lines       23003    23003           
=======================================
  Hits         9641     9641           
  Misses      11979    11979           
  Partials     1383     1383           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2b5697...b83baa0. Read the comment docs.

[kvendingoldo]

any news here? idea is amazing!

[todaywasawesome]

@morangoldboim This is something we should do a design review on as scale is something very important to our users. Maybe loop in @itai-codefresh

[ericblackburn]

One of the possible examples for selecting the leader is by checking the load handled by each healthy candidate and selecting the candidate which has the least load / number of processes running on it.

The load question is also a cost to run Argo question, given each application controller pod gets configured with the same cpu/mem request. Unfortunately, solving what it means to load a controller looks like it is non-trivial, a response to this issue.

In our team's setup, the memory usage imbalances is what ends up costing us the most. So, if possible, please consider the success of load distribution with a weighting on its impact for situations such as memory imbalances.

[roeizavida]

Is there any update regarding this proposal?

[bofeng96]

May I know what's the latest state on this proposal?

[chris-ng-scmp]

This is a very important feature, this affects ArgoCD stability, any news?

[krijan-devops]

This is one of the key features we have been waiting on for a while now. Any update would be appreciated

[mstrYoda]

What is the current state of this issue?

There is a PoC implementation in here: #9568

Is there a plan to prioritize this issue?

[jannfis]

@mstrYoda I think the PoC you referenced was targeting applicationset controller, not application controller.

There will be movement for this topic soon. We decided to split up this proposal into multiple, easier to ingest, discuss and align on proposals that are focusing on specific parts to move things forward.

[maartengo]

We decided to split up this proposal into multiple, easier to ingest, discuss and align on proposals that are focusing on specific parts to move things forward.

Sounds good! Do you have links towards those proposals, or will you post them when they are available?

[ishitasequeira]

@maartengo We are still working on the proposals. We will add the proposals once they are available for review.

[ishitasequeira]

The new proposal PR #13221 is opened to address the dynamic rebalancing of clusters across shards. Please feel free to go through it and share your thoughts on the proposal.

[csantanapr]

@ishitasequeira Since the new proposal in PR #13221 got merged do you think is a good idea to close this PR? So folks external to the project don't get confused on having 2 docs.

[ishitasequeira]

@ishitasequeira Since the new proposal in PR #13221 got merged do you think is a good idea to close this PR? So folks external to the project don't get confused on having 2 docs.

Sure, I will close this one