feat: server-side manifest generation for local diff (#8145)

[notfromstatefarm]

Signed-off-by: notfromstatefarm 86763948+notfromstatefarm@users.noreply.github.com

Closes #8145

This implements diffs of local manifests without having config management tools or CMPs locally installed, by streaming manifests from the CLI to the repo-server for manifest generation.

Adds:

• --server-side-generate parameter to argocd app diff
• Maximum tar size and extracted file size, configurable via repo server params. This is to prevent someone overwhelming the repo server. By default 100mb for tar size, 1024mb for extracted size.
• Only sends yaml/json files by default, configurable with --local-include

It also refactors a little bit of the CMP stream for DRYness since that stream and this stream shares functionality.

Demo showing server-side diff and local diff being identical:

[codecov]

Codecov Report

Merging #10019 (a1d488d) into master (68d0ef0) will decrease coverage by 0.15%.
The diff coverage is 30.49%.

@@            Coverage Diff             @@
##           master   #10019      +/-   ##
==========================================
- Coverage   46.25%   46.09%   -0.16%     
==========================================
  Files         228      229       +1     
  Lines       27883    28140     +257     
==========================================
+ Hits        12897    12972      +75     
- Misses      13247    13404     +157     
- Partials     1739     1764      +25     

Impacted Files	Coverage Δ
cmd/argocd/commands/app.go	19.25% <0.00%> (-0.27%)	⬇️
reposerver/repository/repository.go	61.81% <0.00%> (-2.60%)	⬇️
server/application/application.go	30.28% <1.78%> (-1.58%)	⬇️
util/cmp/stream.go	54.95% <42.85%> (+2.71%)	⬆️
util/io/files/tar.go	52.99% <48.14%> (-3.58%)	⬇️
util/manifeststream/stream.go	55.88% <55.88%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[crenshaw-dev]

Nitpicky stuff. This looks awesome!

[crenshaw-dev]

@notfromstatefarm can you resolve that conflict?

[crenshaw-dev]

@notfromstatefarm apologies for the slow review, can you resolve conflicts again? :-)

[notfromstatefarm]

@crenshaw-dev I thiiiiink that's a flake in Integration tests / Run unit tests for Go packages (pull_request)? can you rerun?

[crenshaw-dev]

lgtm! I think we need a follow-up issue because --local . is going to cause the RelativePath() check to fail. But otherwise, local testing checks out.

[MeNsaaH]

@crenshaw-dev @notfromstatefarm please, what release is this in? I've installed the latest release and can't find the changes in this yet

[crenshaw-dev]

@MeNsaaH it will be in 2.5 (RC1 coming in the next 2 weeks).

[MeNsaaH]

Okay. Thank you @crenshaw-dev .

[myrondev]

Hi. Is this issue: agocd app diff failing with the URL scheme 'secrets' is not allowed still hasn't been solved in any argocd release?