Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large binary log output in Loki #18342

Open
3 tasks done
maggie44 opened this issue May 21, 2024 · 2 comments
Open
3 tasks done

Large binary log output in Loki #18342

maggie44 opened this issue May 21, 2024 · 2 comments
Labels
component:repo-server enhancement New feature or request good first issue Good for newcomers

Comments

@maggie44
Copy link

Checklist:

  • I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
  • I've included steps to reproduce the bug.
  • I've pasted the output of argocd version.

Describe the bug

When viewing the JSON logs of argocd-repo-server in Loki there is a huge binary printed in &runtime.RawExtension{Raw. I have truncated the output, but it is flooding the logs and filling up the storage.

time="2024-05-21T20:37:31Z" level=info msg="manifest cache hit: &ApplicationSource{RepoURL:registry-1.docker.io,Path:,TargetRevision:8.23.4,Helm:&ApplicationSourceHelm{ValueFiles:[],Parameters:[]HelmParameter{},ReleaseName:,Values:,FileParameters:[]HelmFileParameter{},Version:,PassCredentials:false,IgnoreMissingValueFiles:false,SkipCrds:false,ValuesObject:&runtime.RawExtension{Raw:*[123 34 ...],},},Kustomize:nil,Directory:nil,Plugin:nil,Chart:bitnamicharts/kube-prometheus,Ref:,}/8.23.4"

To Reproduce

Deploy ArgoCD to Kubernetes cluster and view the logs through Loki.

Version

v2.11.1+9f40df0

@maggie44 maggie44 added the bug Something isn't working label May 21, 2024
@agaudreault
Copy link
Member

@maggie44 can you share you Application Manifest that correspond to this application. If your application source contains a large object, it will be reflected in the log. This code is generated automatically but might be configurable to exclude some fields:

https://github.com/argoproj/argo-cd/blob/0f72c19e31481c38774a26e8147c1e3bfd5e0ecc/pkg/apis/application/v1alpha1/generated.pb.go#L18524-L18552

Another option would be to sanitize if object before logging it.

@agaudreault agaudreault added enhancement New feature or request good first issue Good for newcomers and removed bug Something isn't working labels Jun 21, 2024
@dejanzele
Copy link

dejanzele commented Oct 25, 2024

Hi,

Is it better to sanitize or try and remove from the generated String method?

I am interested in helping out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:repo-server enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants