Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose streaming aggregation config in vmagent chart #930

Closed
spoofedpacket opened this issue Mar 12, 2024 · 4 comments
Closed

Expose streaming aggregation config in vmagent chart #930

spoofedpacket opened this issue Mar 12, 2024 · 4 comments
Labels
question Further information is requested

Comments

@spoofedpacket
Copy link

Streaming aggregation looks like a great alternative to recording rules and I'd like to try it out:
https://docs.victoriametrics.com/stream-aggregation/

However the -remoteWrite.streamAggr.config setting is not available through the vmagent helm chart, requiring some manual intervention to configure. Would it be possible to expose it and a method for defining streaming aggregation rules?

Thanks!

@Haleygo
Copy link
Contributor

Haleygo commented Mar 20, 2024

Hello!
Now you can add -remoteWrite.streamAggr.config under .Values.extraArgs and mount the config file using extra configmap defined in extraObjects

# Add extra specs dynamically to this chart
extraObjects: []

Do you have better idea on how to expose this?

@Haleygo Haleygo added the question Further information is requested label Mar 20, 2024
@nemobis
Copy link
Contributor

nemobis commented Mar 20, 2024

In the latest version of the helm chart, which I use in terraform, I do something like

resource "helm_release" "victoriametrics" {
  name       = "victoriametrics"
  chart      = var.vmcluster_chart_name
  version    = var.vmcluster_chart_version
  repository = var.chart_repository
  namespace  = kubernetes_namespace.victoriametrics.metadata[0].name
  timeout    = 1200

  values = [local.vmcluster_helm_chart_values]
}

resource "kubernetes_config_map" "victoriametrics_relabel_configs" {
  metadata {
    name      = "victoriametrics-agent-vmagent-relabel-configs"
    namespace = kubernetes_namespace.victoriametrics.metadata[0].name
  }

  data = {
    "remotewrite_relabelconfig.yml"     = "${file("${path.module}/manifests/remotewrite_relabelconfig.yaml")}"
    "remotewrite_streamAggr_config.yml" = "${file("${path.module}/manifests/remotewrite_streamAggr_config.yaml")}"
  }
}

and then I just use it in extraArgs like

extraArgs:
  remoteWrite.relabelConfig: /config/extra/remotewrite_relabelconfig.yml
  remoteWrite.streamAggr.config: /config/extra/remotewrite_streamAggr_config.yml

I prefer it this way because I can keep a real YAML file in my terraform repository and use all the usual YAML tools on it.

@spoofedpacket
Copy link
Author

Thanks for the suggestions, I'll take a look

@Haleygo
Copy link
Contributor

Haleygo commented Jun 26, 2024

Close as answered, feel free to re-open if there is further question.

@Haleygo Haleygo closed this as completed Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants