Aggregator is a new backend for Kubecost. It is used in a Federated ETL configuration without Thanos, replacing the Federator component. Aggregator serves a critical subset of Kubecost APIs, but will eventually be the default model for Kubecost and serve all APIs. Currently, Aggregator supports all major monitoring and savings APIs, and also budgets and reporting.
{% hint style="info" %} Existing documentation for Kubecost APIs will use endpoints for non-Aggregator environments unless otherwise specified, but will still be compatible after configuring Aggregator. {% endhint %}
Aggregator is designed to accommodate queries of large-scale datasets by improving API load times and reducing UI errors. It is not designed to introduce new functionality; it is meant to improve functionality at scale.
Aggregator is currently free for all Enterprise users to configure, and is always able to be rolled back.
- Aggregator can only be configured in a Federated ETL environment
- Must be using v1.107.0 of Kubecost or newer
- Your values.yaml file must have set
kubecostDeployment.queryServiceReplicas
to its default value0
. - You must have your context set to your primary cluster. Kubecost Aggregator cannot be deployed on secondary clusters.
Select from one of the two templates below and save the content as federated-store.yaml. This will be your configuration template required to set up Aggregator.
{% hint style="warning" %} The name of the .yaml file used to create the secret must be named federated-store.yaml or Aggregator will not start. {% endhint %}
Basic configuration:
kubecostAggregator:
replicas: 1
enabled: true
cloudCost:
enabled: true
federatedETL:
federatedCluster: true
kubecostModel:
containerStatsEnabled: true
cloudCost:
enabled: false
federatedStorageConfigSecret: federated-store
kubecostProductConfigs:
clusterName: YOUR_CLUSTER_NAME
cloudIntegrationSecret: cloud-integration
productKey:
enabled: true
key: YOUR_KEY
prometheus:
server:
global:
external_labels:
cluster_id: YOUR_CLUSTER_NAME
# when using managed identity/irsa, set the service account accordingly:
serviceAccount:
create: false
name: kubecost-irsa-sa
Advanced configuration (for larger deployments):
kubecostAggregator:
replicas: 1
enabled: true
cloudCost:
enabled: true
env:
# governs parallelism of derivation step
# more threads speeds derivation, but requires significantly more
# log level
# default: info
LOG_LEVEL: info
aggregatorDbStorage:
# governs storage size of aggregator DB storage
# !!NOTE!! disk performance is _critically important_ to aggregator performance
# ensure disk is specd high enough, and check for bottlenecks
# default: 128Gi
storageRequest: 512Gi
federatedETL:
federatedCluster: true
kubecostModel:
containerStatsEnabled: true
federatedStorageConfigSecret: federated-store
kubecostProductConfigs:
clusterName: YOUR_CLUSTER_NAME
cloudIntegrationSecret: cloud-integration
productKey:
enabled: true
key: YOUR_KEY
prometheus:
server:
global:
external_labels:
cluster_id: YOUR_CLUSTER_NAME
# when using managed identity/irsa, set the service account accordingly:
serviceAccount:
create: false
name: kubecost-irsa-sa
There is no baseline for what is considered a larger deployment, which will be dependent on load times in your Kubecost environment.
Once you’ve configured your federated-store.yaml_, create a secret using the following command:
{% code overflow="wrap" %}
kubectl create secret generic federated-storage -n kubecost --from-file=federated-store.yaml
{% endcode %}
Next, you will need to create an additional cloud-integration
secret. Follow this tutorial on creating cloud integration secrets to generate your cloud-integration.json file, then run the following command:
{% code overflow="wrap" %}
kubectl create secret generic cloud-integration -n kubecost --from-file=cloud-integration.json
{% endcode %}
Finally, upgrade your existing Kubecost installation. This command will install Kubecost if it does not already exist:
{% hint style="warning" %}
Upgrading your existing Kubecost using your configured federated-store.yaml_ file above will reset all existing Helm values configured in your values.yaml. If you wish to preserve any of those changes, append your values.yaml by adding the contents of your federated-store.yaml file into it, then replacing federated-store.yaml
with values.yaml
in the upgrade command below:
{% endhint %}
helm upgrade --install "kubecost-primary" \
--namespace kubecost-primary \
--repo https://kubecost.github.io/cost-analyzer/ cost-analyzer \
-f aggregator.yaml
When first enabled, the aggregator pod will ingest the last three years (if applicable) of ETL data from the federated-store. This may take several hours. Because the combined folder is ignored, the federator pod is not used here, but can still run if needed. You can run kubectl get pods
and ensure the aggregator
pod is running, but should still wait for all data to be ingested.