Opencost integration by Azanul · Pull Request #995 · mlabouardy/komiser

Azanul · 2023-09-25T18:03:29Z

Problem

#801

Solution

Used Opencost for cost calculation of k8s components

Changes Made

Added a new field opencostBaseUrl in k8s config
Used opencostBaseUrl for opencost integration

Notes

Assumed that Opencost is already correctly configure with the cluster.
Do pay attention to the way opencostBaseUrl is parsed from config, I'm not sure if that's the preferred way of doing this.
Fixes Integrate OpenCost for Kubernetes clusters #801

Checklist

Code follows the contributing guidelines
Changes have been thoroughly tested
Documentation has been updated, if necessary
Any dependencies have been added to the project, if necessary

Reviewers

@mlabouardy
@jakepage91

Signed-off-by: Azanul <azanulhaque@gmail.com>

mlabouardy

Great job @Azanul, this a major feature for Komiser 👏🏻
I've left few comments :)

Signed-off-by: Azanul <azanulhaque@gmail.com>

mlabouardy · 2023-09-26T09:04:19Z

Thanks for the quick changes @Azanul :)

AvineshTripathi · 2023-09-26T09:06:12Z

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

mlabouardy · 2023-09-26T09:10:48Z

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

since the namespace wasn't provided, I think all Pods will fetched no matter which namespace they belong to, no?
https://github.com/tailwarden/komiser/blob/c8237dcc5b4bda016e38cc1e981f078f1b0fa356/providers/k8s/core/pods.go#L29

Azanul · 2023-09-26T09:14:54Z

$ kubectl get ns
NAME              STATUS   AGE
default           Active   28h
kube-node-lease   Active   28h
kube-public       Active   28h
kube-system       Active   28h
opencost          Active   28h
prometheus        Active   28h

Opencost "pod" response for a local minikube cluster

{
  "code": 200,
  "status": "success",
  "data": [
    {
      "coredns-5d78c9869d-2ktxj": {
        "name": "coredns-5d78c9869d-2ktxj",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "coredns",
          "controller": "coredns",
          "controllerKind": "deployment",
          "namespace": "kube-system",
          "pod": "coredns-5d78c9869d-2ktxj",
          "services": [
            "kube-dns"
          ],
          "labels": {
            "k8s_app": "kube-dns",
            "kubernetes_io_metadata_name": "kube-system",
            "pod_template_hash": "5d78c9869d"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 73400320,
        "ramByteRequestAverage": 73400320,
        "ramByteUsageAverage": 0,
        "ramByteHours": 1773841066.66667,
        "ramCost": 0.007,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.08339,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "etcd-minikube": {
        "name": "etcd-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "etcd",
          "namespace": "kube-system",
          "pod": "etcd-minikube",
          "labels": {
            "component": "etcd",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 104857600,
        "ramByteRequestAverage": 104857600,
        "ramByteUsageAverage": 0,
        "ramByteHours": 2534058666.66667,
        "ramCost": 0.01,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.08639,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-apiserver-minikube": {
        "name": "kube-apiserver-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-apiserver",
          "namespace": "kube-system",
          "pod": "kube-apiserver-minikube",
          "labels": {
            "component": "kube-apiserver",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.25,
        "cpuCoreRequestAverage": 0.25,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 6.04167,
        "cpuCost": 0.19098,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.19098,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-controller-manager-minikube": {
        "name": "kube-controller-manager-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-controller-manager",
          "namespace": "kube-system",
          "pod": "kube-controller-manager-minikube",
          "labels": {
            "component": "kube-controller-manager",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.2,
        "cpuCoreRequestAverage": 0.2,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 4.83333,
        "cpuCost": 0.15279,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.15279,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-proxy-f6clv": {
        "name": "kube-proxy-f6clv",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-proxy",
          "controller": "kube-proxy",
          "controllerKind": "daemonset",
          "namespace": "kube-system",
          "pod": "kube-proxy-f6clv",
          "labels": {
            "controller_revision_hash": "56999f657b",
            "k8s_app": "kube-proxy",
            "kubernetes_io_metadata_name": "kube-system",
            "pod_template_generation": "1"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "kube-scheduler-minikube": {
        "name": "kube-scheduler-minikube",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-scheduler",
          "namespace": "kube-system",
          "pod": "kube-scheduler-minikube",
          "labels": {
            "component": "kube-scheduler",
            "kubernetes_io_metadata_name": "kube-system",
            "tier": "control-plane"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.1,
        "cpuCoreRequestAverage": 0.1,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 2.41667,
        "cpuCost": 0.07639,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.07639,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls": {
        "name": "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "kube-state-metrics",
          "controller": "my-prometheus-kube-state-metrics",
          "controllerKind": "deployment",
          "namespace": "prometheus",
          "pod": "my-prometheus-kube-state-metrics-6c4b65c7b9-497ls",
          "services": [
            "my-prometheus-kube-state-metrics"
          ],
          "labels": {
            "app_kubernetes_io_component": "metrics",
            "app_kubernetes_io_instance": "my-prometheus",
            "app_kubernetes_io_managed_by": "Helm",
            "app_kubernetes_io_name": "kube-state-metrics",
            "app_kubernetes_io_part_of": "kube-state-metrics",
            "app_kubernetes_io_version": "2.10.0",
            "helm_sh_chart": "kube-state-metrics-5.13.0",
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus",
            "pod_template_hash": "6c4b65c7b9"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-prometheus-node-exporter-gsr4h": {
        "name": "my-prometheus-prometheus-node-exporter-gsr4h",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "node-exporter",
          "controller": "my-prometheus-prometheus-node-exporter",
          "controllerKind": "daemonset",
          "namespace": "prometheus",
          "pod": "my-prometheus-prometheus-node-exporter-gsr4h",
          "services": [
            "my-prometheus-prometheus-node-exporter"
          ],
          "labels": {
            "app_kubernetes_io_component": "metrics",
            "app_kubernetes_io_instance": "my-prometheus",
            "app_kubernetes_io_managed_by": "Helm",
            "app_kubernetes_io_name": "prometheus-node-exporter",
            "app_kubernetes_io_part_of": "prometheus-node-exporter",
            "app_kubernetes_io_version": "1.6.1",
            "controller_revision_hash": "769f45ff7c",
            "helm_sh_chart": "prometheus-node-exporter-4.23.0",
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus",
            "pod_template_generation": "1"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "my-prometheus-server-6fffb9f-gfjxf": {
        "name": "my-prometheus-server-6fffb9f-gfjxf",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "controller": "my-prometheus-server",
          "controllerKind": "deployment",
          "namespace": "prometheus",
          "pod": "my-prometheus-server-6fffb9f-gfjxf",
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "prometheus",
            "name": "prometheus"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 8589934592,
        "pvByteHours": 207590085973.3333,
        "pvCost": 0.01059,
        "pvs": {
          "cluster=cluster-one:name=pvc-8a9fc731-530d-4344-a574-f4c8cb5c0f91": {
            "byteHours": 207590085973.3333,
            "cost": 0.0105936072
          }
        },
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.01059,
        "totalEfficiency": 0,
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "opencost-785545d6d7-w5bj4": {
        "name": "opencost-785545d6d7-w5bj4",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "controller": "opencost",
          "controllerKind": "deployment",
          "namespace": "opencost",
          "pod": "opencost-785545d6d7-w5bj4",
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "opencost"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0.02,
        "cpuCoreRequestAverage": 0.02,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0.48333,
        "cpuCost": 0.01528,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 110000000,
        "ramByteRequestAverage": 110000000,
        "ramByteUsageAverage": 0,
        "ramByteHours": 2658333333.33333,
        "ramCost": 0.01049,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0.02577,
        "totalEfficiency": 0,
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      },
      "storage-provisioner": {
        "name": "storage-provisioner",
        "properties": {
          "cluster": "cluster-one",
          "node": "minikube",
          "container": "storage-provisioner",
          "namespace": "kube-system",
          "pod": "storage-provisioner",
          "labels": {
            "addonmanager_kubernetes_io_mode": "Reconcile",
            "integration_test": "storage-provisioner",
            "kubernetes_io_metadata_name": "kube-system"
          },
          "namespaceLabels": {
            "kubernetes_io_metadata_name": "kube-system"
          }
        },
        "window": {
          "start": "2023-09-01T00:00:00Z",
          "end": "2023-09-26T09:11:37Z"
        },
        "start": "2023-09-25T04:55:00Z",
        "end": "2023-09-26T05:05:00Z",
        "minutes": 1450,
        "cpuCores": 0,
        "cpuCoreRequestAverage": 0,
        "cpuCoreUsageAverage": 0,
        "cpuCoreHours": 0,
        "cpuCost": 0,
        "cpuCostAdjustment": 0,
        "cpuEfficiency": 0,
        "gpuCount": 0,
        "gpuHours": 0,
        "gpuCost": 0,
        "gpuCostAdjustment": 0,
        "networkTransferBytes": 0,
        "networkReceiveBytes": 0,
        "networkCost": 0,
        "networkCrossZoneCost": 0,
        "networkCrossRegionCost": 0,
        "networkInternetCost": 0,
        "networkCostAdjustment": 0,
        "loadBalancerCost": 0,
        "loadBalancerCostAdjustment": 0,
        "pvBytes": 0,
        "pvByteHours": 0,
        "pvCost": 0,
        "pvs": null,
        "pvCostAdjustment": 0,
        "ramBytes": 0,
        "ramByteRequestAverage": 0,
        "ramByteUsageAverage": 0,
        "ramByteHours": 0,
        "ramCost": 0,
        "ramCostAdjustment": 0,
        "ramEfficiency": 0,
        "externalCost": 0,
        "sharedCost": 0,
        "totalCost": 0,
        "totalEfficiency": 0,
        "rawAllocationOnly": {
          "cpuCoreUsageMax": 0,
          "ramByteUsageMax": 0
        },
        "proportionalAssetResourceCosts": {},
        "lbAllocations": null,
        "sharedCostBreakdown": {}
      }
    }
  ]
}

ref: https://www.opencost.io/docs/integrations/api

Azanul · 2023-09-26T09:15:34Z

I think we are only fetching for default namespace deployment however when it comes to production we avoid placing anything in the default namespace so should we also look into all the namespaces in the same PR? cc @mlabouardy

since the namespace wasn't provided, I think all Pods will fetched no matter which namespace they belong to, no?

https://github.com/tailwarden/komiser/blob/c8237dcc5b4bda016e38cc1e981f078f1b0fa356/providers/k8s/core/pods.go#L29

Correct

Azanul added 4 commits September 25, 2023 18:59

go mod tidy

6dcac82

Signed-off-by: Azanul <azanulhaque@gmail.com>

opencost integration

2b8d55b

Signed-off-by: Azanul <azanulhaque@gmail.com>

opencost url through config

2f628ff

Signed-off-by: Azanul <azanulhaque@gmail.com>

k8s pod cost calculation

aa1ad26

Signed-off-by: Azanul <azanulhaque@gmail.com>

Azanul requested review from AvineshTripathi, ShubhamPalriwala and mlabouardy as code owners September 25, 2023 18:03

Merge branch 'develop' into opencost-integration

bd719c9

mlabouardy requested changes Sep 26, 2023

View reviewed changes

Comment thread providers/k8s/core/pods.go Outdated

Comment thread providers/k8s/core/pods.go Outdated

Azanul and others added 2 commits September 26, 2023 14:32

opencost made optional

248fc97

Signed-off-by: Azanul <azanulhaque@gmail.com>

Merge branch 'develop' into opencost-integration

e6b7884

Azanul requested a review from mlabouardy September 26, 2023 09:03

mlabouardy approved these changes Sep 26, 2023

View reviewed changes

Merge branch 'develop' into opencost-integration

c8237dc

mlabouardy added this to the v3.1.1 milestone Sep 26, 2023

mlabouardy added the kubernetes label Sep 26, 2023

mlabouardy merged commit b51ff81 into mlabouardy:develop Sep 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Opencost integration#995

Opencost integration#995
mlabouardy merged 8 commits intomlabouardy:developfrom
Azanul:opencost-integration

Azanul commented Sep 25, 2023

Uh oh!

mlabouardy left a comment

Uh oh!

Uh oh!

Uh oh!

mlabouardy commented Sep 26, 2023

Uh oh!

AvineshTripathi commented Sep 26, 2023

Uh oh!

mlabouardy commented Sep 26, 2023

Uh oh!

Azanul commented Sep 26, 2023

Uh oh!

Azanul commented Sep 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Azanul commented Sep 25, 2023

Problem

Solution

Changes Made

Notes

Checklist

Reviewers

Uh oh!

mlabouardy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mlabouardy commented Sep 26, 2023

Uh oh!

AvineshTripathi commented Sep 26, 2023

Uh oh!

mlabouardy commented Sep 26, 2023

Uh oh!

Azanul commented Sep 26, 2023

Uh oh!

Azanul commented Sep 26, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants