CPU usage of Promitor Agent Resource Discovery goes insane 💀

[Mahagon]

Report

After deploying the resource discovery agent the cpu usages raises to 3000+ milliCPU and it stays there

I'm unsure where the cause of the high cpu usage is. I tried to reduce the resourceDiscoveryGroups but that didn't help much.

for now I added limits, so that it doesn't effect other pods that much

      - resources:
          limits:
            cpu: 500m
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 128Mi

Expected Behavior

Less CPU usage :)

Actual Behavior

Resource Discovery Agent uses more then 3 cores of an Azure Kubernetes Node

Steps to Reproduce the Problem

1. Deploy the configuration below
2. After a few seconds the cpu usage raises and stays high

Component

Resource Discovery

Version

0.7.3

Configuration

Configuration:

---
repositories:
  - name: promitor
    url: https://charts.promitor.io/
  - name: bedag
    url: https://bedag.github.io/helm-charts

releases:
  - name: promitor-scraper
    namespace: promitor
    chart: promitor/promitor-agent-scraper
    values:
      - azureAuthentication:
          mode: UserAssignedManagedIdentity
          identity:
            id: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_scraper_client_id") }}
            binding: promitor-scraper-identity
      - azureMetadata:
          tenantId: {{ requiredEnv "ARM_TENANT_ID" }}
          subscriptionId: {{ requiredEnv "ARM_SUBSCRIPTION_ID" }}
          resourceGroupName: REDACTED
      - resourceDiscovery:
          enabled: true
          host: promitor-agent-resource-discovery
          port: 8889
      - metricSinks:
          prometheusScrapingEndpoint:
            enabled: true
            serviceMonitor:
              enabled : true
              namespace: promitor
          enableServiceDiscovery: true
      - metrics:
          - name: azure_container_registry_agent_pool_cpu_time
            description: "AgentPool CPU Time in seconds"
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: AgentPoolCPUTime
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_container_registry_run_duration
            description: "Run Duration in milliseconds"
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: RunDuration
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_container_registry_storage_used
            description: "The amount of storage used by the container registry. \
            For a registry account, it's the sum of capacity used by all the repositories within a registry. \
            It's sum of capacity used by shared layers, manifest files, and replica copies in each of its repositories."
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: StorageUsed
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_container_registry_successful_pull_count
            description: "Number of successful image pulls"
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: SuccessfulPullCount
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_container_registry_successful_push_count
            description: "Number of successful image pushes"
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: SuccessfulPushCount
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_container_registry_total_pull_count
            description: "Number of image pulls in total"
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: TotalPullCount
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_container_registry_total_push_count
            description: "Number of image pushes in total"
            resourceType: ContainerRegistry
            azureMetricConfiguration:
              metricName: TotalPushCount
              aggregation:
                type: Total
            resourceDiscoveryGroups:
              - name: container-registry-landscape
          - name: azure_key_vault_availability
            description: "Vault requests availability"
            resourceType: KeyVault
            azureMetricConfiguration:
              metricName: Availability
              aggregation:
                type: Average
            resources:
              - vaultName: akvzvoovesaas{{ requiredEnv "ENVIRONMENT" }}
          - name: azure_key_vault_saturation_shoebox
            description: "Vault capacity used"
            resourceType: KeyVault
            azureMetricConfiguration:
              metricName: SaturationShoebox
              aggregation:
                type: Average
            resources:
              - vaultName: akvzvoovesaas{{ requiredEnv "ENVIRONMENT" }}
          - name: azure_key_vault_api_hit_count
            description: "Number of total service api hits"
            resourceType: KeyVault
            azureMetricConfiguration:
              metricName: ServiceApiHit
              aggregation:
                type: Count
            resources:
              - vaultName: akvzvoovesaas{{ requiredEnv "ENVIRONMENT" }}
          - name: azure_key_vault_api_latency
            description: "Overall latency of service api requests"
            resourceType: KeyVault
            azureMetricConfiguration:
              metricName: ServiceApiLatency
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: key-vault-landscape
          - name: azure_key_vault_api_result_count
            description: "Number of total service api results"
            resourceType: KeyVault
            azureMetricConfiguration:
              metricName: ServiceApiResult
              aggregation:
                type: Count
            resourceDiscoveryGroups:
              - name: key-vault-landscape
          - name: cluster_autoscaler_cluster_safe_to_autoscale
            description: "Determines whether or not cluster autoscaler will take action on the cluster"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: cluster_autoscaler_cluster_safe_to_autoscale
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: cluster_autoscaler_scale_down_in_cooldown
            description: "Determines if the scale down is in cooldown - No nodes will be removed during this timeframe"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: cluster_autoscaler_scale_down_in_cooldown
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: cluster_autoscaler_unneeded_nodes_count
            description: "Cluster auotscaler marks those nodes as candidates for deletion and are eventually deleted"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: cluster_autoscaler_unneeded_nodes_count
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: cluster_autoscaler_unschedulable_pods_count
            description: "Number of pods that are currently unschedulable in the cluster"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: cluster_autoscaler_unschedulable_pods_count
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: kube_node_status_allocatable_cpu_cores
            description: "Total number of available cpu cores in a managed cluster"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: kube_node_status_allocatable_cpu_cores
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: kube_node_status_allocatable_memory_bytes
            description: "Total amount of available memory in a managed cluster"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: kube_node_status_allocatable_memory_bytes
              aggregation:
                type: Average
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_cpu_usage_millicores
            description: "Aggregated measurement of CPU utilization in millicores across the cluster"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_cpu_usage_millicores
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_cpu_usage_percentage
            description: "Aggregated average CPU utilization measured in percentage across the cluster"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_cpu_usage_percentage
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_disk_usage_bytes
            description: "Disk space used in bytes by device"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_disk_usage_bytes
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_disk_usage_percentage
            description: "Disk space used in percent by device"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_disk_usage_percentage
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_memory_rss_bytes
            description: "Container RSS memory used in bytes"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_memory_rss_bytes
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_memory_rss_percentage
            description: "Container RSS memory used in percent"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_memory_rss_percentage
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_memory_working_set_bytes
            description: "Container working set memory used in bytes"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_memory_working_set_bytes
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_memory_working_set_percentage
            description: "Container working set memory used in percent"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_memory_working_set_percentage
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_network_in_bytes
            description: "Network received bytes"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_network_in_bytes
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: node_network_out_bytes
            description: "Network transmitted bytes"
            resourceType: KubernetesService
            azureMetricConfiguration:
              metricName: node_network_out_bytes
              aggregation:
                type: Average
              dimension:
                name: node
            resourceDiscoveryGroups:
              - name: azure-kubernetes-service
          - name: azure_storage_account_availability
            description: "The percentage of availability for the storage service or the specified API operation. \
            Availability is calculated by taking the TotalBillableRequests value and dividing it by the number of applicable requests, \
            including those that produced unexpected errors. \
            All unexpected errors result in reduced availability for the storage service or the specified API operation."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: Availability
              aggregation:
                type: Average
              dimension:
                name: ApiName
            resourceDiscoveryGroups:
              - name: storage-account-landscape
          - name: azure_storage_account_bytes_sent_rate
            description: "The amount of egress data, in bytes. \
            This number includes egress from an external client into Azure Storage as well as egress within Azure. \
            As a result, this number does not reflect billable egress."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: Egress
              aggregation:
                type: Total
              dimension:
                name: ApiName
            resourceDiscoveryGroups:
              - name: storage-account-landscape
          - name: azure_storage_account_bytes_received_rate
            description: "The amount of ingress data, in bytes. \
            This number includes ingress from an external client into Azure Storage as well as ingress within Azure."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: Ingress
              aggregation:
                type: Total
              dimension:
                name: ApiName
            resourceDiscoveryGroups:
              - name: storage-account-landscape
          - name: azure_storage_account_success_end2end_latency
            description: "The average end-to-end latency of successful requests made to a storage service or the specified API operation, in milliseconds. \
            This value includes the required processing time within Azure Storage to read the request, \
            send the response, and receive acknowledgment of the response."
            resourceType: NetworkInterface
            azureMetricConfiguration:
              metricName: SuccessE2ELatency
              aggregation:
                type: Average
              dimension:
                name: ApiName
            resourceDiscoveryGroups:
              - name: storage-account-landscape
          - name: azure_virtual_network_ping_mesh_average_roundtrip_ms
            description: "Round trip time for Pings sent to a destination VM."
            resourceType: VirtualNetwork
            azureMetricConfiguration:
              metricName: PingMeshAverageRoundtripMs
              aggregation:
                type: Average
              dimension:
                name: DestinationCustomerAddress
            resourceDiscoveryGroups:
              - name: virtual-network-landscape
          - name: azure_virtual_network_mesh_probe_failed_percent
            description: "Percent of number of failed Pings to total sent Pings of a destination VM."
            resourceType: VirtualNetwork
            azureMetricConfiguration:
              metricName: PingMeshProbesFailedPercent
              aggregation:
                type: Average
              dimension:
                name: DestinationCustomerAddress
            resourceDiscoveryGroups:
              - name: virtual-network-landscape
      - tolerations:
          - key: "CriticalAddonsOnly"
            operator: "Exists"
            effect: "NoSchedule"
  - name: promitor-discovery
    namespace: promitor
    chart: promitor/promitor-agent-resource-discovery
    values:
      - azureAuthentication:
          mode: UserAssignedManagedIdentity
          identity:
            id: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_discovery_client_id") }}
            binding: promitor-discovery-identity
      - azureLandscape:
          cloud: Global
          tenantId: {{ requiredEnv "ARM_TENANT_ID" }}
          subscriptions:
            - {{ requiredEnv "ARM_SUBSCRIPTION_ID" }}
      - prometheus:
          serviceMonitor:
            enabled: true
            namespace: promitor
      - resourceDiscoveryGroups:
          - name: container-registry-landscape
            type: ContainerRegistry
          - name: key-vault-landscape
            type: KeyVault
          - name: azure-kubernetes-service
            type: KubernetesService
          - name: virtual-network-landscape
            type: VirtualNetwork
          - name: storage-account-landscape
            type: StorageAccount
      - tolerations:
          - key: "CriticalAddonsOnly"
            operator: "Exists"
            effect: "NoSchedule"
  - name: promitor-scraper-identity
    namespace: promitor
    chart: ../../charts/azure-identity
    values:
      - identityName: promitor-scraper-identity
      - resourceId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_scraper_resource_id") }}
      - clientId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_scraper_client_id") }}
      - selector: promitor-scraper-identity
  - name: promitor-discovery-identity
    namespace: promitor
    chart: ../../charts/azure-identity
    values:
      - identityName: promitor-discovery-identity
      - resourceId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_discovery_resource_id") }}
      - clientId: {{ exec "terraform" (list "-chdir=../.." "output" "-json" "promitor_discovery_client_id") }}
      - selector: promitor-discovery-identity

Logs

top output in promitor-agent-resource-discovery shell:

  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
    1     0 root     S    20.1g 121%   1  42% dotnet Promitor.Agents.ResourceDiscovery.dll

k9s view

NAMESPACE              NAME                                                              PF  READY   RESTARTS STATUS         CPU↓    MEM   %CPU/R   %CPU/L   %MEM/R   %MEM/L IP              NODE                              AGE
promitor               promitor-agent-resource-discovery-585f9d9b8d-69wwq                ●   1/1            0 Running        3319    259      n/a      n/a      n/a      n/a xx.xxx.x.xx     aks-system-33053085-vmss00000e    18h   

logs

[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.3114ms                                                                                
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/azure-kubernetes-service/discover?currentPage=1 - - - 200 654 application/json 2.7396ms                               
[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 1.1231ms                                                                                
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/container-registry-landscape/discover?currentPage=1 - - - 200 658 application/json 23.8754ms                          
[09:05:00 INF] Request starting HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/container-registry-landscape/discover?currentPage=1 - -                                                               
[09:05:00 INF] Route matched with {action = "Get", controller = "DiscoveryV2"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(System.String, Int32, Int32) on con troller Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                      
[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.2467ms                                                                                
[09:05:00 INF] Route matched with {action = "Get", controller = "DiscoveryV2"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(System.String, Int32, Int32) on con troller Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                      
[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.2758ms                                                                                
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/azure-kubernetes-service/discover?currentPage=1 - - - 200 654 application/json 33.5840ms                              
[09:05:00 INF] Request starting HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/azure-kubernetes-service/discover?currentPage=1 - -                                                                   
[09:05:00 INF] Request starting HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/container-registry-landscape/discover?currentPage=1 - -                                                               
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/network-interfaces-landscape/discover?currentPage=1 - - - 200 2701 application/json 3.7663ms                          
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/container-registry-landscape/discover?currentPage=1 - - - 200 658 application/json 3.7642ms                           
[09:05:00 INF] Route matched with {action = "Get", controller = "DiscoveryV2"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(System.String, Int32, Int32) on con troller Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                      
[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.2377ms                                                                                
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/azure-kubernetes-service/discover?currentPage=1 - - - 200 654 application/json 3.0454ms                               
[09:05:00 INF] Request starting HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/virtual-network-landscape/discover?currentPage=1 - -                                                                  
[09:05:00 INF] Route matched with {action = "Get", controller = "DiscoveryV2"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(System.String, Int32, Int32) on con troller Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                      
[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.2297ms                                                                                
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/virtual-network-landscape/discover?currentPage=1 - - - 200 640 application/json 8.9746ms                              
[09:05:00 INF] Route matched with {action = "Get", controller = "DiscoveryV2"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(System.String, Int32, Int32) on con troller Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                      
[09:05:00 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v2.DiscoveryV2Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.1904ms                                                                                
[09:05:00 INF] Request finished HTTP/1.1 GET http://promitor-agent-resource-discovery:8889/api/v2/resources/groups/container-registry-landscape/discover?currentPage=1 - - - 200 658 application/json 7.9096ms                           
[09:05:01 INF] Request starting HTTP/1.1 GET http://xx.xxx.x.xx:88/api/v1/health?includeDependencies=false - -                                                                                                                           
[09:05:01 INF] Route matched with {action = "Get", controller = "HealthV1"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(Boolean) on controller Promitor.Agents .ResourceDiscovery.Controllers.v1.HealthV1Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                                                
[09:05:01 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v1.HealthV1Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.2385ms                                                                                   
[09:05:01 INF] Request finished HTTP/1.1 GET http://xx.xxx.x.xx:88/api/v1/health?includeDependencies=false - - - 200 60 application/json;+charset=utf-8 18.9811ms                                                                        
[09:05:06 INF] Request starting HTTP/1.1 GET http://xx.xxx.x.xx:88/api/v1/health?includeDependencies=false - -                                                                                                                           
[09:05:06 INF] Route matched with {action = "Get", controller = "HealthV1"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(Boolean) on controller Promitor.Agents .ResourceDiscovery.Controllers.v1.HealthV1Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                                                
[09:05:06 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v1.HealthV1Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.2379ms                                                                                   
[09:05:06 INF] Request finished HTTP/1.1 GET http://xx.xxx.x.xx:88/api/v1/health?includeDependencies=false - - - 200 60 application/json;+charset=utf-8 1.4680ms                                                                         
[09:05:11 INF] Request starting HTTP/1.1 GET http://xx.xxx.x.xx:88/api/v1/health?includeDependencies=false - -                                                                                                                           
[09:05:11 INF] Route matched with {action = "Get", controller = "HealthV1"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Get(Boolean) on controller Promitor.Agents .ResourceDiscovery.Controllers.v1.HealthV1Controller (Promitor.Agents.ResourceDiscovery).                                                                                                                                                
[09:05:11 INF] Executed action Promitor.Agents.ResourceDiscovery.Controllers.v1.HealthV1Controller.Get (Promitor.Agents.ResourceDiscovery) in 0.9228ms                                                                                   
[09:05:11 INF] Request finished HTTP/1.1 GET http://xx.xxx.x.xx:88/api/v1/health?includeDependencies=false - - - 200 60 application/json;+charset=utf-8 4.3626ms  

Platform

Microsoft Azure

Contact Details

No response

[tomkerkhove]

How many resources are being discovered?

[Mahagon]

Unsure how to count those couldn't find it in the promitor discovery metrics
I configured 31 ResourceTypes to fetch Metrics from
curl on the scraper pod port 5000/metrics returns 831 lines of metrics

discovery agent metrics:

# HELP promitor_runtime_dotnet_collection_count_total GC collection count
# TYPE promitor_runtime_dotnet_collection_count_total counter
promitor_runtime_dotnet_collection_count_total{generation="0"} 110987
promitor_runtime_dotnet_collection_count_total{generation="1"} 1294
promitor_runtime_dotnet_collection_count_total{generation="2"} 109
# HELP promitor_runtime_dotnet_totalmemory Total known allocated memory
# TYPE promitor_runtime_dotnet_totalmemory gauge
promitor_runtime_dotnet_totalmemory 7270240
# HELP promitor_runtime_process_cpu_seconds_total Total user and system CPU time spent in seconds
# TYPE promitor_runtime_process_cpu_seconds_total counter
promitor_runtime_process_cpu_seconds_total 5115.33
# HELP promitor_runtime_process_virtual_bytes Process virtual memory size
# TYPE promitor_runtime_process_virtual_bytes gauge
promitor_runtime_process_virtual_bytes 3492896768
# HELP promitor_runtime_process_working_set Process working set
# TYPE promitor_runtime_process_working_set gauge
promitor_runtime_process_working_set 112762880
# HELP promitor_runtime_process_private_bytes Process private memory size
# TYPE promitor_runtime_process_private_bytes gauge
promitor_runtime_process_private_bytes 142860288
# HELP promitor_runtime_process_num_threads Total number of threads
# TYPE promitor_runtime_process_num_threads gauge
promitor_runtime_process_num_threads 36
# HELP promitor_runtime_process_processid Process ID
# TYPE promitor_runtime_process_processid gauge
promitor_runtime_process_processid 1
# HELP promitor_runtime_process_start_time_seconds Start time of the process since unix epoch in seconds
# TYPE promitor_runtime_process_start_time_seconds gauge
promitor_runtime_process_start_time_seconds 1651138048
# HELP promitor_azure_landscape_resource_group_info Provides information concerning the Azure resource groups in the landscape that Promitor has access to.
# TYPE promitor_azure_landscape_resource_group_info gauge
promitor_azure_landscape_resource_group_info{tenant_id="TENANTID",subscription_id="SUBSCRIPTIONID",resource_group_name="redactedTERRAFORM",region="germanywestcentral",provisioning_state="Succeeded",managed_by="n/a"} 1 1651148316632
promitor_azure_landscape_resource_group_info{tenant_id="TENANTID",subscription_id="SUBSCRIPTIONID",resource_group_name="redacted",region="germanywestcentral",provisioning_state="Succeeded",managed_by="n/a"} 1 1651148316632
promitor_azure_landscape_resource_group_info{tenant_id="TENANTID",subscription_id="SUBSCRIPTIONID",resource_group_name="cloud-shell-storage-westeurope",region="westeurope",provisioning_state="Succeeded",managed_by="n/a"} 1 1651148316632
promitor_azure_landscape_resource_group_info{tenant_id="TENANTID",subscription_id="SUBSCRIPTIONID",resource_group_name="MC_redacted_redacted-saas-dev_germanywestcentral",region="germanywestcentral",provisioning_state="Succeeded",managed_by="/subscriptions/SUBSCRIPTIONID/resourcegroups/redacted/providers/Microsoft.ContainerService/managedClusters/redacted-saas-dev"} 1 1651148316632
promitor_azure_landscape_resource_group_info{tenant_id="TENANTID",subscription_id="SUBSCRIPTIONID",resource_group_name="NetworkWatcherRG",region="germanywestcentral",provisioning_state="Succeeded",managed_by="n/a"} 1 1651148316632
# HELP promitor_azure_landscape_subscription_info Provides information concerning the Azure subscriptions in the landscape that Promitor has access to.
# TYPE promitor_azure_landscape_subscription_info gauge
promitor_azure_landscape_subscription_info{tenant_id="TENANTID",subscription_name="redacted-dev",subscription_id="SUBSCRIPTIONID",state="Enabled",spending_limit="Off",quota_id="PayAsYouGo_2014-09-01",authorization="n/a"} 1 1651148316625
# HELP promitor_ratelimit_resource_graph_remaining Indication how many calls are still available before Azure Resource Graph is going to throttle us.
# TYPE promitor_ratelimit_resource_graph_remaining gauge
promitor_ratelimit_resource_graph_remaining{tenant_id="TENANTID",cloud="Global",auth_mode="UserAssignedManagedIdentity",app_id="722ff7a7-0796-49fc-b5d6-6e88e28e77d4"} 4 1651148301217
# HELP promitor_ratelimit_resource_graph_throttled Indication concerning Azure Resource Graph are being throttled. (1 = yes, 0 = no).
# TYPE promitor_ratelimit_resource_graph_throttled gauge
promitor_ratelimit_resource_graph_throttled{tenant_id="TENANTID",cloud="Global",auth_mode="UserAssignedManagedIdentity",app_id="722ff7a7-0796-49fc-b5d6-6e88e28e77d4"} 0 1651148301217
# HELP promitor_runtime_http_request_duration_seconds duration histogram of http responses labeled with: status_code, method, path
# TYPE promitor_runtime_http_request_duration_seconds histogram
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.005"} 251
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.01"} 256
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.025"} 257
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.05"} 258
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.075"} 262
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.1"} 269
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.25"} 272
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.5"} 272
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="0.75"} 272
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="1"} 274
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="2.5"} 323
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="5"} 349
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="7.5"} 404
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="10"} 459
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover",le="+Inf"} 544
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover"} 2577.6878482769994
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/api/v2/resources/groups/azure-kubernetes-service/discover"} 544
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.005"} 162
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.01"} 163
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.025"} 164
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.05"} 164
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.075"} 164
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.1"} 170
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.25"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.5"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="0.75"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="1"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="2.5"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="5"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="7.5"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="10"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/metrics",le="+Inf"} 172
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/metrics"} 1.0590273119999996
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/metrics"} 172
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.005"} 105
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.01"} 110
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.025"} 114
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.05"} 114
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.075"} 114
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.1"} 118
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.25"} 119
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.5"} 119
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="0.75"} 119
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="1"} 121
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="2.5"} 145
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="5"} 161
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="7.5"} 182
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="10"} 206
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover",le="+Inf"} 238
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover"} 980.4459984569994
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/api/v2/resources/groups/container-registry-landscape/discover"} 238
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.005"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.01"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.025"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.05"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.075"} 33
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.1"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.25"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.5"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="0.75"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="1"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="2.5"} 42
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="5"} 44
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="7.5"} 48
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="10"} 55
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover",le="+Inf"} 68
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover"} 318.468902593
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/api/v2/resources/groups/key-vault-landscape/discover"} 68
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.005"} 32
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.01"} 32
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.025"} 32
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.05"} 32
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.075"} 32
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.1"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.25"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.5"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="0.75"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="1"} 34
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="2.5"} 39
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="5"} 43
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="7.5"} 50
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="10"} 58
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover",le="+Inf"} 68
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover"} 324.198868577
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/api/v2/resources/groups/virtual-network-landscape/discover"} 68
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.005"} 2284
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.01"} 2332
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.025"} 2366
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.05"} 2385
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.075"} 2389
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.1"} 2393
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.25"} 2394
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.5"} 2394
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="0.75"} 2394
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="1"} 2395
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="2.5"} 2395
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="5"} 2395
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="7.5"} 2395
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="10"} 2395
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v1/health",le="+Inf"} 2395
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/api/v1/health"} 4.290495213999996
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/api/v1/health"} 2395
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.005"} 54
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.01"} 57
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.025"} 61
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.05"} 61
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.075"} 62
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.1"} 65
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.25"} 68
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.5"} 68
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="0.75"} 68
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="1"} 69
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="2.5"} 80
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="5"} 91
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="7.5"} 101
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="10"} 114
promitor_runtime_http_request_duration_seconds_bucket{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover",le="+Inf"} 136
promitor_runtime_http_request_duration_seconds_sum{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover"} 566.3055687809999
promitor_runtime_http_request_duration_seconds_count{status_code="200",method="GET",path="/api/v2/resources/groups/storage-account-landscape/discover"} 136

[ahurtaud]

Hello, Same for us with v0.7.1, discovery agent now consumes ~13 cpu cores :(

We have an increase of generation=0 promitor_runtime_dotnet_collection_count_total of ~80000/hour where it was ~7 in v0.7.0:

[tomkerkhove]

Hm that's not nice. So this was introduced from v0.7.1 -> v0.7.1 @ahurtaud?

[ahurtaud]

yes fallback to v0.7.0 make cpu go very lower (0.01 cores)

[tomkerkhove]

Oh boy, so that went from 13 CPUs to 0.01 CPU with same discovery landscape?!

[ahurtaud]

yes ^^', maybe that limit 1000 resources change?

[tomkerkhove]

I'll add this to the next release. It's probably related to the # of resources being discovered indeed but it has to be improved.

I'm still wrapping up the new docs though so it will not be tomorrow; sorry.

[tomkerkhove]

@Mahagon How big is your subscription in terms of # resources being discovered?