Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix usage stats report resend logic to not generate a new report each time we retry sending it #2736

Merged
merged 2 commits into from
Aug 17, 2022
Merged

Fix usage stats report resend logic to not generate a new report each time we retry sending it #2736

merged 2 commits into from
Aug 17, 2022

Conversation

pracucci
Copy link
Collaborator

What this PR does

While working on #2685 I realized it's not very good we regenerate a new report when we retry sending it after a failure. The main reason is that counter metrics are reset each time we generate a new report.

In this PR I'm proposing to resend the previously generated report when we resend it.

Which issue(s) this PR fixes or relates to

Part of #1815

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@pracucci pracucci requested a review from colega August 16, 2022 13:20
@pracucci pracucci mentioned this pull request Aug 16, 2022
Closed
colega
colega reviewed Aug 16, 2022
View reviewed changes
pkg/usagestats/reporter.go Outdated

// We define a function to update the timestamp of the next report to make sure
// we also reset the next report when doing it.
updateNextReportAt := func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opinionated: I think this function's name is hiding the side effect of upsating nextReport and makes the code below harder to read than if we repeated the two lines in both places.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think the function may help avoiding to forget to reset the nextReport to nil in future code changes. I agree updateNextReportAt() naming doesn't clarify it, so I've renamed it to scheduleNextReport().

colega
colega reviewed Aug 16, 2022
View reviewed changes
pkg/usagestats/reporter.go Outdated Show resolved Hide resolved
@pracucci pracucci marked this pull request as ready for review August 16, 2022 13:56
@pracucci
Fix usage stats report resend logic to not generate a new report each…
22dabb9 
… time we retry sending it

Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Renamed updateNextReportAt() to scheduleNextReport()
ef83c46 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci pracucci force-pushed the fix-usage-stats-report-retry branch from dafd820 to ef83c46 Compare August 17, 2022 08:03
@pracucci pracucci enabled auto-merge (squash) August 17, 2022 08:04
@github-actions
Copy link
Contributor

Helm <> Jsonnet Diff

⚠️ A difference was detected between the Helm chart and the Jsonnet library.

  1. Use make check-helm-jsonnet-diff to reproduce the output locally.
  2. This test is experimental while we gather feedback about its usefulness.
  3. It does not block your PR from being merged, but we would appreciate you trying to keep feature parity between the Helm chart and Jsonnet library if possible.

If you get stuck on this step and the Mimir maintainers aren't able to help, feel free to merge without making this step pass and tag @Logiraptor so the Mimir maintainers can gather feedback later.

Please see the contribution docs here for more info.

Expand to see the output

Output of https://github.com/grafana/mimir/actions/runs/2873741146

Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
diff -r -u -N scratch/./helm/07-config/mimir-query-scheduler-Deployment.yml scratch/./jsonnet/08-config/mimir-query-scheduler-Deployment.yml
--- scratch/./helm/07-config/mimir-query-scheduler-Deployment.yml	2022-08-17 08:08:14.533895997 +0000
+++ scratch/./jsonnet/08-config/mimir-query-scheduler-Deployment.yml	1970-01-01 00:00:00.000000000 +0000
@@ -1,55 +0,0 @@
----
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  generation: 1
-  name: mimir-query-scheduler
-  namespace: default
-spec:
-  progressDeadlineSeconds: 600
-  revisionHistoryLimit: 10
-  selector: {}
-  strategy:
-    rollingUpdate: {}
-    type: RollingUpdate
-  template:
-    metadata:
-      annotations:
-        checksum/config: aebdb82885e8ccf4345909101f408f7a24773c8ef0eccbcfb5b13f41e7667e82
-    spec:
-      affinity:
-        podAntiAffinity: {}
-      containers:
-        - imagePullPolicy: IfNotPresent
-          name: mimir
-          ports:
-            - containerPort: 8080
-              name: http-metrics
-              protocol: TCP
-            - containerPort: 9095
-              name: grpc
-              protocol: TCP
-          readinessProbe:
-            failureThreshold: 3
-            httpGet:
-              path: /ready
-              port: http-metrics
-              scheme: HTTP
-            periodSeconds: 10
-            successThreshold: 1
-            timeoutSeconds: 1
-          terminationMessagePath: /dev/termination-log
-          terminationMessagePolicy: File
-      dnsPolicy: ClusterFirst
-      restartPolicy: Always
-      schedulerName: default-scheduler
-      volumes:
-        - configMap:
-            defaultMode: 420
-            name: mimir-runtime
-          name: runtime-config
-        - emptyDir: {}
-          name: storage
-  updateStrategy:
-    rollingUpdate: {}
-status: {}
diff -r -u -N scratch/./helm/07-config/mimir-query-scheduler-Service.yml scratch/./jsonnet/08-config/mimir-query-scheduler-Service.yml
--- scratch/./helm/07-config/mimir-query-scheduler-Service.yml	2022-08-17 08:08:14.529895905 +0000
+++ scratch/./jsonnet/08-config/mimir-query-scheduler-Service.yml	1970-01-01 00:00:00.000000000 +0000
@@ -1,23 +0,0 @@
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: mimir-query-scheduler
-  namespace: default
-spec:
-  ipFamilies:
-    - IPv4
-  ipFamilyPolicy: SingleStack
-  ports:
-    - name: http-metrics
-      port: 8080
-      protocol: TCP
-      targetPort: http-metrics
-    - name: grpc
-      port: 9095
-      protocol: TCP
-      targetPort: grpc
-  sessionAffinity: None
-  type: ClusterIP
-status:
-  loadBalancer: {}
diff -r -u -N scratch/./helm/07-config/mimir-query-scheduler-headless-Service.yml scratch/./jsonnet/08-config/mimir-query-scheduler-headless-Service.yml
--- scratch/./helm/07-config/mimir-query-scheduler-headless-Service.yml	2022-08-17 08:08:14.529895905 +0000
+++ scratch/./jsonnet/08-config/mimir-query-scheduler-headless-Service.yml	1970-01-01 00:00:00.000000000 +0000
@@ -1,27 +0,0 @@
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: mimir-query-scheduler-headless
-  namespace: default
-spec:
-  clusterIP: None
-  clusterIPs:
-    - None
-  ipFamilies:
-    - IPv4
-  ipFamilyPolicy: SingleStack
-  ports:
-    - name: http-metrics
-      port: 8080
-      protocol: TCP
-      targetPort: http-metrics
-    - name: grpc
-      port: 9095
-      protocol: TCP
-      targetPort: grpc
-  publishNotReadyAddresses: true
-  sessionAffinity: None
-  type: ClusterIP
-status:
-  loadBalancer: {}
diff -r -u -N scratch/./helm/07-config/query-scheduler-MimirConfig.yml scratch/./jsonnet/08-config/query-scheduler-MimirConfig.yml
--- scratch/./helm/07-config/query-scheduler-MimirConfig.yml	2022-08-17 08:08:14.597897467 +0000
+++ scratch/./jsonnet/08-config/query-scheduler-MimirConfig.yml	1970-01-01 00:00:00.000000000 +0000
@@ -1,987 +0,0 @@
----
-apiVersion: grafana.com/v1alpha1
-config:
-  activity_tracker:
-    max_entries: 1024 (default)
-  alertmanager:
-    alertmanager_client:
-      backoff_config:
-        max_period: 10s (default)
-        max_retries: 10 (default)
-        min_period: 100ms (default)
-      backoff_on_ratelimits: false (default)
-      grpc_compression: ' (default)'
-      max_recv_msg_size: 104857600 (default)
-      max_send_msg_size: 104857600 (default)
-      rate_limit: 0 (default)
-      rate_limit_burst: 0 (default)
-      remote_timeout: 2s (default)
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    enable_api: true (default)
-    fallback_config_file: ' (default)'
-    max_concurrent_get_requests_per_tenant: 0 (default)
-    max_recv_msg_size: 104857600 (default)
-    peer_timeout: 15s (default)
-    persist_interval: 15m0s (default)
-    poll_interval: 15s (default)
-    retention: 120h0m0s (default)
-    sharding_ring:
-      heartbeat_period: 15s (default)
-      heartbeat_timeout: 1m0s (default)
-      instance_addr: ' (default)'
-      instance_availability_zone: ' (default)'
-      instance_id: fv-az459-214 (default)
-      instance_interface_names:
-        - eth0 (default)
-        - docker0 (default)
-        - br-feec2824b200 (default)
-      instance_port: 0 (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          endpoints: []
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-        prefix: alertmanagers/ (default)
-      replication_factor: 3 (default)
-      zone_awareness_enabled: false (default)
-  alertmanager_storage:
-    azure:
-      account_key: ' (default)'
-      account_name: ' (default)'
-      container_name: ' (default)'
-      endpoint_suffix: ' (default)'
-      max_retries: 20 (default)
-      msi_resource: ' (default)'
-      user_assigned_id: ' (default)'
-    filesystem:
-      dir: alertmanager (default)
-    gcs:
-      service_account: ' (default)'
-    local:
-      path: ' (default)'
-    s3:
-      access_key_id: ' (default)'
-      bucket_name: ' (default)'
-      endpoint: ' (default)'
-      http:
-        expect_continue_timeout: 1s (default)
-        idle_conn_timeout: 1m30s (default)
-        insecure_skip_verify: false (default)
-        max_connections_per_host: 0 (default)
-        max_idle_connections: 100 (default)
-        max_idle_connections_per_host: 100 (default)
-        response_header_timeout: 2m0s (default)
-        tls_handshake_timeout: 10s (default)
-      insecure: false (default)
-      region: ' (default)'
-      secret_access_key: ' (default)'
-      signature_version: v4 (default)
-      sse:
-        kms_encryption_context: ' (default)'
-        kms_key_id: ' (default)'
-        type: ' (default)'
-    storage_prefix: ' (default)'
-    swift:
-      auth_url: ' (default)'
-      auth_version: 0 (default)
-      connect_timeout: 10s (default)
-      container_name: ' (default)'
-      domain_id: ' (default)'
-      domain_name: ' (default)'
-      max_retries: 3 (default)
-      password: ' (default)'
-      project_domain_id: ' (default)'
-      project_domain_name: ' (default)'
-      project_id: ' (default)'
-      project_name: ' (default)'
-      region_name: ' (default)'
-      request_timeout: 5s (default)
-      user_domain_id: ' (default)'
-      user_domain_name: ' (default)'
-      user_id: ' (default)'
-      username: ' (default)'
-  api:
-    alertmanager_http_prefix: /alertmanager (default)
-    prometheus_http_prefix: /prometheus (default)
-    skip_label_name_validation_header_enabled: false (default)
-  blocks_storage:
-    azure:
-      account_key: ' (default)'
-      account_name: ' (default)'
-      container_name: ' (default)'
-      endpoint_suffix: ' (default)'
-      max_retries: 20 (default)
-      msi_resource: ' (default)'
-      user_assigned_id: ' (default)'
-    bucket_store:
-      block_sync_concurrency: 20 (default)
-      bucket_index:
-        enabled: true (default)
-        idle_timeout: 1h0m0s (default)
-        max_stale_period: 1h0m0s (default)
-        update_on_error_interval: 1m0s (default)
-      chunk_pool_max_bucket_size_bytes: 50000000 (default)
-      chunk_pool_min_bucket_size_bytes: 16000 (default)
-      chunks_cache:
-        attributes_in_memory_max_items: 50000 (default)
-        attributes_ttl: 168h0m0s (default)
-        backend: memcached
-        max_get_range_requests: 3 (default)
-        memcached:
-          max_async_buffer_size: 25000 (default)
-          max_async_concurrency: 50 (default)
-          max_get_multi_batch_size: 100 (default)
-          max_get_multi_concurrency: 100 (default)
-          max_idle_connections: 100 (default)
-          max_item_size: 1048576 (default)
-          timeout: 450ms
-        subrange_size: 16000 (default)
-        subrange_ttl: 24h0m0s (default)
-      consistency_delay: 0s (default)
-      ignore_blocks_within: 10h0m0s (default)
-      ignore_deletion_mark_delay: 1h0m0s (default)
-      index_cache:
-        backend: memcached
-        inmemory:
-          max_size_bytes: 1073741824 (default)
-        memcached:
-          max_async_buffer_size: 25000 (default)
-          max_async_concurrency: 50 (default)
-          max_get_multi_batch_size: 100 (default)
-          max_get_multi_concurrency: 100 (default)
-          max_idle_connections: 100 (default)
-          timeout: 200ms (default)
-      index_header:
-        map_populate_enabled: false (default)
-      index_header_lazy_loading_enabled: true (default)
-      index_header_lazy_loading_idle_timeout: 1h0m0s (default)
-      max_concurrent: 100 (default)
-      meta_sync_concurrency: 20 (default)
-      metadata_cache:
-        backend: memcached
-        block_index_attributes_ttl: 168h0m0s (default)
-        bucket_index_content_ttl: 5m0s (default)
-        bucket_index_max_size_bytes: 1048576 (default)
-        chunks_list_ttl: 24h0m0s (default)
-        memcached:
-          max_async_buffer_size: 25000 (default)
-          max_async_concurrency: 50 (default)
-          max_get_multi_batch_size: 100 (default)
-          max_get_multi_concurrency: 100 (default)
-          max_idle_connections: 100 (default)
-          max_item_size: 1048576 (default)
-          timeout: 200ms (default)
-        metafile_attributes_ttl: 168h0m0s (default)
-        metafile_content_ttl: 24h0m0s (default)
-        metafile_doesnt_exist_ttl: 5m0s (default)
-        metafile_exists_ttl: 2h0m0s (default)
-        metafile_max_size_bytes: 1048576 (default)
-        tenant_blocks_list_ttl: 5m0s (default)
-        tenants_list_ttl: 15m0s (default)
-      partitioner_max_gap_bytes: 524288 (default)
-      postings_offsets_in_mem_sampling: 32 (default)
-      series_hash_cache_max_size_bytes: 1073741824 (default)
-      sync_interval: 15m0s (default)
-      tenant_sync_concurrency: 10 (default)
-    filesystem:
-      dir: blocks (default)
-    gcs:
-      service_account: ' (default)'
-    s3:
-      access_key_id: ' (default)'
-      bucket_name: ' (default)'
-      endpoint: ' (default)'
-      http:
-        expect_continue_timeout: 1s (default)
-        idle_conn_timeout: 1m30s (default)
-        insecure_skip_verify: false (default)
-        max_connections_per_host: 0 (default)
-        max_idle_connections: 100 (default)
-        max_idle_connections_per_host: 100 (default)
-        response_header_timeout: 2m0s (default)
-        tls_handshake_timeout: 10s (default)
-      insecure: false (default)
-      region: ' (default)'
-      secret_access_key: ' (default)'
-      signature_version: v4 (default)
-      sse:
-        kms_encryption_context: ' (default)'
-        kms_key_id: ' (default)'
-        type: ' (default)'
-    storage_prefix: ' (default)'
-    swift:
-      auth_url: ' (default)'
-      auth_version: 0 (default)
-      connect_timeout: 10s (default)
-      container_name: ' (default)'
-      domain_id: ' (default)'
-      domain_name: ' (default)'
-      max_retries: 3 (default)
-      password: ' (default)'
-      project_domain_id: ' (default)'
-      project_domain_name: ' (default)'
-      project_id: ' (default)'
-      project_name: ' (default)'
-      region_name: ' (default)'
-      request_timeout: 5s (default)
-      user_domain_id: ' (default)'
-      user_domain_name: ' (default)'
-      user_id: ' (default)'
-      username: ' (default)'
-    tsdb:
-      block_ranges_period:
-        - 2h0m0s (default)
-      close_idle_tsdb_timeout: 13h0m0s (default)
-      flush_blocks_on_shutdown: false (default)
-      head_chunks_end_time_variance: 0 (default)
-      head_chunks_write_buffer_size_bytes: 4194304 (default)
-      head_chunks_write_queue_size: 0 (default)
-      head_compaction_concurrency: 5 (default)
-      head_compaction_idle_timeout: 1h0m0s (default)
-      head_compaction_interval: 1m0s (default)
-      isolation_enabled: false (default)
-      max_tsdb_opening_concurrency_on_startup: 10 (default)
-      memory_snapshot_on_shutdown: false (default)
-      new_chunk_disk_mapper: false (default)
-      out_of_order_capacity_max: 32 (default)
-      out_of_order_capacity_min: 4 (default)
-      retention_period: 24h0m0s (default)
-      series_hash_cache_max_size_bytes: 1073741824 (default)
-      ship_concurrency: 10 (default)
-      ship_interval: 1m0s (default)
-      stripe_size: 16384 (default)
-      wal_compression_enabled: false (default)
-      wal_segment_size_bytes: 134217728 (default)
-  common:
-    storage:
-      azure:
-        account_key: ' (default)'
-        account_name: ' (default)'
-        container_name: ' (default)'
-        endpoint_suffix: ' (default)'
-        max_retries: 20 (default)
-        msi_resource: ' (default)'
-        user_assigned_id: ' (default)'
-      backend: filesystem (default)
-      filesystem:
-        dir: ' (default)'
-      gcs:
-        bucket_name: ' (default)'
-        service_account: ' (default)'
-      s3:
-        access_key_id: ' (default)'
-        bucket_name: ' (default)'
-        endpoint: ' (default)'
-        http:
-          expect_continue_timeout: 1s (default)
-          idle_conn_timeout: 1m30s (default)
-          insecure_skip_verify: false (default)
-          max_connections_per_host: 0 (default)
-          max_idle_connections: 100 (default)
-          max_idle_connections_per_host: 100 (default)
-          response_header_timeout: 2m0s (default)
-          tls_handshake_timeout: 10s (default)
-        insecure: false (default)
-        region: ' (default)'
-        secret_access_key: ' (default)'
-        signature_version: v4 (default)
-        sse:
-          kms_encryption_context: ' (default)'
-          kms_key_id: ' (default)'
-          type: ' (default)'
-      swift:
-        auth_url: ' (default)'
-        auth_version: 0 (default)
-        connect_timeout: 10s (default)
-        container_name: ' (default)'
-        domain_id: ' (default)'
-        domain_name: ' (default)'
-        max_retries: 3 (default)
-        password: ' (default)'
-        project_domain_id: ' (default)'
-        project_domain_name: ' (default)'
-        project_id: ' (default)'
-        project_name: ' (default)'
-        region_name: ' (default)'
-        request_timeout: 5s (default)
-        user_domain_id: ' (default)'
-        user_domain_name: ' (default)'
-        user_id: ' (default)'
-        username: ' (default)'
-  compactor:
-    block_ranges:
-      - 2h0m0s (default)
-      - 12h0m0s (default)
-      - 24h0m0s (default)
-    block_sync_concurrency: 8 (default)
-    cleanup_concurrency: 20 (default)
-    cleanup_interval: 15m0s (default)
-    compaction_concurrency: 1 (default)
-    compaction_jobs_order: smallest-range-oldest-blocks-first (default)
-    compaction_retries: 3 (default)
-    consistency_delay: 0s (default)
-    disabled_tenants: ' (default)'
-    enabled_tenants: ' (default)'
-    max_compaction_time: 1h0m0s (default)
-    meta_sync_concurrency: 20 (default)
-    sharding_ring:
-      heartbeat_period: 5s (default)
-      heartbeat_timeout: 1m0s (default)
-      instance_addr: ' (default)'
-      instance_id: fv-az459-214 (default)
-      instance_interface_names:
-        - eth0 (default)
-        - docker0 (default)
-        - br-feec2824b200 (default)
-      instance_port: 0 (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          host: localhost:8500 (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          endpoints: []
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-        store: memberlist (default)
-      wait_active_instance_timeout: 10m0s (default)
-      wait_stability_max_duration: 5m0s (default)
-    tenant_cleanup_delay: 6h0m0s (default)
-  distributor:
-    forwarding:
-      enabled: false (default)
-      propagate_errors: true (default)
-      request_concurrency: 10 (default)
-      request_timeout: 10s (default)
-    ha_tracker:
-      ha_tracker_failover_timeout: 30s (default)
-      ha_tracker_update_timeout: 15s (default)
-      ha_tracker_update_timeout_jitter_max: 5s (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          host: localhost:8500 (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-    instance_limits:
-      max_inflight_push_requests: 2000 (default)
-      max_inflight_push_requests_bytes: 0 (default)
-      max_ingestion_rate: 0 (default)
-    max_recv_msg_size: 104857600 (default)
-    pool:
-      client_cleanup_period: 15s (default)
-      health_check_ingesters: true (default)
-    remote_timeout: 20s (default)
-    ring:
-      heartbeat_period: 5s (default)
-      heartbeat_timeout: 1m0s (default)
-      instance_addr: ' (default)'
-      instance_id: fv-az459-214 (default)
-      instance_interface_names:
-        - eth0 (default)
-        - docker0 (default)
-        - br-feec2824b200 (default)
-      instance_port: 0 (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          host: localhost:8500 (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          endpoints: []
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-        store: memberlist (default)
-  flusher:
-    exit_after_flush: true (default)
-  frontend:
-    address: ' (default)'
-    cache_results: true
-    cache_unaligned_requests: false (default)
-    downstream_url: ' (default)'
-    grpc_client_config:
-      backoff_config:
-        max_period: 10s (default)
-        max_retries: 10 (default)
-        min_period: 100ms (default)
-      backoff_on_ratelimits: false (default)
-      grpc_compression: ' (default)'
-      max_recv_msg_size: 104857600 (default)
-      max_send_msg_size: 104857600 (default)
-      rate_limit: 0 (default)
-      rate_limit_burst: 0 (default)
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    instance_interface_names:
-      - eth0 (default)
-      - docker0 (default)
-      - br-feec2824b200 (default)
-    max_body_size: 10485760 (default)
-    max_outstanding_per_tenant: 100 (default)
-    max_retries: 5 (default)
-    parallelize_shardable_queries: false (default)
-    port: 0 (default)
-    querier_forget_delay: 0s (default)
-    query_stats_enabled: true (default)
-    results_cache:
-      backend: memcached
-      compression: ' (default)'
-      memcached:
-        max_async_buffer_size: 25000 (default)
-        max_async_concurrency: 50 (default)
-        max_get_multi_batch_size: 100 (default)
-        max_get_multi_concurrency: 100 (default)
-        max_idle_connections: 100 (default)
-        max_item_size: 1048576 (default)
-    scheduler_dns_lookup_period: 10s (default)
-    scheduler_worker_concurrency: 5 (default)
-    split_queries_by_interval: 24h0m0s (default)
-  frontend_worker:
-    dns_lookup_duration: 10s (default)
-    grpc_client_config:
-      backoff_config:
-        max_period: 10s (default)
-        max_retries: 10 (default)
-        min_period: 100ms (default)
-      backoff_on_ratelimits: false (default)
-      grpc_compression: ' (default)'
-      max_recv_msg_size: 104857600 (default)
-      max_send_msg_size: 104857600 (default)
-      rate_limit: 0 (default)
-      rate_limit_burst: 0 (default)
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    id: ' (default)'
-  ingester:
-    active_series_metrics_enabled: true (default)
-    active_series_metrics_idle_timeout: 10m0s (default)
-    active_series_metrics_update_period: 1m0s (default)
-    ignore_series_limit_for_metric_names: ' (default)'
-    instance_limits:
-      max_inflight_push_requests: 30000 (default)
-      max_ingestion_rate: 0 (default)
-      max_series: 0 (default)
-      max_tenants: 0 (default)
-    metadata_retain_period: 10m0s (default)
-    rate_update_period: 15s (default)
-    ring:
-      excluded_zones: ' (default)'
-      final_sleep: 0s (default)
-      instance_addr: ' (default)'
-      instance_id: fv-az459-214 (default)
-      instance_interface_names:
-        - eth0 (default)
-        - docker0 (default)
-        - br-feec2824b200 (default)
-      instance_port: 0 (default)
-      join_after: 0s (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          host: localhost:8500 (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          endpoints: []
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-        store: memberlist (default)
-      min_ready_duration: 15s (default)
-      observe_period: 0s (default)
-      replication_factor: 3 (default)
-      zone_awareness_enabled: false (default)
-    tsdb_config_update_period: 15s (default)
-  ingester_client:
-    grpc_client_config:
-      backoff_config:
-        max_period: 10s (default)
-        max_retries: 10 (default)
-        min_period: 100ms (default)
-      backoff_on_ratelimits: false (default)
-      grpc_compression: ' (default)'
-      max_recv_msg_size: 104857600 (default)
-      max_send_msg_size: 104857600 (default)
-      rate_limit: 0 (default)
-      rate_limit_burst: 0 (default)
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-  limits:
-    active_series_custom_trackers: {}
-    active_series_custom_trackers_config: {}
-    alertmanager_max_alerts_count: 0 (default)
-    alertmanager_max_alerts_size_bytes: 0 (default)
-    alertmanager_max_config_size_bytes: 0 (default)
-    alertmanager_max_dispatcher_aggregation_groups: 0 (default)
-    alertmanager_max_template_size_bytes: 0 (default)
-    alertmanager_max_templates_count: 0 (default)
-    alertmanager_notification_rate_limit: 0 (default)
-    alertmanager_notification_rate_limit_per_integration: {}
-    alertmanager_receivers_firewall_block_cidr_networks: ' (default)'
-    alertmanager_receivers_firewall_block_private_addresses: false (default)
-    cardinality_analysis_enabled: false (default)
-    compactor_block_upload_enabled: false (default)
-    compactor_blocks_retention_period: 0s (default)
-    compactor_partial_block_deletion_delay: 0s (default)
-    compactor_split_and_merge_shards: 0 (default)
-    compactor_split_groups: 1 (default)
-    creation_grace_period: 10m (default)
-    drop_labels: []
-    enforce_metadata_metric_name: true (default)
-    forwarding_rules: {}
-    ha_cluster_label: cluster (default)
-    ha_max_clusters: 100 (default)
-    ha_replica_label: __replica__ (default)
-    ingestion_burst_size: 200000 (default)
-    ingestion_rate: 10000 (default)
-    ingestion_tenant_shard_size: 0 (default)
-    label_names_and_values_results_max_size_bytes: 419430400 (default)
-    label_values_max_cardinality_label_names_per_request: 100 (default)
-    max_fetched_chunk_bytes_per_query: 0 (default)
-    max_fetched_chunks_per_query: 2000000 (default)
-    max_fetched_series_per_query: 0 (default)
-    max_global_exemplars_per_user: 0 (default)
-    max_global_series_per_metric: 0 (default)
-    max_global_series_per_user: 150000 (default)
-    max_label_name_length: 1024 (default)
-    max_label_names_per_series: 30 (default)
-    max_label_value_length: 2048 (default)
-    max_labels_query_length: 0s (default)
-    max_metadata_length: 1024 (default)
-    max_queriers_per_tenant: 0 (default)
-    max_query_lookback: 0s (default)
-    max_query_parallelism: 14 (default)
-    out_of_order_time_window: 0s (default)
-    query_sharding_max_sharded_queries: 128 (default)
-    query_sharding_total_shards: 16 (default)
-    request_burst_size: 0 (default)
-    request_rate: 0 (default)
-    ruler_evaluation_delay_duration: 0s (default)
-    ruler_max_rules_per_rule_group: 20 (default)
-    ruler_tenant_shard_size: 0 (default)
-    s3_sse_kms_encryption_context: ' (default)'
-    s3_sse_kms_key_id: ' (default)'
-    s3_sse_type: ' (default)'
-    split_instant_queries_by_interval: 0s (default)
-    store_gateway_tenant_shard_size: 0 (default)
-  memberlist:
-    abort_if_cluster_join_fails: false (default)
-    advertise_addr: ' (default)'
-    advertise_port: 7946 (default)
-    bind_addr: []
-    bind_port: 7946 (default)
-    cluster_label: ' (default)'
-    cluster_label_verification_disabled: false (default)
-    dead_node_reclaim_time: 0s (default)
-    gossip_interval: 200ms (default)
-    gossip_nodes: 3 (default)
-    gossip_to_dead_nodes_time: 30s (default)
-    leave_timeout: 20s (default)
-    left_ingesters_timeout: 5m0s (default)
-    max_join_backoff: 1m0s (default)
-    max_join_retries: 10 (default)
-    message_history_buffer_bytes: 0 (default)
-    min_join_backoff: 1s (default)
-    node_name: ' (default)'
-    packet_dial_timeout: 2s (default)
-    packet_write_timeout: 5s (default)
-    pull_push_interval: 30s (default)
-    randomize_node_name: true (default)
-    rejoin_interval: 0s (default)
-    retransmit_factor: 4 (default)
-    stream_timeout: 10s (default)
-    tls_ca_path: ' (default)'
-    tls_cert_path: ' (default)'
-    tls_enabled: false (default)
-    tls_insecure_skip_verify: false (default)
-    tls_key_path: ' (default)'
-    tls_server_name: ' (default)'
-  no_auth_tenant: anonymous (default)
-  querier:
-    batch_iterators: true (default)
-    default_evaluation_interval: 1m0s (default)
-    iterators: false (default)
-    lookback_delta: 5m0s (default)
-    max_query_into_future: 10m0s (default)
-    max_samples: 50000000 (default)
-    query_ingesters_within: 13h0m0s (default)
-    query_store_after: 12h0m0s (default)
-    shuffle_sharding_ingesters_enabled: true (default)
-    store_gateway_client:
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    timeout: 2m0s (default)
-  query_scheduler:
-    grpc_client_config:
-      backoff_config:
-        max_period: 10s (default)
-        max_retries: 10 (default)
-        min_period: 100ms (default)
-      backoff_on_ratelimits: false (default)
-      grpc_compression: ' (default)'
-      max_recv_msg_size: 104857600 (default)
-      max_send_msg_size: 104857600 (default)
-      rate_limit: 0 (default)
-      rate_limit_burst: 0 (default)
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    max_outstanding_requests_per_tenant: 100 (default)
-    querier_forget_delay: 0s (default)
-  ruler:
-    alertmanager_client:
-      basic_auth_password: ' (default)'
-      basic_auth_username: ' (default)'
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    alertmanager_refresh_interval: 1m0s (default)
-    disabled_tenants: ' (default)'
-    enable_api: true (default)
-    enabled_tenants: ' (default)'
-    evaluation_interval: 1m0s (default)
-    external_url: ' (default)'
-    for_grace_period: 10m0s (default)
-    for_outage_tolerance: 1h0m0s (default)
-    notification_queue_capacity: 10000 (default)
-    notification_timeout: 10s (default)
-    poll_interval: 1m0s (default)
-    query_frontend:
-      address: ' (default)'
-      grpc_client_config:
-        backoff_config:
-          max_period: 10s (default)
-          max_retries: 10 (default)
-          min_period: 100ms (default)
-        backoff_on_ratelimits: false (default)
-        grpc_compression: ' (default)'
-        max_recv_msg_size: 104857600 (default)
-        max_send_msg_size: 104857600 (default)
-        rate_limit: 0 (default)
-        rate_limit_burst: 0 (default)
-        tls_ca_path: ' (default)'
-        tls_cert_path: ' (default)'
-        tls_enabled: false (default)
-        tls_insecure_skip_verify: false (default)
-        tls_key_path: ' (default)'
-        tls_server_name: ' (default)'
-    query_stats_enabled: false (default)
-    resend_delay: 1m0s (default)
-    ring:
-      heartbeat_period: 5s (default)
-      heartbeat_timeout: 1m0s (default)
-      instance_addr: ' (default)'
-      instance_id: fv-az459-214 (default)
-      instance_interface_names:
-        - eth0 (default)
-        - docker0 (default)
-        - br-feec2824b200 (default)
-      instance_port: 0 (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          host: localhost:8500 (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          endpoints: []
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-        prefix: rulers/ (default)
-        store: memberlist (default)
-      num_tokens: 128 (default)
-    ruler_client:
-      backoff_config:
-        max_period: 10s (default)
-        max_retries: 10 (default)
-        min_period: 100ms (default)
-      backoff_on_ratelimits: false (default)
-      grpc_compression: ' (default)'
-      max_recv_msg_size: 104857600 (default)
-      max_send_msg_size: 104857600 (default)
-      rate_limit: 0 (default)
-      rate_limit_burst: 0 (default)
-      tls_ca_path: ' (default)'
-      tls_cert_path: ' (default)'
-      tls_enabled: false (default)
-      tls_insecure_skip_verify: false (default)
-      tls_key_path: ' (default)'
-      tls_server_name: ' (default)'
-    tenant_federation:
-      enabled: false (default)
-  ruler_storage:
-    azure:
-      account_key: ' (default)'
-      account_name: ' (default)'
-      container_name: ' (default)'
-      endpoint_suffix: ' (default)'
-      max_retries: 20 (default)
-      msi_resource: ' (default)'
-      user_assigned_id: ' (default)'
-    filesystem:
-      dir: ruler (default)
-    gcs:
-      service_account: ' (default)'
-    local:
-      directory: ' (default)'
-    s3:
-      access_key_id: ' (default)'
-      bucket_name: ' (default)'
-      endpoint: ' (default)'
-      http:
-        expect_continue_timeout: 1s (default)
-        idle_conn_timeout: 1m30s (default)
-        insecure_skip_verify: false (default)
-        max_connections_per_host: 0 (default)
-        max_idle_connections: 100 (default)
-        max_idle_connections_per_host: 100 (default)
-        response_header_timeout: 2m0s (default)
-        tls_handshake_timeout: 10s (default)
-      insecure: false (default)
-      region: ' (default)'
-      secret_access_key: ' (default)'
-      signature_version: v4 (default)
-      sse:
-        kms_encryption_context: ' (default)'
-        kms_key_id: ' (default)'
-        type: ' (default)'
-    storage_prefix: ' (default)'
-    swift:
-      auth_url: ' (default)'
-      auth_version: 0 (default)
-      connect_timeout: 10s (default)
-      container_name: ' (default)'
-      domain_id: ' (default)'
-      domain_name: ' (default)'
-      max_retries: 3 (default)
-      password: ' (default)'
-      project_domain_id: ' (default)'
-      project_domain_name: ' (default)'
-      project_id: ' (default)'
-      project_name: ' (default)'
-      region_name: ' (default)'
-      request_timeout: 5s (default)
-      user_domain_id: ' (default)'
-      user_domain_name: ' (default)'
-      user_id: ' (default)'
-      username: ' (default)'
-  runtime_config:
-    period: 10s (default)
-  server:
-    graceful_shutdown_timeout: 30s (default)
-    grpc_listen_address: ' (default)'
-    grpc_listen_conn_limit: 0 (default)
-    grpc_listen_network: tcp (default)
-    grpc_listen_port: 9095 (default)
-    grpc_server_keepalive_time: 2h0m0s (default)
-    grpc_server_keepalive_timeout: 20s (default)
-    grpc_server_max_recv_msg_size: 104857600 (default)
-    grpc_server_max_send_msg_size: 104857600 (default)
-    grpc_server_min_time_between_pings: 10s (default)
-    grpc_server_ping_without_stream_allowed: true (default)
-    grpc_tls_config:
-      cert_file: ' (default)'
-      client_auth_type: ' (default)'
-      client_ca_file: ' (default)'
-      key_file: ' (default)'
-    http_listen_address: ' (default)'
-    http_listen_conn_limit: 0 (default)
-    http_listen_network: tcp (default)
-    http_listen_port: 8080 (default)
-    http_path_prefix: ' (default)'
-    http_server_idle_timeout: 2m0s (default)
-    http_server_read_timeout: 30s (default)
-    http_tls_config:
-      cert_file: ' (default)'
-      client_auth_type: ' (default)'
-      client_ca_file: ' (default)'
-      key_file: ' (default)'
-    log_format: logfmt (default)
-    log_level: info (default)
-    log_request_at_info_level_enabled: false (default)
-    log_source_ips_enabled: false (default)
-    log_source_ips_header: ' (default)'
-    log_source_ips_regex: ' (default)'
-    register_instrumentation: true (default)
-  store_gateway:
-    sharding_ring:
-      heartbeat_period: 15s (default)
-      heartbeat_timeout: 1m0s (default)
-      instance_addr: ' (default)'
-      instance_availability_zone: ' (default)'
-      instance_id: fv-az459-214 (default)
-      instance_interface_names:
-        - eth0 (default)
-        - docker0 (default)
-        - br-feec2824b200 (default)
-      instance_port: 0 (default)
-      kvstore:
-        consul:
-          acl_token: ' (default)'
-          cas_retry_delay: 1s (default)
-          consistent_reads: false (default)
-          host: localhost:8500 (default)
-          http_client_timeout: 20s (default)
-          watch_burst_size: 1 (default)
-          watch_rate_limit: 1 (default)
-        etcd:
-          dial_timeout: 10s (default)
-          endpoints: []
-          max_retries: 10 (default)
-          password: ' (default)'
-          tls_ca_path: ' (default)'
-          tls_cert_path: ' (default)'
-          tls_enabled: false (default)
-          tls_insecure_skip_verify: false (default)
-          tls_key_path: ' (default)'
-          tls_server_name: ' (default)'
-          username: ' (default)'
-        multi:
-          mirror_enabled: false (default)
-          mirror_timeout: 2s (default)
-          primary: ' (default)'
-          secondary: ' (default)'
-        store: memberlist (default)
-      replication_factor: 3 (default)
-      unregister_on_shutdown: true (default)
-      wait_stability_max_duration: 5m0s (default)
-      zone_awareness_enabled: false (default)
-  target: query-scheduler
-  tenant_federation:
-    enabled: false (default)
-  usage_stats:
-    enabled: false (default)
-kind: MimirConfig
-metadata:
-  name: query-scheduler
-  namespace: default

@pracucci pracucci merged commit 1fb9f6a into main Aug 17, 2022
@pracucci pracucci deleted the fix-usage-stats-report-retry branch August 17, 2022 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@colega colega colega approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pracucci @colega