Skip to content

Controller and server total documents gauge for realtime tables are inconsistent #14390

@dang-stripe

Description

@dang-stripe

We noticed there was a discrepancy between the controller gauge for total documents vs. the one from servers for realtime tables. It seems like the former does not include docs from consuming segments and does not handle upsert tables well.

Example stats from an realtime non-upsert table:

  • Controller gauge: 1,127,658,711 docs
  • Server gauge: 1,095,767,023 docs

Example stats from a realtime upsert table, an order of magnitude different:

  • Controller gauge: 30,271,741 docs
  • Server gauge: 6,016,144 docs

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is not working as expectedingestionRelated to data ingestion pipelinemetricsRelated to metrics emission and collectionobservabilityRelated to observability (logging, tracing, metrics)

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions