Instance Resource Count Metrics

[hifabienne]

As an administrator, I want to retrieve counts for organizations, users (with various filters), projects, applications, and authorizations within my ZITADEL instance, so that I can gain insights into resource utilization and manage my instance effectively.

Description

To effectively manage a ZITADEL instance, administrators need to understand resource utilization. This feature provides metrics for resource counts, including organizations, users (with filtering options), projects, applications, and authorizations. For users, we will offer filters to retrieve the total count, counts per organization, and counts by user type (human or machine). These metrics will provide administrators with valuable insights into the scale and complexity of their ZITADEL instance.

Acceptance Criteria:

• Organization Count Metric:
  • Given: An administrator requests instance resource counts.
  • Then: The system should provide the total count of organizations within the instance.
• User Count Metric (Total):
  • Given: An administrator requests instance resource counts.
  • Then: The system should provide the total count of users within the instance.
• User Count Metric (Per Organization):
  • Given: An administrator requests user counts filtered by organization.
  • Then: The system should provide a list of organization IDs and corresponding user counts for each organization.
  • User Count Metric (Per User Type):
    • Given: An administrator requests user counts filtered by user type.
    • Then: The system should provide counts for human users and machine users separately.
• Project Count Metric:
  • Given: An administrator requests instance resource counts.
  • Then: The system should provide the total count of projects within the instance.
• Project Grant Count Metric:
  • Given: An administrator requests instance resource counts.
  • Then: The system should provide the total count of projects grants within the instance.
• Application Count Metric:
  • Given: An administrator requests instance resource counts.
  • Then: The system should provide the total count of applications within the instance.
• Authorization Count Metric:
  • Given: An administrator requests instance resource counts.
  • Then: The system should provide the total count of authorizations within the instance.
• Data Accessibility (UI):
  • Given: An administrator accesses the management console UI.
  • Then: There should be a section or page dedicated to displaying instance resource count metrics.
  • And: The metrics should be presented in a clear and understandable format.
  • And: There should be filtering capability for the user counts.
• Data Accessibility (API):
  • Given: An administrator requests instance resource count metrics through an API.
  • Then: The system should provide an API endpoint to retrieve resource counts.
  • And: The API should return data in a standard format (e.g., JSON).
  • And: The API should support filtering for user counts.
• Performance:
  • Given: The system retrieves instance resource count metrics.
  • Then: The retrieval should be performed efficiently without significant performance degradation.
• Documentation:
  • Given: An administrator or developer accesses the system documentation.
  • Then: The documentation should explain:
    • The available resource count metrics.
    • How to access the metrics through the UI and API.
    • The filtering options for user counts.
    • The format of the data.
• Testing:
  • Given: The instance resource count metrics feature is implemented.
  • Then:
    • Unit tests should verify individual metric retrieval functions.
    • Integration tests should verify the interaction between different data access components.
    • End-to-end tests should simulate user workflows, including UI and API access.
    • Test cases should cover valid and invalid filtering scenarios for user counts.

Architecture

• This is implemented in a grpc analytics API
• We don't add an additional storage, all of this will be implemented with postgreSQL
• The metrics will be calculated in a scheduled job and stored to a specific table (e.g resource counts)
  • Computed number should be stored. (e.g users, count, day)
  • Granularity should be on days
  • Interval should be defined
  • Counts for different resources should be sequentially to avoid blocking when we compute all the metrics