Skip to content

Introduce Stats API #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 4, 2022
Merged

Introduce Stats API #191

merged 5 commits into from
Nov 4, 2022

Conversation

trueleo
Copy link
Contributor

@trueleo trueleo commented Nov 3, 2022

Description

Adds stats implementation based on atomic u64 counters. On each event ingestion counter is updated and on every sync parquet file size is added to compressed counter. On call to metadata api for stats it produces a json serializable Stats instance which is copy of metadata stats counter. Stats are synced on every s3 sync cycle and is stored inside parseable.json file.

Changes:

  • Stats module with StatsCounter and Stats type
  • Define stats handler for Actix
  • Update events and metadata method for stats
  • Add put_stats requirement for storage trait and provide s3 implementation

This PR has:

  • been tested to ensure log ingestion and log query works.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

@trueleo trueleo changed the title Stats Introduce Stats API Nov 3, 2022
@trueleo trueleo requested a review from nitisht November 3, 2022 12:40
Adds stats implementation based on atomic u64 counters. On each event ingestion
counter is updated and on every sync parquet file size is added to compressed
counter. On call to metadata api for stats it produces a json serializable
`Stats` instance which is copy of metadata stats counter. Stats are synced on
every s3 sync cycle and is stored inside parseable.json file.
@nitisht nitisht merged commit 0fa77b8 into parseablehq:main Nov 4, 2022
@trueleo trueleo deleted the stats branch November 19, 2022 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants