Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

admin: Include tokio runtime metrics in the default metrics export #215

Merged
merged 11 commits into from
Dec 11, 2023

Conversation

olix0r
Copy link
Owner

@olix0r olix0r commented Dec 10, 2023

This change adds a new independent crate, kubert-prometheus-tokio, that exports Tokio runtime metrics when the tokio_unstable compiler feature is enabled. When these unstable features are enabled, a set of tokio runtime metrics is included in the default metrics export. For example:

# HELP tokio_rt_workers The number of worker threads used by the runtime.
# TYPE tokio_rt_workers gauge
tokio_rt_workers 16
# HELP tokio_rt_park Total number of times worker threads parked.
# TYPE tokio_rt_park counter
tokio_rt_park_total 1237
# HELP tokio_rt_noop Number of times workers unparked but found no new work.
# TYPE tokio_rt_noop counter
tokio_rt_noop_total 776
# HELP tokio_rt_steal Number of tasks stolen by workers from others.
# TYPE tokio_rt_steal counter
tokio_rt_steal_total 19
# HELP tokio_rt_steal_operations Number of times workers stole tasks from other.
# TYPE tokio_rt_steal_operations counter
tokio_rt_steal_operations_total 19
# HELP tokio_rt_remote_schedule Total number of remote schedule operations.
# TYPE tokio_rt_remote_schedule counter
tokio_rt_remote_schedule_total 27
# HELP tokio_rt_local_schedule Total number of local schedule operations.
# TYPE tokio_rt_local_schedule counter
tokio_rt_local_schedule_total 546
# HELP tokio_rt_overflow Total number of overflow operations.
# TYPE tokio_rt_overflow counter
tokio_rt_overflow_total 0
# HELP tokio_rt_polls The number of tasks that have been polled across all worker threads.
# TYPE tokio_rt_polls counter
tokio_rt_polls_total 468
# HELP tokio_rt_busy_seconds Total duration of time when worker threads were busy processing tasks.
# TYPE tokio_rt_busy_seconds counter
# UNIT tokio_rt_busy_seconds seconds
tokio_rt_busy_seconds_total 0.08628424000000004
# HELP tokio_rt_injection_queue_depth The number of tasks currently scheduled in the runtime's injection queue.
# TYPE tokio_rt_injection_queue_depth gauge
tokio_rt_injection_queue_depth 0
# HELP tokio_rt_local_queue_depth The total number of tasks currently scheduled in workers' local queues.
# TYPE tokio_rt_local_queue_depth gauge
tokio_rt_local_queue_depth 0
# HELP tokio_rt_budget_forced_yield Number of times a worker thread was forced to yield due to budget exhaustion.
# TYPE tokio_rt_budget_forced_yield counter
tokio_rt_budget_forced_yield_total 0
# HELP tokio_rt_io_driver_ready Number of times the IO driver was woken up.
# TYPE tokio_rt_io_driver_ready counter
tokio_rt_io_driver_ready_total 80

The kubert-prometheus-tokio crate provides standalone prometheus-client integration using the tokio-metrics crate. This is achieved by spawning a background task that updates these metrics once per second. The polling interval for the default metrics endpoint cannot currently be configured.

This change adds a new independent crate, kubert-prometheus-tokio, that
exports Tokio runtime metrics when the `tokio_unstable` compiler feature
is enabled. When these unstable features are enabled, a set of tokio
runtime metrics is included in the default metrics export. For example:

    # HELP tokio_rt_workers The number of worker threads used by the runtime.
    # TYPE tokio_rt_workers gauge
    tokio_rt_workers 16
    # HELP tokio_rt_park Total number of times worker threads parked.
    # TYPE tokio_rt_park counter
    tokio_rt_park_total 1237
    # HELP tokio_rt_noop Number of times workers unparked but found no new work.
    # TYPE tokio_rt_noop counter
    tokio_rt_noop_total 776
    # HELP tokio_rt_steal Number of tasks stolen by workers from others.
    # TYPE tokio_rt_steal counter
    tokio_rt_steal_total 19
    # HELP tokio_rt_steal_operations Number of times workers stole tasks from other.
    # TYPE tokio_rt_steal_operations counter
    tokio_rt_steal_operations_total 19
    # HELP tokio_rt_remote_schedule Total number of remote schedule operations.
    # TYPE tokio_rt_remote_schedule counter
    tokio_rt_remote_schedule_total 27
    # HELP tokio_rt_local_schedule Total number of local schedule operations.
    # TYPE tokio_rt_local_schedule counter
    tokio_rt_local_schedule_total 546
    # HELP tokio_rt_overflow Total number of overflow operations.
    # TYPE tokio_rt_overflow counter
    tokio_rt_overflow_total 0
    # HELP tokio_rt_polls The number of tasks that have been polled across all worker threads.
    # TYPE tokio_rt_polls counter
    tokio_rt_polls_total 468
    # HELP tokio_rt_busy_seconds Total duration of time when worker threads were busy processing tasks.
    # TYPE tokio_rt_busy_seconds counter
    # UNIT tokio_rt_busy_seconds seconds
    tokio_rt_busy_seconds_total 0.08628424000000004
    # HELP tokio_rt_injection_queue_depth The number of tasks currently scheduled in the runtime's injection queue.
    # TYPE tokio_rt_injection_queue_depth gauge
    tokio_rt_injection_queue_depth 0
    # HELP tokio_rt_local_queue_depth The total number of tasks currently scheduled in workers' local queues.
    # TYPE tokio_rt_local_queue_depth gauge
    tokio_rt_local_queue_depth 0
    # HELP tokio_rt_budget_forced_yield Number of times a worker thread was forced to yield due to budget exhaustion.
    # TYPE tokio_rt_budget_forced_yield counter
    tokio_rt_budget_forced_yield_total 0
    # HELP tokio_rt_io_driver_ready Number of times the IO driver was woken up.
    # TYPE tokio_rt_io_driver_ready counter
    tokio_rt_io_driver_ready_total 80

The kubert-prometheus-tokio crate provides standalone prometheus-client
integration using the tokio-metrics crate. This is achieved by spawning
a background task that updates these metrics once per second. The
polling interval for the default metrics endpoint cannot currently be
configured.
@olix0r olix0r merged commit a4efe89 into main Dec 11, 2023
45 of 46 checks passed
@olix0r olix0r deleted the ver/prom-tokio branch December 11, 2023 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant