Skip to content

Add a metric for task duration in the pending queue#12492

Merged
zachjsh merged 4 commits intoapache:masterfrom
rockc2020:feature-task-pending-time
May 3, 2022
Merged

Add a metric for task duration in the pending queue#12492
zachjsh merged 4 commits intoapache:masterfrom
rockc2020:feature-task-pending-time

Conversation

@rockc2020
Copy link
Contributor

@rockc2020 rockc2020 commented Apr 28, 2022

Description

This PR is to measure how long a task stays in the pending queue and emits the value with the metric task/pending/time. The metric is measured in RemoteTaskRunner and HttpRemoteTaskRunner.

An example of the metric:

2022-04-26T21:59:09,488 INFO [rtr-pending-tasks-runner-0] org.apache.druid.java.util.emitter.core.LoggingEmitter - {"feed":"metrics","timestamp":"2022-04-26T21:59:09.487Z","service":"druid/coordinator","host":"localhost:8081","version":"2022.02.0-iap-SNAPSHOT","metric":"task/pending/time","value":8,"dataSource":"wikipedia","taskId":"index_parallel_wikipedia_gecpcglg_2022-04-26T21:59:09.432Z","taskType":"index_parallel"}

Key changed/added classes in this PR
  • Emit metric task/pending/time in classes RemoteTaskRunner and HttpRemoteTaskRunner.
  • Update related factory classes and tests.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

Copy link
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems reasonable to me 👍

@zachjsh zachjsh merged commit 770ad95 into apache:master May 3, 2022
@rockc2020 rockc2020 deleted the feature-task-pending-time branch May 3, 2022 03:47
@abhishekagarwal87 abhishekagarwal87 added this to the 24.0.0 milestone Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants