Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-30090] Support timespan for TimerGauges #21374

Merged
merged 1 commit into from
Nov 29, 2022

Conversation

gyfora
Copy link
Contributor

@gyfora gyfora commented Nov 23, 2022

What is the purpose of the change

Allow TimerGauges to define a time window similar to how MeterViews work in Flink. This allows us to collect more relevant metrics without losing information based on the metrics collection interval

Brief change log

  • Change logic to support custom timespan in TimerGauge
  • Add unit tests

Verifying this change

Extended existing unit tests to cover different timespan.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): yes
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

@gyfora gyfora requested a review from mxm November 23, 2022 13:11
@flinkbot
Copy link
Collaborator

flinkbot commented Nov 23, 2022

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

/** Circular array containing the history of values. */
private final long[] values;
/** The index in the array for the current time. */
private int time = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think idx would be a better name for this. Time is confusing because this actually gets reset to zero when the array is full.

long totalTime = 0;
for (int i = 0; i < maxIndex; i++) {
totalTime += values[i];
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the attention to detail here on properly calculating the totalTime based on the number of available observations :) Also, I very much prefer it over only releasing the Gauge once we have all observations. That way, this feature shouldn't cause any regressions because the Gauge as quickly as it does now but its accuracy will improve after a minute.

this.timeSpanInSeconds =
Math.max(
timeSpanInSeconds - (timeSpanInSeconds % UPDATE_INTERVAL_SECONDS),
UPDATE_INTERVAL_SECONDS);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be kind of nice not having to adjust this but we are constrained by the UPDATE_INTERVAL_SECONDS which we can't easily change. So looks good.

@gyfora
Copy link
Contributor Author

gyfora commented Nov 29, 2022

@flinkbot run azure

@gyfora gyfora merged commit a7fdab8 into apache:master Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants