Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stale markers #73

Merged
merged 1 commit into from
Dec 6, 2022
Merged

Stale markers #73

merged 1 commit into from
Dec 6, 2022

Conversation

codebien
Copy link
Contributor

@codebien codebien commented Nov 30, 2022

Added a Stale marker at the end of the test for all the seen time series.

Closes #68 #37

@codebien codebien self-assigned this Nov 30, 2022
@codebien codebien changed the base branch from main to stats-time-seconds November 30, 2022 17:39
@codebien codebien force-pushed the stale-marker branch 2 times, most recently from 12a0f06 to 3f60417 Compare December 1, 2022 11:01
@codebien codebien force-pushed the stale-marker branch 3 times, most recently from d1dc351 to 023b1f2 Compare December 1, 2022 11:58
Base automatically changed from stats-time-seconds to main December 1, 2022 11:59
@codebien codebien force-pushed the stale-marker branch 5 times, most recently from 53068f2 to 7946600 Compare December 2, 2022 22:31
@codebien codebien marked this pull request as ready for review December 2, 2022 22:33
Copy link
Member

@oleiade oleiade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👏🏻 🎅🏻 🎄

// a time series as stale.
//
// https://pkg.go.dev/github.com/prometheus/prometheus/pkg/value#pkg-constants
staleNaN = math.Float64frombits(0x7ff0000000000002)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to make this particular value a constant?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just use the StaleNaN constant from the prometheus package directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm looking into this but I'm not sure that the alternative is really better because the const would be a uint64 and it would be accepted if assigned to a float64. So if we forget to convert with math.Float64frombits( then we introduce a bug because the compiler doesn't complain. We should only rely on unit tests. @oleiade WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just use the StaleNaN constant from the prometheus package directly?

@imiric It would require including the Prometheus project as a direct dependency, but it has a lot of dependencies.

Copy link

@imiric imiric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'm not very familiar with the codebase yet, so pardon the possibly dumb questions 😅

// a time series as stale.
//
// https://pkg.go.dev/github.com/prometheus/prometheus/pkg/value#pkg-constants
staleNaN = math.Float64frombits(0x7ff0000000000002)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just use the StaleNaN constant from the prometheus package directly?

pkg/remotewrite/remotewrite.go Outdated Show resolved Hide resolved

// TODO: warn if it the request is taking too long (5 seconds?, the same timeout set for flushing)

err := o.client.Store(context.Background(), staleMarkers)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hhmm shouldn't this use some extension-wide context? Preferably one passed from k6? I now see that output.Params doesn't have a context. Maybe it should?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO, this falls down to grafana/k6#2430 (comment). We should consider prioritizing it during the next cycle.

For this version, I think the unique way we have is to implement the TODO directly now logging in case of a long Stop execution. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this version, I think the unique way we have is to implement the TODO directly now logging in case of a long Stop execution. WDYT?

Thinking better on this, the current client is already initialized with the Timeout value from the configuration, so the request timed out when the timeout interval is passed. It should be enough for warning in the case of a long-time request and to alarm for a required investigation into eventual bad performance between k6 and the Remote write endpoint.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the best approach would be. Just thought it would probably make more sense to use an existing context, instead of using context.Background() everywhere.

Intuitively, it makes sense to me for the extension context to be based on one received by k6, so that the extension code can cancel any operations in progress if the k6 context is cancelled (due to a signal, test abort, etc.). But if you think that the timeout value from the configuration is enough, then we could go with that instead. I think it can be done in a separate PR, though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intuitively, it makes sense to me for the extension context to be based on one received by k6, so that the extension code can cancel any operations in progress if the k6 context is cancelled (due to a signal, test abort, etc.).

Just to be clear, I agree here, and this is something we want to add in the future, this is why the context is passed to the wait function in the Output refactor proposal (wait func(context.Context) error, err error).

Unfortunately, I think we can't do it now, because it would require some changes to k6 when the code is almost frozen.

pkg/remotewrite/remotewrite.go Show resolved Hide resolved
Copy link

@imiric imiric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done 👍

pkg/remotewrite/remotewrite.go Outdated Show resolved Hide resolved

// TODO: warn if it the request is taking too long (5 seconds?, the same timeout set for flushing)

err := o.client.Store(context.Background(), staleMarkers)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the best approach would be. Just thought it would probably make more sense to use an existing context, instead of using context.Background() everywhere.

Intuitively, it makes sense to me for the extension context to be based on one received by k6, so that the extension code can cancel any operations in progress if the k6 context is cancelled (due to a signal, test abort, etc.). But if you think that the timeout value from the configuration is enough, then we could go with that instead. I think it can be done in a separate PR, though.

pkg/remotewrite/remotewrite.go Show resolved Hide resolved
@codebien codebien merged commit 6d77bf6 into main Dec 6, 2022
@codebien codebien deleted the stale-marker branch December 6, 2022 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stale markers
3 participants