Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugins: Fix status_source always being "plugin" in plugin request logs #77433

Merged
merged 3 commits into from
Oct 31, 2023

Conversation

xnyo
Copy link
Member

@xnyo xnyo commented Oct 31, 2023

What is this feature?

Fixes a bug where status_source in plugin request logs is always plugin, even if it was set to downstream by a plugin.

This happened because the logic for the status source was all contained in the metrics middleware, and the logger middleware is at a deeper level:

flowchart TD
a["MetricsMiddleware pre: Create pluginrequestmeta"]
--> QueryData
--> b["LoggerMiddleware post: Log status_source from context.Context"]
--> c["MetricsMiddleware post: Set status_source in context.Context"]
--> d["MetricsMiddleware post: Instrument using status_source in context.Context"]

This PR adds two new middlewares:

  • PluginRequestMetaMiddleware, which is at the top of the middleware stack, creates the plugin request meta in the context.Context, which is then available to all subsequent middlewares
  • StatusSourceMiddleware, at the bottom of the middleware stack, which sets the status source in the context.Context before returning, so all middlewares above it can access the correct status source after QueryData has returned
flowchart TD
a["PluginRequestMetaMiddleware: Create pluginrequestmeta"]
--> QueryData
--> c["StatusSourceMiddleware post: Set status_source in context.Context"]
--> b["LoggerMiddleware post: Log status_source from context.Context"]
--> d["MetricsMiddleware post: Instrument using status_source in context.Context"]

Why do we need this feature?

Bugfix

Who is this feature for?

Grafana operators

Which issue(s) does this PR fix?:

Fixes #

Special notes for your reviewer:

Please check that:

  • It works as expected from a user's perspective.
  • If this is a pre-GA feature, it is behind a feature toggle.
  • The docs are updated, and if this is a notable improvement, it's added to our What's New doc.

@xnyo xnyo added this to the 10.3.x milestone Oct 31, 2023
@xnyo xnyo self-assigned this Oct 31, 2023
@xnyo xnyo requested a review from a team as a code owner October 31, 2023 11:57
@xnyo xnyo requested review from wbrowne, marefr and oshirohugo and removed request for a team October 31, 2023 11:57
@xnyo
Copy link
Member Author

xnyo commented Oct 31, 2023

cc @ivanahuckova

@ivanahuckova
Copy link
Member

Thank you @xnyo for looking into this and fixing it! 🧡

Comment on lines +35 to +56
// Set downstream status source in the context if there's at least one response with downstream status source,
// and if there's no plugin error
var hasPluginError bool
var hasDownstreamError bool
for _, r := range resp.Responses {
if r.Error == nil {
continue
}
if r.ErrorSource == backend.ErrorSourceDownstream {
hasDownstreamError = true
} else {
hasPluginError = true
}
}

// A plugin error has higher priority than a downstream error,
// so set to downstream only if there's no plugin error
if hasDownstreamError && !hasPluginError {
if err := pluginrequestmeta.WithDownstreamStatusSource(ctx); err != nil {
return resp, fmt.Errorf("failed to set downstream status source: %w", err)
}
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same logic as before, but it has been moved from MetricsMiddleware

Copy link
Member

@marefr marefr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xnyo xnyo merged commit 46261de into main Oct 31, 2023
28 checks passed
@xnyo xnyo deleted the giuseppe/fix-error-source-context-logs branch October 31, 2023 12:42
grafana-delivery-bot bot pushed a commit that referenced this pull request Oct 31, 2023
…gs (#77433)

* Plugins: Fix status_source always being "plugin" in plugin logs

* add tests

* Fix TestInstrumentationMiddlewareStatusSource

(cherry picked from commit 46261de)
xnyo added a commit that referenced this pull request Oct 31, 2023
…request logs (#77436)

Plugins: Fix status_source always being "plugin" in plugin request logs (#77433)

* Plugins: Fix status_source always being "plugin" in plugin logs

* add tests

* Fix TestInstrumentationMiddlewareStatusSource

(cherry picked from commit 46261de)

Co-authored-by: Giuseppe Guerra <giuseppe.guerra@grafana.com>
ssama88 pushed a commit to ssama88/grafana that referenced this pull request Oct 31, 2023
…gs (grafana#77433)

* Plugins: Fix status_source always being "plugin" in plugin logs

* add tests

* Fix TestInstrumentationMiddlewareStatusSource
@aangelisc aangelisc modified the milestones: 10.3.x, 10.2.3 Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants