Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE REQ] Enable application insights end to end tracing for Event Grid #13272

Closed
Expecho opened this issue Jul 7, 2020 · 26 comments
Closed

[FEATURE REQ] Enable application insights end to end tracing for Event Grid #13272

Expecho opened this issue Jul 7, 2020 · 26 comments
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Event Grid Monitor Monitor, Monitor Ingestion, Monitor Query OpenTelemetry OpenTelemetry instrumentation (not Monitor-specific) question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention This issue is responsible by Azure service team.

Comments

@Expecho
Copy link

Expecho commented Jul 7, 2020 •
edited
Loading

Library or service name.
Microsoft.Azure.EventGrid

Is your feature request related to a problem? Please describe.
When using the SDK to send events and receive them (using Azure Functions for example) correlation data is not send which results in Application Insights not being able to correlate the event sending with the event handling.

I would like to see something similar as done with the Azure Service Bus.

Currently it is very hard to get end-to-end tracing when Event Grid is involved.

References
microsoft/ApplicationInsights-dotnet#1427
https://feedback.azure.com/forums/909934-azure-event-grid/suggestions/33306310-header-pass-through
https://feedback.azure.com/forums/357324-azure-monitor-application-insights/suggestions/40586104-support-for-correlated-tracing-through-event-grid
@ghost ghost added needs-triage This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jul 7, 2020
@jsquire jsquire added Client This issue points to a problem in the data-plane of the library. Event Grid needs-team-attention This issue needs attention from Azure service team or SDK team Service Attention This issue is responsible by Azure service team. labels Jul 7, 2020
@ghost ghost removed the needs-triage This is a new issue that needs to be triaged to the appropriate team. label Jul 7, 2020
@ghost
Copy link

ghost commented Jul 7, 2020

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl.

@jsquire
Copy link
Member

jsquire commented Jul 7, 2020

//fyi: @ellismg, @JoshLove-msft

@Expecho Expecho mentioned this issue Jul 22, 2020
Closed
@ahamad-MS ahamad-MS self-assigned this Jul 22, 2020
@ahamad-MS
Copy link
Contributor

Copied the same answer from issue #13575

We are adding distributed tracing and OpenTelemetry support for CloudEvent 1.0 in our Data plane SDKs which we are currently working on (No final ETA yet). We will follow the approach documented here for CloudEvent 1.0: https://github.com/cloudevents/spec/blob/master/extensions/distributed-tracing.md

We are still in discussion on how to support that in the SDK for event grid event schema. This will require modifying the current event grid schema to include new properties for tracing. Again, no commitment on this work or ETA yet.

Will you be able to use CloudEvent initially? Does this answer your question/request?

@ahamad-MS
Copy link
Contributor

@Expecho Please feel free to close one of the two issues and keep the other one for tracking since both are about the same support/ask. thanks

@jfggdl
Copy link

jfggdl commented Oct 2, 2020

@Expecho, in a week or so we are going to be releasing a new SDK preview version that provides distributed tracing support. Please note that this support is not offered in the current version of the SDKs (Microsoft.Azure.EventGrid which is the one that you are using), but for the new version currently in preview (Azure.Messaging.EventGrid). Please try it and let us know if you still have unmet requirements. This is the link to Azure SDKs. Just enter Event Grid in the search field: https://azure.github.io/azure-sdk/

@lmazuel lmazuel added this to the [2020] November milestone Oct 5, 2020
@lmazuel lmazuel assigned JoshLove-msft and unassigned ahamad-MS Oct 5, 2020
@jfggdl jfggdl removed the Service Attention This issue is responsible by Azure service team. label Oct 9, 2020
@ghost ghost added the needs-team-triage This issue needs the team to triage. label Oct 9, 2020
@jsquire jsquire modified the milestones: [2020] November, Backlog Nov 20, 2020
@jsquire jsquire added needs-author-feedback More information is needed from author to address the issue. and removed needs-team-attention This issue needs attention from Azure service team or SDK team needs-team-triage This issue needs the team to triage. labels Nov 23, 2020
@jsquire jsquire removed this from the Backlog milestone Nov 23, 2020
@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label Nov 30, 2020
@ghost
Copy link

ghost commented Nov 30, 2020

Hi, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@Expecho
Copy link
Author

Expecho commented Dec 1, 2020 •
edited
Loading

I suppose I have to wait for an update of the Microsoft.Azure.WebJobs.Extensions.Event package, otherwise my azure function won;t be able to use the new Azure.Messaging.EventGrid package. Are there plans to create a preview of Microsoft.Azure.WebJobs.Extensions.Event that references the new Azure.Messaging.EventGrid package?

And does the new package provide tracing for the event grid event schema as well, or is the CloudEvent schema the only supported schema? Looking at the source I suspect only CloudEvent is supported for the end to end tracing. Correct? If so, the bad thing is that Event Grid triggered Azure Functions only support the Event Grid Event schema:

Currently, you can't use an Event Grid trigger for an Azure Functions app when the event is delivered in the CloudEvents schema.

Of course I could use an http triggered function as outlined here but:

It is strongly recommended that you use an Event Grid Trigger when triggering an Azure Function with Event Grid. The use of a generic WebHook trigger here is demonstrative.

@ghost ghost added the needs-team-attention This issue needs attention from Azure service team or SDK team label Dec 1, 2020
@ghost
Copy link

ghost commented May 18, 2021

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl.

Issue Details

Library or service name.
Microsoft.Azure.EventGrid

Is your feature request related to a problem? Please describe.
When using the SDK to send events and receive them (using Azure Functions for example) correlation data is not send which results in Application Insights not being able to correlate the event sending with the event handling.

I would like to see something similar as done with the Azure Service Bus.

Currently it is very hard to get end-to-end tracing when Event Grid is involved.

References
microsoft/ApplicationInsights-dotnet#1427
https://feedback.azure.com/forums/909934-azure-event-grid/suggestions/33306310-header-pass-through
https://feedback.azure.com/forums/357324-azure-monitor-application-insights/suggestions/40586104-support-for-correlated-tracing-through-event-grid

Author: Expecho
Assignees: -
Labels:

Client, Event Grid, Service Attention, customer-reported, needs-team-attention, question
Milestone: -

@JoshLove-msft JoshLove-msft removed the needs-team-attention This issue needs attention from Azure service team or SDK team label May 18, 2021
@SaurabhSharma-MSFT
Copy link
Member

@jfggdl Can you please help on this.
@kevinp Were you able to add this feature to uservoice earlier ?

@SaurabhSharma-MSFT SaurabhSharma-MSFT added the needs-author-feedback More information is needed from author to address the issue. label Oct 29, 2021
@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label Nov 6, 2021
@ghost
Copy link

ghost commented Nov 6, 2021

Hi, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@ghost ghost closed this as completed Nov 21, 2021
@JoshLove-msft
Copy link
Member

This should not have been closed.

@JoshLove-msft JoshLove-msft reopened this Nov 21, 2021
@ghost ghost removed the no-recent-activity There has been no recent activity on this issue. label Nov 21, 2021
@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label Nov 28, 2021
@ghost
Copy link

ghost commented Nov 28, 2021

Hi, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@JoshLove-msft JoshLove-msft removed the needs-author-feedback More information is needed from author to address the issue. label Nov 28, 2021
@UsmanMohammad
Copy link

Just wondering if there has been any progress on this?

The combination of tracing and cloud events between event grid and azure functions is finnicky

We are seeing quite a gap as the documentation also says:

The Event Grid library supports distributing tracing out of the box. In order to adhere to the CloudEvents specification's guidance on distributing tracing, the library will set the traceparent and tracestate on the ExtensionAttributes of a CloudEvent when distributed tracing is enabled.

However we've not been able to see that working, in addition to noticing that eventgrid is ignoring traceparent header in the outgoing request.

There also seems to be a disconnect with the functions sdk and the event grid service. The functions sdk supports event grid trigger with cloud event schema, however event grid itself only sends EventGrid schema events to functions.

Is there a working roadmap for when these quirks will be ironed out?

@ghost ghost removed the no-recent-activity There has been no recent activity on this issue. label Nov 30, 2021
@navba-MSFT navba-MSFT removed the Service Attention This issue is responsible by Azure service team. label Feb 28, 2022
@ghost ghost added the needs-team-triage This issue needs the team to triage. label Feb 28, 2022
@navba-MSFT navba-MSFT added Service Attention This issue is responsible by Azure service team. and removed needs-team-triage This issue needs the team to triage. labels Feb 28, 2022
@ghost ghost added the needs-team-attention This issue needs attention from Azure service team or SDK team label Feb 28, 2022
@ghost
Copy link

ghost commented Feb 28, 2022

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl.

Issue Details

Library or service name.
Microsoft.Azure.EventGrid

Is your feature request related to a problem? Please describe.
When using the SDK to send events and receive them (using Azure Functions for example) correlation data is not send which results in Application Insights not being able to correlate the event sending with the event handling.

I would like to see something similar as done with the Azure Service Bus.

Currently it is very hard to get end-to-end tracing when Event Grid is involved.

References
microsoft/ApplicationInsights-dotnet#1427
https://feedback.azure.com/forums/909934-azure-event-grid/suggestions/33306310-header-pass-through
https://feedback.azure.com/forums/357324-azure-monitor-application-insights/suggestions/40586104-support-for-correlated-tracing-through-event-grid

Author: Expecho
Assignees: -
Labels:

Event Grid, Service Attention, Client, customer-reported, question
Milestone: -

@navba-MSFT navba-MSFT removed the needs-team-attention This issue needs attention from Azure service team or SDK team label Feb 28, 2022
@navba-MSFT
Copy link
Contributor

@Expecho @UsmanMohammad Apologies for the late reply. We are looking into this and we will provide an update once we have more details on this.

@JoshLove-msft @jfggdl Could you please provide an update on this github thread ? Awaiting your reply.

@lmolkova
Copy link
Member

lmolkova commented Jun 16, 2022 •
edited
Loading

I believe this was addressed in Microsoft.Azure.WebJobs.Extensions.EventGrid package version 3.1.0 for Event Grid trigger with CloudEvents.

Here's the blog with more information and customizations for EventGrid schema or HTTP hook:
Distributed tracing with Azure Functions Event Grid triggers

I'm closing this issue, but please feel to reopen.

@lmolkova lmolkova added Monitor Monitor, Monitor Ingestion, Monitor Query Monitor - ApplicationInsights OpenTelemetry OpenTelemetry instrumentation (not Monitor-specific) labels Jun 16, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Mar 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Event Grid Monitor Monitor, Monitor Ingestion, Monitor Query OpenTelemetry OpenTelemetry instrumentation (not Monitor-specific) question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Attention This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
14 participants
@kevinp @jsquire @lmazuel @pakrym @lmolkova @UsmanMohammad @jfggdl @scottaddie @Expecho @ahamad-MS @SaurabhSharma-MSFT @JoshLove-msft @navba-MSFT and others