Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Avoid overwriting existing trace header #449

Merged
merged 1 commit into from
Sep 13, 2023

Conversation

limhjgrace
Copy link
Contributor

@limhjgrace limhjgrace commented Sep 12, 2023

When a CW Synthetics canary has ActiveTracing enabled and generates activity on a web application monitored by a CW RUM app monitor that also has ActiveTracing enabled, users experience a broken X-Ray trace experience where the segment from the canary to the web application is disconnected from the downstream segments. This is because the RUM Web Client overwrites the X-Ray trace header in the HTTP requests with a new trace while the RUM DataPlane drops activity generated by CW Synthetics canaries (and thus the trace is never created/sent to X-Ray for ingestion).

This PR will update the Web Client tracing behavior to (1) not trace HTTP requests if the activity is generated by a CW Synthetics canary and (2) not overwrite existing trace headers.

Unit tests should be enough to cover this change, so no integrations were written.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@williazz
Copy link
Contributor

Question: why does Xhr not need to test for the following required for Fetch?

  1. "when fetch is called and request has existing trace header then the plugin records a trace with existing trace data"
  2. "when fetch is called and request has existing trace header then existing trace data is added to the http event"

});
test('when fetch is called and request has existing trace header then existing trace data is added to the http event', async () => {
// Init
console.log(navigator.userAgent);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
console.log(navigator.userAgent);

@qhanam
Copy link
Member

qhanam commented Sep 12, 2023

nit: "Use the imperative mood in the subject line"

E.g., fix: Avoid overwriting existing trace header

@limhjgrace limhjgrace changed the title fix: don't trace synthetic activity or overwrite existing trace header fix: Avoid overwriting existing trace header Sep 12, 2023
@limhjgrace
Copy link
Contributor Author

Question: why does Xhr not need to test for the following required for Fetch?

  1. "when fetch is called and request has existing trace header then the plugin records a trace with existing trace data"
  2. "when fetch is called and request has existing trace header then existing trace data is added to the http event"

For XHR requests, we're unable to access/read the request headers and thus, can't detect if there is an existing trace header. This results in the changes for the XHR plugin to be a subset of the changes for the Fetch plugin and so, some of the tests added for the Fetch plugin don't apply for the XHR plugin changes.

Comment on lines 186 to 188
export const isSyntheticsUA = () => {
return navigator.userAgent.includes('CloudWatchSynthetics');
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Slightly better might be to set a flag inside the constructor of XhrPlugin, so we don't text match against the user agent each time.

Comment on lines 108 to 109
traceId,
segmentId
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Should we create a new segment?

I think we only need to send a trace segment if we create a new one. Otherwise, we can assume that whoever set the trace header will also send the trace segment to X-Ray.

  1. Don't create a new segment. Use the trace Id and segment Id from the trace header to set trace_id and segment_id in htttp-event.
  2. Create a new segment. The new segment will be the child of the segment in the trace header. Change the segment Id in the trace header to be the Id of the new segment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we shouldn't create a new segment. Updated accordingly.

Comment on lines 93 to 100
const headers = (input as Request).headers;
if (headers) {
const headerComponents = headers.get(X_AMZN_TRACE_ID)?.split(';');
if (headerComponents?.length === 3) {
traceId = headerComponents[0].split('Root=')[1];
segmentId = headerComponents[1].split('Parent=')[1];
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Stick this in a helper function, and add unit tests. http-utils might be a good place.

@@ -272,7 +289,11 @@ export class FetchPlugin extends MonkeyPatched<Window, 'fetch'> {
return original.apply(thisArg, argsArray as any);
}

if (this.isTracingEnabled() && this.isSessionRecorded()) {
if (
!isSyntheticsUA() &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Look for an existing trace header before we reach this point. If there is a header, then simply set httpEvent.trace_id and httpEvent.segment_id. If there is no header, then beginTrace(...)

I don't think we need to use isSyntheticsUA here, since we know whether or not there is a trace header.

@limhjgrace limhjgrace merged commit 965ea07 into aws-observability:main Sep 13, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants