Redacting urls before sending to telemetry data. #2549

rads-1996 · 2025-05-22T21:08:33Z

No description provided.

Copilot

Pull Request Overview

This PR introduces URL field redaction before sending telemetry data to prevent the exposure of sensitive information. Key changes include the addition and export of a new function (fieldRedaction) to sanitize URLs, updates to configuration and tests to control and validate the redaction behavior, and integration of redaction in multiple telemetry modules.

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
shared/AppInsightsCore/src/applicationinsights-core-js.ts	Exporting the new redaction function from EnvUtils.
shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts	Introducing constants and the fieldRedaction function to sanitize URLs.
shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/IConfiguration.ts	Adding a new redactionEnabled configuration property.
shared/AppInsightsCore/Tests/Unit/src/ApplicationInsightsCore.Tests.ts	Adding various test cases to validate the redaction function.
shared/AppInsightsCommon/src/Telemetry/Common/DataSanitizer.ts	Integrating fieldRedaction in the URL sanitization logic.
shared/AppInsightsCommon/Tests/Unit/src/AppInsightsCommon.tests.ts	Adding tests to verify dataSanitizeUrl behavior with redaction.
shared/1ds-core-js/test/Unit/src/FileSizeCheckTest.ts	Adjusting file size limits possibly related to size changes from redaction.
shared/1ds-core-js/src/Index.ts	Exporting fieldRedaction for use in other modules.
extensions/applicationinsights-dependencies-js/src/ajax.ts	Redacting URLs in AJAX calls when configured.
extensions/applicationinsights-clickanalytics-js/src/DataCollector.ts	Applying fieldRedaction in URL sanitization for click analytics.
extensions/applicationinsights-analytics-js/src/JavaScriptSDK/Telemetry/PageViewManager.ts	Updating page view tracking to include URL redaction.
extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts	Incorporating fieldRedaction across telemetry events to ensure sensitive data is redacted.

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts

shared/1ds-core-js/test/Unit/src/FileSizeCheckTest.ts

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

.github/workflows/ci.yml

.github/workflows/copilot-triage-assignment.yml

rads-1996 · 2025-05-28T22:22:39Z

Please note, this is not the complete set of changes, I am still working on adding fieldRedaction in places where the url needs to be redacted.

shared/AppInsightsCommon/src/Telemetry/Common/DataSanitizer.ts

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/Telemetry/PageViewManager.ts

extensions/applicationinsights-dependencies-js/src/ajax.ts

extensions/applicationinsights-clickanalytics-js/src/DataCollector.ts

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

hectorhdzg · 2025-06-19T21:23:21Z

examples/dependency/src/dependencies-example-index.ts

@@ -35,6 +36,12 @@ function addDependencyInitializerOnClick() {
        // Change properties of telemetry event "before" it's been processed
        details.item.name = "dependency-name";
        details.item.properties.url = details.item?.target;
+        if (isString(details.item.properties.url)) {


You don't need to add this code in example apps we have here, we should be redacting the URL internally so this is more confusing than helpful

hectorhdzg · 2025-06-19T21:25:15Z

extensions/applicationinsights-clickanalytics-js/src/DataCollector.ts

@@ -141,7 +143,9 @@ export function sanitizeUrl(config: IClickAnalyticsConfiguration, location: Loca
    if (!!config.urlCollectQuery) { // false by default
        url += (isValueAssigned(location.search)? location.search : "");
    }
-
+    if (rootConfig) {


Instead of checking config everywhere you call fieldRedaction, add a config validation inside that method if is not already there

Yes, the config validation is inside the fieldRedaction method. I will remove this extra check.

hectorhdzg · 2025-06-19T21:27:25Z

extensions/applicationinsights-clickanalytics-js/src/events/PageAction.ts

@@ -126,6 +126,9 @@ export class PageAction extends WebEvent {

                pageActionEvent.timeToAction = _getTimeToClick();
                pageActionEvent.refUri = isValueAssigned(overrideValues.refUri) ? overrideValues.refUri : _self._config.coreData.referrerUri;
+                if (_self._clickAnalyticsPlugin.core.config) {
+                    pageActionEvent.refUri = fieldRedaction(pageActionEvent.refUri, _self._clickAnalyticsPlugin.core.config);


Do we want to redact URLs that are overridden and not automatically tracked?, in this case customer is explicitly adding this value right?

I think it is a good point! Does the spec mention about customer defined URLs? or should we add configs to allow disabling auto-redact urls?

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

shared/AppInsightsCore/src/applicationinsights-core-js.ts

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

hectorhdzg · 2025-06-19T21:44:40Z

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

+        return input;
+    }
+    if (input.indexOf(" ") !== -1) {
+        return input; // Checking for URLs with spaces


Why the redaction is cancelled if there is a space?

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/IConfiguration.ts

extensions/applicationinsights-clickanalytics-js/src/DataCollector.ts

Karlie-777 · 2025-06-20T18:24:02Z

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts

+    // Check if any parameters need redaction
+    let anyParamRedacted = false;
+    if (config && config.redactQueryParams) {
+        sensitiveParams = [...DEFAULT_SENSITIVE_PARAMS, ...config.redactQueryParams];


nit use concat instead of ... for minification purpose

shared/AppInsightsCommon/Tests/Unit/src/AppInsightsCommon.tests.ts

…lemetry data

…ieldredaction

Copilot AI review requested due to automatic review settings May 22, 2025 21:08

rads-1996 requested a review from a team as a code owner May 22, 2025 21:08

Copilot AI reviewed May 22, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Outdated Show resolved Hide resolved

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts Outdated Show resolved Hide resolved

shared/1ds-core-js/test/Unit/src/FileSizeCheckTest.ts Show resolved Hide resolved

MSNev reviewed May 22, 2025

View reviewed changes

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts Outdated Show resolved Hide resolved

MSNev reviewed May 22, 2025

View reviewed changes

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts Outdated Show resolved Hide resolved

github-advanced-security bot found potential problems May 28, 2025

View reviewed changes

MSNev reviewed May 28, 2025

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

MSNev reviewed May 28, 2025

View reviewed changes

.github/workflows/copilot-triage-assignment.yml Outdated Show resolved Hide resolved

rads-1996 force-pushed the rads-1996/page-view-telemetry branch from 777ec80 to 0e7a8e5 Compare May 28, 2025 22:19

rads-1996 requested a review from MSNev May 28, 2025 22:31

MSNev reviewed May 28, 2025

View reviewed changes

shared/AppInsightsCommon/src/Telemetry/Common/DataSanitizer.ts Outdated Show resolved Hide resolved

MSNev reviewed May 28, 2025

View reviewed changes

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/AnalyticsPlugin.ts Outdated Show resolved Hide resolved

MSNev reviewed May 28, 2025

View reviewed changes

extensions/applicationinsights-analytics-js/src/JavaScriptSDK/Telemetry/PageViewManager.ts Show resolved Hide resolved

MSNev reviewed May 28, 2025

View reviewed changes

extensions/applicationinsights-dependencies-js/src/ajax.ts Show resolved Hide resolved

MSNev reviewed May 28, 2025

View reviewed changes

extensions/applicationinsights-clickanalytics-js/src/DataCollector.ts Outdated Show resolved Hide resolved

rads-1996 added this to the 3.3.9 milestone Jun 3, 2025

Karlie-777 reviewed Jun 6, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Show resolved Hide resolved

Karlie-777 reviewed Jun 6, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Outdated Show resolved Hide resolved

rads-1996 requested a review from Karlie-777 June 10, 2025 21:15

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

Karlie-777 reviewed Jun 19, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Show resolved Hide resolved

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

shared/AppInsightsCore/src/applicationinsights-core-js.ts Show resolved Hide resolved

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Show resolved Hide resolved

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Show resolved Hide resolved

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK/EnvUtils.ts Outdated Show resolved Hide resolved

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

shared/AppInsightsCore/src/JavaScriptSDK.Interfaces/IConfiguration.ts Show resolved Hide resolved

hectorhdzg reviewed Jun 19, 2025

View reviewed changes

extensions/applicationinsights-clickanalytics-js/src/DataCollector.ts Show resolved Hide resolved

Karlie-777 reviewed Jun 20, 2025

View reviewed changes

JacksonWeber reviewed Jun 20, 2025

View reviewed changes

shared/AppInsightsCommon/Tests/Unit/src/AppInsightsCommon.tests.ts Show resolved Hide resolved

rads-1996 added 6 commits June 23, 2025 16:49

Implement URL Redaction feature

5b8b1fa

Modified how IConfiguration config was being declared for some files

291d75b

Calling fieldRedaction method to redact URLs before being added to te…

1a2f8dd

…lemetry data

Fixed the AISKU failing test and added edge cases to unit tests for f…

9dab772

…ieldredaction

Sensitive query keys is now configurable. Added corresponding unit tests

3d9a354

Addressed review comments

65a77aa

rads-1996 force-pushed the rads-1996/page-view-telemetry branch from b62bd9d to 65a77aa Compare June 23, 2025 23:50

rads-1996 added 2 commits June 23, 2025 18:47

Added concat for minification

40ef1cb

Using default params from internal constants

8392bcd

rads-1996 requested a review from Karlie-777 June 24, 2025 02:40

Redacting urls before sending to telemetry data. #2549

Are you sure you want to change the base?

Redacting urls before sending to telemetry data. #2549

Uh oh!

Conversation

rads-1996 commented May 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rads-1996 commented May 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hectorhdzg Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

hectorhdzg Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

rads-1996 Jun 21, 2025

Choose a reason for hiding this comment

Uh oh!

hectorhdzg Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

Karlie-777 Jun 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hectorhdzg Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Karlie-777 Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Karlie-777 Jun 20, 2025 •

edited

Loading