From ff52799622bf8cbb40c7d0289739baebf8ea5db4 Mon Sep 17 00:00:00 2001 From: Anna Urbiztondo Date: Wed, 9 Mar 2022 14:19:57 +0100 Subject: [PATCH 01/25] chore(Alerts DD): 1st entry --- .../events/NrAiSignal/NrAiSignal.event-definition.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/accountId.md | 8 ++++++++ 2 files changed, 16 insertions(+) create mode 100644 src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md create mode 100644 src/data-dictionary/events/NrAiSignal/accountId.md diff --git a/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md b/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md new file mode 100644 index 00000000000..1f93bfdf1d4 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md @@ -0,0 +1,8 @@ +--- +name: NrAiSignal +type: event +dataSources: + - Alerts +--- + +'NrAiSignal' shows details from every NRQL alert condition and every signal on your account, for every aggregation window that passes. This data is posted immediately after each aggregation window is aggregated and evaluated, so it will show you exactly what [New Relic Alerts](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/get-started/alerts-ai-overview-page) is seeing. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/accountId.md b/src/data-dictionary/events/NrAiSignal/accountId.md new file mode 100644 index 00000000000..736db8b3b27 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/accountId.md @@ -0,0 +1,8 @@ +--- +name: aggregatedDataPointsCount +type: numeric +events: + - NrAiSignal +--- + +Count of the number of data points that were aggregated for this window. \ No newline at end of file From 848f3cc7d18f03799c53560a0d33f6f36bd379c8 Mon Sep 17 00:00:00 2001 From: Anna Urbiztondo Date: Wed, 9 Mar 2022 14:50:27 +0100 Subject: [PATCH 02/25] fix(Alerts DD): Trying to fix file --- .../events/NrAiSignal/NrAiSignal.event-definition.md | 2 +- .../NrAiSignal/{accountId.md => aggregatedDataPointsCount.md} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename src/data-dictionary/events/NrAiSignal/{accountId.md => aggregatedDataPointsCount.md} (100%) diff --git a/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md b/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md index 1f93bfdf1d4..99182b6f9a8 100644 --- a/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md +++ b/src/data-dictionary/events/NrAiSignal/NrAiSignal.event-definition.md @@ -5,4 +5,4 @@ dataSources: - Alerts --- -'NrAiSignal' shows details from every NRQL alert condition and every signal on your account, for every aggregation window that passes. This data is posted immediately after each aggregation window is aggregated and evaluated, so it will show you exactly what [New Relic Alerts](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/get-started/alerts-ai-overview-page) is seeing. \ No newline at end of file +NrAiSignal shows details from every NRQL alert condition and every signal on your account, for every aggregation window that passes. This data is posted immediately after each aggregation window is aggregated and evaluated, so it will show you exactly what [New Relic Alerts](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/get-started/alerts-ai-overview-page) is seeing. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/accountId.md b/src/data-dictionary/events/NrAiSignal/aggregatedDataPointsCount.md similarity index 100% rename from src/data-dictionary/events/NrAiSignal/accountId.md rename to src/data-dictionary/events/NrAiSignal/aggregatedDataPointsCount.md From 8ef38d51de3ce8a363c11ce6a02d320b62c6114e Mon Sep 17 00:00:00 2001 From: Anna Urbiztondo Date: Wed, 9 Mar 2022 20:22:00 +0100 Subject: [PATCH 03/25] fix(DD): Fixing type --- .../events/NrAiSignal/aggregatedDataPointsCount.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/data-dictionary/events/NrAiSignal/aggregatedDataPointsCount.md b/src/data-dictionary/events/NrAiSignal/aggregatedDataPointsCount.md index 736db8b3b27..94eea4247a1 100644 --- a/src/data-dictionary/events/NrAiSignal/aggregatedDataPointsCount.md +++ b/src/data-dictionary/events/NrAiSignal/aggregatedDataPointsCount.md @@ -1,6 +1,6 @@ --- name: aggregatedDataPointsCount -type: numeric +type: attribute events: - NrAiSignal --- From db2c8f7f4ecbc49cf1ece5db1f3a3f62ffc2617c Mon Sep 17 00:00:00 2001 From: Anna Urbiztondo Date: Thu, 10 Mar 2022 11:18:07 +0100 Subject: [PATCH 04/25] chore(Alerts DD): Adding rest of attributes --- .../events/NrAiSignal/aggregationDuration.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/conditionId.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/endTimestamp.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/entity.guid.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/entity.type.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/error.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/event.md | 11 +++++++++++ .../events/NrAiSignal/expirationDuration.md | 8 ++++++++ .../events/NrAiSignal/expirationLastSeenTime.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/fillOption.md | 8 ++++++++ .../events/NrAiSignal/numberOfDeviations.md | 8 ++++++++ .../events/NrAiSignal/predictedValue.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/resetCause.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/serverTime.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/signalId.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/signalValue.md | 8 ++++++++ .../events/NrAiSignal/slideByDuration.md | 8 ++++++++ .../events/NrAiSignal/standardDeviation.md | 8 ++++++++ .../events/NrAiSignal/tags..md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/timestamp.md | 8 ++++++++ src/data-dictionary/events/NrAiSignal/type.md | 8 ++++++++ 21 files changed, 171 insertions(+) create mode 100644 src/data-dictionary/events/NrAiSignal/aggregationDuration.md create mode 100644 src/data-dictionary/events/NrAiSignal/conditionId.md create mode 100644 src/data-dictionary/events/NrAiSignal/endTimestamp.md create mode 100644 src/data-dictionary/events/NrAiSignal/entity.guid.md create mode 100644 src/data-dictionary/events/NrAiSignal/entity.type.md create mode 100644 src/data-dictionary/events/NrAiSignal/error.md create mode 100644 src/data-dictionary/events/NrAiSignal/event.md create mode 100644 src/data-dictionary/events/NrAiSignal/expirationDuration.md create mode 100644 src/data-dictionary/events/NrAiSignal/expirationLastSeenTime.md create mode 100644 src/data-dictionary/events/NrAiSignal/fillOption.md create mode 100644 src/data-dictionary/events/NrAiSignal/numberOfDeviations.md create mode 100644 src/data-dictionary/events/NrAiSignal/predictedValue.md create mode 100644 src/data-dictionary/events/NrAiSignal/resetCause.md create mode 100644 src/data-dictionary/events/NrAiSignal/serverTime.md create mode 100644 src/data-dictionary/events/NrAiSignal/signalId.md create mode 100644 src/data-dictionary/events/NrAiSignal/signalValue.md create mode 100644 src/data-dictionary/events/NrAiSignal/slideByDuration.md create mode 100644 src/data-dictionary/events/NrAiSignal/standardDeviation.md create mode 100644 src/data-dictionary/events/NrAiSignal/tags..md create mode 100644 src/data-dictionary/events/NrAiSignal/timestamp.md create mode 100644 src/data-dictionary/events/NrAiSignal/type.md diff --git a/src/data-dictionary/events/NrAiSignal/aggregationDuration.md b/src/data-dictionary/events/NrAiSignal/aggregationDuration.md new file mode 100644 index 00000000000..8bfd5c4335d --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/aggregationDuration.md @@ -0,0 +1,8 @@ +--- +name: aggregationDuration +type: attribute +events: + - NrAiSignal +--- + +Duration of the aggregation window of the active condition, in seconds. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/conditionId.md b/src/data-dictionary/events/NrAiSignal/conditionId.md new file mode 100644 index 00000000000..87bd488490c --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/conditionId.md @@ -0,0 +1,8 @@ +--- +name: conditionId +type: attribute +events: + - NrAiSignal +--- + +ID of the active condition. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/endTimestamp.md b/src/data-dictionary/events/NrAiSignal/endTimestamp.md new file mode 100644 index 00000000000..3c68524dba0 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/endTimestamp.md @@ -0,0 +1,8 @@ +--- +name: endTimestamp +type: attribute +events: + - NrAiSignal +--- + +Moment when this event was created, and the end of the aggregation window. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/entity.guid.md b/src/data-dictionary/events/NrAiSignal/entity.guid.md new file mode 100644 index 00000000000..c8c0ac03551 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/entity.guid.md @@ -0,0 +1,8 @@ +--- +name: entity.guid +type: attribute +events: + - NrAiSignal +--- + +GUID of the active entity. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/entity.type.md b/src/data-dictionary/events/NrAiSignal/entity.type.md new file mode 100644 index 00000000000..8cabce7406c --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/entity.type.md @@ -0,0 +1,8 @@ +--- +name: entity.type +type: attribute +events: + - NrAiSignal +--- + +Type of the active entity. Note that `NA` indicates an infrastructure entity. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/error.md b/src/data-dictionary/events/NrAiSignal/error.md new file mode 100644 index 00000000000..2189dc3db7c --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/error.md @@ -0,0 +1,8 @@ +--- +name: error +type: attribute +events: + - NrAiSignal +--- + +This captures any errors when evaluating the signal. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/event.md b/src/data-dictionary/events/NrAiSignal/event.md new file mode 100644 index 00000000000..cd0e3b2b65e --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/event.md @@ -0,0 +1,11 @@ +--- +name: event +type: attribute +events: + - NrAiSignal +--- + +The type of event captured in this data point. +* Value: A single value was evaluated (most common). +* Expiration: A signal loss is detected. +* Reset: Indicates a detail of the condition was changed, and the signal has been reset. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/expirationDuration.md b/src/data-dictionary/events/NrAiSignal/expirationDuration.md new file mode 100644 index 00000000000..6c9314682e2 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/expirationDuration.md @@ -0,0 +1,8 @@ +--- +name: expirationDuration +type: attribute +events: + - NrAiSignal +--- + +Duration of the loss of signal of the active condition. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/expirationLastSeenTime.md b/src/data-dictionary/events/NrAiSignal/expirationLastSeenTime.md new file mode 100644 index 00000000000..e1c8cbb0170 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/expirationLastSeenTime.md @@ -0,0 +1,8 @@ +--- +name: expirationLastSeenTime +type: attribute +events: + - NrAiSignal +--- + +Exclusively for expiration events, it's the wall-clock time of the last data point received. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/fillOption.md b/src/data-dictionary/events/NrAiSignal/fillOption.md new file mode 100644 index 00000000000..b1aaeddc343 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/fillOption.md @@ -0,0 +1,8 @@ +--- +name: fillOption +type: attribute +events: + - NrAiSignal +--- + +Gap-filling setting from the active condition. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/numberOfDeviations.md b/src/data-dictionary/events/NrAiSignal/numberOfDeviations.md new file mode 100644 index 00000000000..a627b7bc03b --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/numberOfDeviations.md @@ -0,0 +1,8 @@ +--- +name: numberOfDeviations +type: attribute +events: + - NrAiSignal +--- + +For baseline conditions only. Indicates the number of standard deviations between the actual value and the predicted value. Formally called z-score or standard score. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/predictedValue.md b/src/data-dictionary/events/NrAiSignal/predictedValue.md new file mode 100644 index 00000000000..b20b3440239 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/predictedValue.md @@ -0,0 +1,8 @@ +--- +name: predictedValue +type: attribute +events: + - NrAiSignal +--- + +For baseline conditions only. Indicates the value that our models predicted for this data point. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/resetCause.md b/src/data-dictionary/events/NrAiSignal/resetCause.md new file mode 100644 index 00000000000..e11068aa5cd --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/resetCause.md @@ -0,0 +1,8 @@ +--- +name: resetCause +type: attribute +events: + - NrAiSignal +--- + +For reset events only. It's the reason why the evaluation stream was reset. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/serverTime.md b/src/data-dictionary/events/NrAiSignal/serverTime.md new file mode 100644 index 00000000000..3148ff1148a --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/serverTime.md @@ -0,0 +1,8 @@ +--- +name: serverTime +type: attribute +events: + - NrAiSignal +--- + +New Relic’s server clock time when the data point was evaluated. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/signalId.md b/src/data-dictionary/events/NrAiSignal/signalId.md new file mode 100644 index 00000000000..4b00a742dc2 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/signalId.md @@ -0,0 +1,8 @@ +--- +name: signalId +type: attribute +events: + - NrAiSignal +--- + +Unique identifier for this data stream. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/signalValue.md b/src/data-dictionary/events/NrAiSignal/signalValue.md new file mode 100644 index 00000000000..e0a88768ffe --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/signalValue.md @@ -0,0 +1,8 @@ +--- +name: signalValue +type: attribute +events: + - NrAiSignal +--- + +Value of the signal for this aggregation window. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/slideByDuration.md b/src/data-dictionary/events/NrAiSignal/slideByDuration.md new file mode 100644 index 00000000000..ed1157db87d --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/slideByDuration.md @@ -0,0 +1,8 @@ +--- +name: slideByDuration +type: attribute +events: + - NrAiSignal +--- + +For sliding windows conditions only. Duration of the slide-by interval, in seconds. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/standardDeviation.md b/src/data-dictionary/events/NrAiSignal/standardDeviation.md new file mode 100644 index 00000000000..755ee56c7d5 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/standardDeviation.md @@ -0,0 +1,8 @@ +--- +name: standardDeviation +type: attribute +events: + - NrAiSignal +--- + +Tells you how closely the signal value and the baseline prediction track each other: Standard deviation will be low when they track each other closely, and high when they don't. This attribute uses the same units as the signal. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/tags..md b/src/data-dictionary/events/NrAiSignal/tags..md new file mode 100644 index 00000000000..b84ca1ca458 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/tags..md @@ -0,0 +1,8 @@ +--- +name: tags. +type: attribute +events: + - NrAiSignal +--- + +All the system and custom tags that have been added to a signal. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/timestamp.md b/src/data-dictionary/events/NrAiSignal/timestamp.md new file mode 100644 index 00000000000..097276e2997 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/timestamp.md @@ -0,0 +1,8 @@ +--- +name: timestamp +type: attribute +events: + - NrAiSignal +--- + +Moment when this event was created, and the start of the aggregation window. \ No newline at end of file diff --git a/src/data-dictionary/events/NrAiSignal/type.md b/src/data-dictionary/events/NrAiSignal/type.md new file mode 100644 index 00000000000..61fb763bb75 --- /dev/null +++ b/src/data-dictionary/events/NrAiSignal/type.md @@ -0,0 +1,8 @@ +--- +name: type +type: attribute +events: + - NrAiSignal +--- + +Always = “signal”. \ No newline at end of file From ab4604e81fd94eff94b7211cee59f4348b571f08 Mon Sep 17 00:00:00 2001 From: helenapm Date: Thu, 10 Mar 2022 16:57:15 +0100 Subject: [PATCH 05/25] Rename availability as success The 'availability' term has generated some confusion because it usually refers to a service being up and running and able to receive requests. This is why we're renaming the suggested SLI as Success from now on. I haven't changed the anchor id, to prevent breaking any existing links. We'll publish a release note as well. --- .../docs/service-level-management/create-slm.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/content/docs/service-level-management/create-slm.mdx b/src/content/docs/service-level-management/create-slm.mdx index 35220a4325c..360e640cffe 100644 --- a/src/content/docs/service-level-management/create-slm.mdx +++ b/src/content/docs/service-level-management/create-slm.mdx @@ -56,9 +56,9 @@ Based on `Transaction` events, these SLIs are the most common for request-driven - Service availability is the ratio of the number of successful responses to the number of all requests. This effectively is an error rate, but you can filter it down, for example removing expected errors. + Service success is the ratio of the number of successful responses to the number of all requests. This effectively is an error rate, but you can filter it down, for example removing expected errors. **Valid events fields** @@ -123,9 +123,9 @@ Based on OpenTelemetry spans, these SLIs are the most common for request-driven - Service availability is the ratio of the number of successful responses to the number of all requests. This effectively is an error rate, but you can filter it down, for example removing expected errors. + Service success is the ratio of the number of successful responses to the number of all requests. This effectively is an error rate, but you can filter it down, for example removing expected errors. **Valid events fields** @@ -191,7 +191,7 @@ The following SLIs are based on Google's Browser Core Web Vitals. It's the proportion of page views that are served without errors. From 4bc083bd3e78baefe979ef72823a07771ff43a74 Mon Sep 17 00:00:00 2001 From: helenapm Date: Thu, 10 Mar 2022 17:51:22 +0100 Subject: [PATCH 06/25] Create service-levels-20220311.mdx Release note for the change on the suggested SLIs --- .../service-levels-20220311.mdx | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 src/content/docs/release-notes/service-levels-release-notes/service-levels-20220311.mdx diff --git a/src/content/docs/release-notes/service-levels-release-notes/service-levels-20220311.mdx b/src/content/docs/release-notes/service-levels-release-notes/service-levels-20220311.mdx new file mode 100644 index 00000000000..a10332a7785 --- /dev/null +++ b/src/content/docs/release-notes/service-levels-release-notes/service-levels-20220311.mdx @@ -0,0 +1,11 @@ +--- +title: 'Renaming availability SLI as success' +subject: Service levels +releaseDate: '2022-03-11' +--- + +New Relic suggests some typical SLIs for APM services and browser applications. If you choose to create one of the suggested SLIs through the UI, New Relic will analyze the latest data available, and will calculate the baseline for your SLI and SLO parameters. + +One of the typical SLIs checks the proportion of requests that are served without errors. When we launched the service levels beta release we decided to name this SLI *availability*; but this term has generated some confusion because it is often interpreted as a service being up and running and able to receive requests. This is why we're renaming the suggested SLI as *success* from now on. + +The existing SLIs created through New Relic suggested flows won't be renamed. From b6fb2d8bc5dda9636fb9160ee2cb7db38cda002a Mon Sep 17 00:00:00 2001 From: ZuluEcho9 Date: Thu, 10 Mar 2022 12:58:57 -0800 Subject: [PATCH 07/25] fix(user migration): some improvements --- .../original-users-roles/user-migration.mdx | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx index 63138ce39a3..d9cc3612ed4 100644 --- a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx +++ b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx @@ -82,13 +82,14 @@ If you've successfully completed migration, learn [how to manage your users](#ma ![User migration page 1](./images/user-migration-page-1.png "user-migration-page-1.png") Tips: +* Check the **Accounts included** dropdown. Note that the user migration will only apply to the account selected. This means if your organization has multiple accounts, you'll have to do the migration for all of them. * You can either a) import all current admins for your account or b) specify the admins that should have access to user management capabilities. Note that you can add more admin users and edit permissions after you complete the migration process. * If you've already used the wizard to set up an admin on the new user model, have the admin sign in using their new user record to access the migration tool. The user migration wizard, when completed, destroys the old user record, but if you've started the user migration process without completing it, you may have users with access to both the original and new record, as shown below: ![A screenshot of what is shown when you have an email address associated with multiple New Relic logins](./images/login-multiple-accounts-view.png "Login UI for email addresses with multiple user records")
If a user on the new model has been created and the migration process hasn't been completed, they may have access to both the original user record and the new user records.
-* If you plan on migrating only a portion of your users to the new user model to start, we recommend leaving some original user model admins so that you have an admin to manage your users on the original model. +* If you plan on migrating only a portion of your users to the new user model to start, we recommend leaving some original user model admins so that you have an admin to manage your original user model users. ## Step 2: Set up organization [#page2] @@ -109,7 +110,7 @@ Name your organization something descriptive and easily recognizable. ![User migration page 4](./images/user-migration-page-4.png "user-migration-page-4.png") -This section controls how users are provisioned (added to New Relic) and how they authenticate (log in). Note that choosing SAML SSO or SCIM setup will require you to exit the migration wizard and configure things elsewhere in the New Relic UI. +This section controls how users are provisioned (added to New Relic) and how they authenticate (log in). Note that choosing SAML SSO or SCIM setup will require you to exit the migration wizard and configure things elsewhere in the New Relic UI and in your identity provider. Here's more detail about the two authentication domain sections: @@ -145,14 +146,16 @@ Okta, Azure, and OneLogin have New Relic apps for both the original user model a **Recommended**: Download the full list of existing original user model users before choosing to import users. This will be a useful resource and serve as a backup, if you need it. -After downloading your original user model users, you can upload all users or just some of them. This step will create user records on the New Relic One user model. In a later step, you’ll be able to transition these users' assets. +After downloading your original user model users, you can upload all users or just some of them. We recommend reviewing this list of users and removing any users you don't want migrated (for example, people who no longer work at your organization). -The new user record that's created has the same login credentials: there is no need to reset passwords. If a user has a pending email verification status (pending being verified), that will also be transitioned over. +When you reupload your list of users and complete this step, it will create user records on the New Relic One user model. In a later step, you’ll be able to transition these users' assets. + +The new user records created for your users have the same login credentials: there's no need to reset passwords. If a user has a pending email verification status (pending being verified), that will also be transitioned over. Important tips: * Ensure the new users' email addresses match their original user record email addresses, including matching exact case. We use email addresses as the key value to match users and, in a later step, to transition their user-associated assets. -* Once you complete this step and create new user records, we highly recommend completing the migration process without delay. If you don't complete the steps to migrate assets and delete the original user record, a user may see two user records when logging in (see [login screenshot from Step 1](#page1)) or else may be missing assets they expect to see (like dashboards). +* Once you complete this step and create new user records, we highly recommend completing the migration process without delay. If you don't complete the steps to migrate assets and delete the original user record, a user may have two user records associated with the same login (see [login screenshot from Step 1](#page1)) or else may be missing assets they expect to see (like dashboards). ## Step 6: Access settings [#access-settings] @@ -185,6 +188,15 @@ If a user has access to several organizations that use New Relic (for example, i If you're migrating users in chunks and not all at once, you can go through the migration workflow several times with different groups of users. You can only click **Finish Setup** when all users in the organization are migrated. +## Troubleshooting [#troubleshooting] + +Some common problems after migration: + +* If you have admin-level roles assigned but get an error message when trying to access New Relic platform features, it may be because you've been assigned organization-scoped roles (**Organization manager** and/or **Authentication domain manager**) but not any account-scoped roles. To access New Relic features in a specific account, you'll need at least one account-scoped role (for example, **All product admin** or a custom role). +* If you've completed the migration, or are partway through the migration, and still see the original user management UI (the UI accessed through the **Account settings** tab), this may be because you are still logged in to your original user model record. Some remedies for this: + * Log out of New Relic and log back in, selecting the **Verify email** option. When you've verified your email, choose the login option that says "Organization" and not the one that says "Original New Relic account." + * If you're still having problems, clear your browser cache and attempt logging in again. + ## After you're done [#manage-users] Once your users are migrated to the new user model, you can find and manage them by clicking the [account dropdown](/docs/using-new-relic/welcome-new-relic/get-started/glossary/#account-dropdown), clicking **Administration**, and using these UI pages: From 965f3713448b6e1b92a494b7fc9bac6a4d340b0d Mon Sep 17 00:00:00 2001 From: zuluecho9 Date: Thu, 10 Mar 2022 13:01:48 -0800 Subject: [PATCH 08/25] fix(user migration): slight readability edit --- .../original-users-roles/user-migration.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx index d9cc3612ed4..d1d058ffd21 100644 --- a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx +++ b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx @@ -82,7 +82,7 @@ If you've successfully completed migration, learn [how to manage your users](#ma ![User migration page 1](./images/user-migration-page-1.png "user-migration-page-1.png") Tips: -* Check the **Accounts included** dropdown. Note that the user migration will only apply to the account selected. This means if your organization has multiple accounts, you'll have to do the migration for all of them. +* Check the **Accounts included** dropdown. Note that the user migration will only apply to the account selected. This means if your organization has multiple accounts, you should do the migration process for each of them. * You can either a) import all current admins for your account or b) specify the admins that should have access to user management capabilities. Note that you can add more admin users and edit permissions after you complete the migration process. * If you've already used the wizard to set up an admin on the new user model, have the admin sign in using their new user record to access the migration tool. The user migration wizard, when completed, destroys the old user record, but if you've started the user migration process without completing it, you may have users with access to both the original and new record, as shown below: ![A screenshot of what is shown when you have an email address associated with multiple New Relic logins](./images/login-multiple-accounts-view.png "Login UI for email addresses with multiple user records") From 1ad9d26c77cbc6285ca898f9796316f4f2e5a7be Mon Sep 17 00:00:00 2001 From: zuluecho9 Date: Thu, 10 Mar 2022 13:25:26 -0800 Subject: [PATCH 09/25] fix(user migration): attempt build retrigger --- .../original-users-roles/user-migration.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx index d1d058ffd21..b5908155bdd 100644 --- a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx +++ b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx @@ -1,7 +1,6 @@ --- title: Migrate your users to the New Relic One user model -metaDescription: >- - For New Relic accounts with users on our original user model: how to migrate your users to our new model. +metaDescription: "For New Relic accounts with users on our original user model: how to migrate your users to our new model." --- Starting April 12, 2021, we're allowing some customers who have users on our original user model to self-serve and migrate those users to be on the New Relic One user model. From 28a5bd49def2aed6d1f3cfce8e5135174b6da773 Mon Sep 17 00:00:00 2001 From: Anna Urbiztondo Date: Fri, 11 Mar 2022 08:35:42 +0100 Subject: [PATCH 10/25] fix(Alerts DD): Fixing definition --- src/data-dictionary/events/NrAiSignal/timestamp.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/data-dictionary/events/NrAiSignal/timestamp.md b/src/data-dictionary/events/NrAiSignal/timestamp.md index 097276e2997..6550f605bf6 100644 --- a/src/data-dictionary/events/NrAiSignal/timestamp.md +++ b/src/data-dictionary/events/NrAiSignal/timestamp.md @@ -5,4 +5,4 @@ events: - NrAiSignal --- -Moment when this event was created, and the start of the aggregation window. \ No newline at end of file +The start of the aggregation window. \ No newline at end of file From b73935354d47c1e239a5805848f9f4322c2a422e Mon Sep 17 00:00:00 2001 From: Barb Bryant Date: Fri, 11 Mar 2022 05:58:51 -0800 Subject: [PATCH 11/25] fix(xrefs): hero request Fixes https://github.com/newrelic/docs-website/issues/6532 --- .../elasticsearch/elasticsearch-integration.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/src/content/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration.mdx b/src/content/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration.mdx index c9c34289422..6727dfa43f3 100644 --- a/src/content/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration.mdx +++ b/src/content/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-integration.mdx @@ -18,14 +18,14 @@ import ecs from './images/ecs.png'; import kubernetes from './images/kubernetes-k8.png'; -Our Elasticsearch [integration](/docs/integrations/host-integrations/getting-started/introduction-host-integrations) collects and sends inventory and metrics from your [Elasticsearch](https://www.elastic.co/) cluster to our platform, where you can see the health of your Elasticsearch environment. We collect metrics at the cluster, node, and index level so you can more easily find the source of any problems. +Our Elasticsearch integration collects and sends inventory and metrics from your [Elasticsearch](https://www.elastic.co/) cluster to our platform, where you can see the health of your Elasticsearch environment. We collect metrics at the cluster, node, and index level so you can more easily find the source of any problems. To install the Elasticsearch monitoring integration, run through the following steps: 1. [Configure the integration](#config). 2. [Install and activate the integration](#install). 3. [Find and use data](#find-and-use). -4. Optionally, see [the advanced configuration settings](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-advanced-config). +4. Optionally, see the [advanced configuration settings](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-advanced-config). ## Compatibility and requirements [#req] @@ -53,7 +53,7 @@ There are several ways to configure the integration, depending on how it was ins - If enabled via ![Kubernetes](./images/kubernetes-k8.png 'Kubernetes')Kubernetes, see [Monitor services running on Kubernetes](/docs/monitor-service-running-kubernetes). - If enabled via ![ECS](./images/ecs.png 'ECS')Amazon ECS, see [Monitor services running on ECS](/docs/integrations/host-integrations/host-integrations-list/monitor-services-running-amazon-ecs). -- If installed on-host, edit the config in the integration's YAML config file, `elasticsearch-config.yml`. An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference. The configuration file has common settings applicable to all integrations, such as `interval`, `timeout`, `inventory_source`. To read all about these common settings, refer to our [Configuration Format](/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-newer-configuration-format/#configuration-basics) document. +- If installed on-host, edit the config in the integration's YAML config file, `elasticsearch-config.yml`. An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference. The configuration file has common settings applicable to all integrations, such as `interval`, `timeout`, `inventory_source`. To read all about these common settings, refer to our [configuration format](/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-newer-configuration-format/#configuration-basics) document. If you are still using our legacy configuration or definition files, check the [standard configuration format](/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-standard-configuration-format/). @@ -76,7 +76,7 @@ To install the Elasticsearch integration, follow the instructions for your envir sudo cp elasticsearch-config.yml.sample elasticsearch-config.yml ``` -4. Edit the `elasticsearch-config.yml` configuration file with your favorite editor. Check out some [great configuration file examples.](#examples). +4. Edit the `elasticsearch-config.yml` configuration file with your favorite editor. Check out some [great configuration file examples](#examples). 5. Restart the infrastructure agent. See how to [restart the infrastructure agent in different Linux environments](/docs/infrastructure/install-infrastructure-agent/manage-your-agent/start-stop-restart-infrastructure-agent/#linux). 6. To enable automatic Elasticsearch error log parsing and forwarding, copy (or rename) the `elasticsearch-log.yml.example` file to `elasticsearch-log.yml`. No need to restart the agent. @@ -278,10 +278,10 @@ Data from this service is reported to an [integration dashboard](/docs/integrati Elasticsearch data is attached to the following [event types](/docs/using-new-relic/data/understand-data/new-relic-data-types#events-new-relic): -- [`ElasticsearchClusterSample`](#elasticsearch-cluster-metrics) -- [`ElasticsearchNodeSample`](#elasticsearch-node-metrics) -- [`ElasticsearchCommonSample`](#elasticsearch-common-metrics) -- [`ElasticsearchIndexSample`](#elasticsearch-index-metrics) +- [`ElasticsearchClusterSample`](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-advanced-config#cluster-metrics) +- [`ElasticsearchNodeSample`](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-advanced-config#node-metrics) +- [`ElasticsearchCommonSample`](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-advanced-config#common-metrics) +- [`ElasticsearchIndexSample`](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch/elasticsearch-advanced-config#index-metrics) You can [query this data](/docs/using-new-relic/data/understand-data/query-new-relic-data) for troubleshooting purposes or to create custom charts and dashboards. From 34063e5a033af38d38c8af21a38c21c43eacfe41 Mon Sep 17 00:00:00 2001 From: tanaka_733 Date: Fri, 11 Mar 2022 23:48:24 +0900 Subject: [PATCH 12/25] Update postgresql-integration.mdx Fix typo. Previous fix is not enough. --- .../postgresql/postgresql-integration.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-integration.mdx b/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-integration.mdx index ccfe1606404..f26dd148ea2 100644 --- a/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-integration.mdx +++ b/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-integration.mdx @@ -59,7 +59,7 @@ There are several ways to configure the integration, depending on how you instal - If enabled via ![Kubernetes](./images/kubernetes-k8.png 'Kubernetes')Kubernetes, see [Monitor services running on Kubernetes](/docs/monitor-service-running-kubernetes). - If enabled via ![ECS](./images/ecs.png 'ECS')Amazon ECS, see [Monitor services running on ECS](/docs/integrations/host-integrations/host-integrations-list/monitor-services-running-amazon-ecs). -- If installed on-host, edit the config in the integration's YAML config file, `elasticsearch-config.yml`. An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference. The configuration file has common settings applicable to all integrations, such as `interval`, `timeout`, `inventory_source`. To read all about these common settings, refer to our [Configuration Format](/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-newer-configuration-format/#configuration-basics) document. +- If installed on-host, edit the config in the integration's YAML config file, `postgresql-config.yml`. An integration's YAML-format configuration is where you can place required login credentials and configure how data is collected. Which options you change depend on your setup and preference. The configuration file has common settings applicable to all integrations, such as `interval`, `timeout`, `inventory_source`. To read all about these common settings, refer to our [Configuration Format](/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-newer-configuration-format/#configuration-basics) document. If you are still using our legacy configuration or definition files, check the [standard configuration format](/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-standard-configuration-format/). @@ -68,7 +68,7 @@ Specific settings related to PostgreSQL are defined using the `env` section of t ## Install and activate the integration [#install] -To install the Elasticsearch integration, follow the instructions for your environment. +To install the PostgreSQL integration, follow the instructions for your environment. ### Linux installation [#linux] From 832f73ef410589a5643d1a7cadd06e51716cb6db Mon Sep 17 00:00:00 2001 From: JimHagan Date: Fri, 11 Mar 2022 09:57:37 -0500 Subject: [PATCH 13/25] Clarified caution statement around POA vs. parent accounts --- .../operational-efficiency/dg-baselining.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx index c37c8f4f0a8..ce1b56dba0c 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx @@ -529,7 +529,7 @@ Detail tabs include: - Pixie - `PixieBytes` -If you are using a POA account, be aware that it will not be included facets by `consumingAccountName`. If you install the dashboard into a parent account the consumption value of that parent account is the `sum` of data ingested directly to the parent account and data sent to any sub-accounts. +If you are using a POA account, be aware that the POA account itself will not be included in facets by `consumingAccountName`. Thats because the POA account receives no actual telemetry. If you install the dashboard into a normal parent account the consumption value of that parent account is the `sum` of data ingested directly to the parent account and data sent to any sub-accounts. For example, when faceting on monthly ingest you may see something like: From 3fd9788b5e00ca29a6739a5f0aafcb233109a7d7 Mon Sep 17 00:00:00 2001 From: zuluecho9 Date: Fri, 11 Mar 2022 08:08:16 -0800 Subject: [PATCH 14/25] fix(user migration): small edit --- .../original-users-roles/user-migration.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx index b5908155bdd..a79f9e2bf9c 100644 --- a/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx +++ b/src/content/docs/accounts/original-accounts-billing/original-users-roles/user-migration.mdx @@ -154,7 +154,7 @@ The new user records created for your users have the same login credentials: the Important tips: * Ensure the new users' email addresses match their original user record email addresses, including matching exact case. We use email addresses as the key value to match users and, in a later step, to transition their user-associated assets. -* Once you complete this step and create new user records, we highly recommend completing the migration process without delay. If you don't complete the steps to migrate assets and delete the original user record, a user may have two user records associated with the same login (see [login screenshot from Step 1](#page1)) or else may be missing assets they expect to see (like dashboards). +* Once you complete this step and create new user records, we highly recommend completing the remainder of the migration process fairly quickly. If you don't complete the steps to migrate assets and delete the original user record, a user may have two user records associated with the same login (see [login screenshot from Step 1](#page1)) or else may be missing assets they expect to see (like dashboards). ## Step 6: Access settings [#access-settings] From af5b2d57b2c04a910d18ad9a25cb26f36e4b2f96 Mon Sep 17 00:00:00 2001 From: Anna Urbiztondo Date: Fri, 11 Mar 2022 17:31:29 +0100 Subject: [PATCH 15/25] chore(Alerts DD): Adding link to reset --- src/data-dictionary/events/NrAiSignal/event.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/data-dictionary/events/NrAiSignal/event.md b/src/data-dictionary/events/NrAiSignal/event.md index cd0e3b2b65e..36101b00310 100644 --- a/src/data-dictionary/events/NrAiSignal/event.md +++ b/src/data-dictionary/events/NrAiSignal/event.md @@ -8,4 +8,4 @@ events: The type of event captured in this data point. * Value: A single value was evaluated (most common). * Expiration: A signal loss is detected. -* Reset: Indicates a detail of the condition was changed, and the signal has been reset. \ No newline at end of file +* Reset: Indicates a detail of the condition was changed, and [the signal has been reset](https://docs.newrelic.com/docs/alerts-applied-intelligence/new-relic-alerts/alert-conditions/create-nrql-alert-conditions/#evaluation-resets). \ No newline at end of file From 40d0104cb5b74319afaf21da753c266854db31c6 Mon Sep 17 00:00:00 2001 From: JimHagan Date: Fri, 11 Mar 2022 11:43:30 -0500 Subject: [PATCH 16/25] Fixed broken link --- .../operational-efficiency/dg-optimizing.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx index beacd6d0d5a..b39a413bfec 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx @@ -1072,7 +1072,7 @@ View [this doc](https://docs.newrelic.com/docs/infrastructure/infrastructure-int ### *Streaming* or *Pushed* Metrics -More and more cloud integrations are offering the option of having the data pushed via a [streaming service]](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/). This has proven to cut down on latency drastically. One issue some users have observed is that it's not as easy to control volume since sampling rate cannot be configured. +More and more cloud integrations are offering the option of having the data pushed via a [streaming service](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/). This has proven to cut down on latency drastically. One issue some users have observed is that it's not as easy to control volume since sampling rate cannot be configured. *Drop rules* will be well covered in the next section. They are the primary way of filtering out streaming metrics that are too high volume. However there are some things that can be done on the cloud provider side to limit the stream volume somewhat. From 209df1d899a6a9765ae620d6a13ca308d574841e Mon Sep 17 00:00:00 2001 From: JimHagan Date: Fri, 11 Mar 2022 11:48:27 -0500 Subject: [PATCH 17/25] Fix broken link --- .../operational-efficiency/dg-optimizing.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx index b39a413bfec..0a4b82ec191 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx @@ -170,7 +170,7 @@ Their estate includes: - Configure load balancer logging to only monitor 5xx response codes (monolith) -- Cust sample rate on ProcessSample, StorageSample, and NetworkSample to 60s for hosts running the monolith +- Custom sample rate on ProcessSample, StorageSample, and NetworkSample to 60s for hosts running the monolith - Disable MSSQL monitoring since currently the new architecture does not use it. - Disable Distributed Tracing for the monolith as it's far less useful then it will be for the microservice architecture. @@ -535,7 +535,7 @@ network_interface_filters: ### Custom attributes -[Custom attributes](Custom attributes are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine.) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine. +[Custom attributes](https://docs.newrelic.com/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine.) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine. Example of custom attributes from `newrelic.yml` From ebb31677be85f54e73574911cf59eea818b28d1e Mon Sep 17 00:00:00 2001 From: Riley Starnes Date: Fri, 11 Mar 2022 08:50:50 -0800 Subject: [PATCH 18/25] Fix spelling and grammer issues I ran the guide through google doc's spelling and grammar checker to fix various issues. --- .../operational-efficiency/dg-optimizing.mdx | 83 ++++++++++--------- 1 file changed, 42 insertions(+), 41 deletions(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx index beacd6d0d5a..99d320de1b6 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx @@ -1,5 +1,5 @@ --- -title: Optimzing Data Ingest +title: Optimizing Data Ingest tags: - Observability maturity - Operational efficiency @@ -11,7 +11,7 @@ tags: - Value drivers - Bill and Usage Data - Data ingest cost -metaDescription: Data governance is a practice of ensuring optimal value for telemetry data collected by an organization particulary a complex organization with numerous business units and working groups. +metaDescription: Data governance is a practice of ensuring optimal value for telemetry data collected by an organization particularly a complex organization with numerous business units and working groups. redirects: - /docs/telemetry-data-platform/manage-data/data-governance - /docs/data-governance @@ -21,11 +21,11 @@ redirects: ## Overview -Optimizing is a very overloaded term. However in our context it means maximimizing observability value of delivered data. In the process of optimizing ingest there can be really reductions in overall spend, but the goal starts from the standpoint of prioritizing different telemetry data and if needed to reduce the volume of certain data to meet certain budgetary guidelines. In this section we will focus on: +Optimizing is a very overloaded term. However in our context it means maximizing observability value of delivered data. In the process of optimizing ingest there can be real reductions in overall spend, but the goal starts from the standpoint of prioritizing different telemetry data and if needed to reduce the volume of certain data to meet certain budgetary guidelines. In this section we will focus on: ## Aligning to Value -Consumption based observability platforms have revolutionized the way in which organizations can achieve maximum visibility in to their cloud-based and on-prem platforms and services. Like with a migration to cloud compute consumption based SaaS platforms give business fine grained control over what they pay. This enables them to align costs more closesly to identifid business value. However this paradigm does require some additional thought into the value of collected telemtry in order to exploit the benefits. +Consumption based observability platforms have revolutionized the way in which organizations can achieve maximum visibility into their cloud-based and on-prem platforms and services. Like with a migration to cloud compute consumption based SaaS platforms give business fine grained control over what they pay. This enables them to align costs more closely to identified business value. However this paradigm does require some additional thought into the value of collected telemetry in order to exploit the benefits. When executed properly this approach will allow us to move beyond purely reactive efforts to reduce and troubleshoot unexplained increases or low value telemetry. Adopting a value driven approach to optimization is a clear indicator of *observability maturity*. @@ -50,7 +50,7 @@ Objectives include: - Meeting an Internal SLA - Meeting an External SLA -- Supporting Feature Innovation (A/B Perfomance & Adoption Testing) +- Supporting Feature Innovation (A/B Performance & Adoption Testing) - Monitor Customer Experience - Hold Vendors and Internal Service Providers to Their SLA - Business Process Health Monitoring @@ -94,12 +94,12 @@ Their estate includes: -- Omit debug logs (knowning they can be turned on if there is an issue) (saves 5%) +- Omit debug logs (knowing they can be turned on if there is an issue) (saves 5%) - Omit several K8s state merics which are not required to display the Kubernetes Cluster Explore (saves 10%) - Drop some custom Browser events they were collecting back when they were doing a lot of A/B testing of new features (saves 10%) -After executing those changes the team is 5% below their budget and has freed up some space to do a NPM pilot. Their manager is satisfied they are not losing any signifanct `Uptime and Reliability` observability. +After executing those changes the team is 5% below their budget and has freed up some space to do a NPM pilot. Their manager is satisfied they are not losing any significant `Uptime and Reliability` observability. - 5% under their original budget @@ -113,7 +113,7 @@ After executing those changes the team is 5% below their budget and has freed up title="Example 2: Focus on Customer Experience" > -A team responsible for a new user facing platform with an emphasis on Mobile and Browser is running 50% over budget. They will need to right size their ingest, but they are adament about not sacrificing any `Customer Experience` observability. +A team responsible for a new user facing platform with an emphasis on Mobile and Browser is running 50% over budget. They will need to right size their ingest, but they are adamant about not sacrificing any `Customer Experience` observability. ![Value Drivers Uptime](images/oma-oe-dg-value-driver-customer.png) @@ -139,7 +139,7 @@ Their estate includes: - 5% over their original budget -- Sufficent to get them trough peak season +- Sufficient to get them trough peak season - No loss of customer experience observability @@ -151,7 +151,7 @@ After executing the changes they are now just 5% over their original budget, but title="Example 3: Focus on Prioritize Innovation" > -A team is in the process of refactoring a large Python monolith to four microservices. The monolith shares much infrastructure with the new architecture including a customer database, and a cache layer. They are 70% over budget and have two months to go before they can officially decommision the monolith. +A team is in the process of refactoring a large Python monolith to four microservices. The monolith shares much infrastructure with the new architecture including a customer database, and a cache layer. They are 70% over budget and have two months to go before they can officially decommission the monolith. ![Value Drivers Uptime](images/oma-oe-dg-value-driver-innovation.png) @@ -213,9 +213,9 @@ We will break them up into three different approaches: The volume of data generated by the APM agent will be determined by several factors: -- The amount of organic traffic generated by the application (i.e, all things being equal an app producin one million transactions per day will generate more data then one processing one thousand) +- The amount of organic traffic generated by the application (i.e, all things being equal an app producing one million transactions per day will generate more data then one processing one thousand) - Some characteristics of the underlying transaction data itself (length and complexity of URLs) -- Whether the application is reporting datase queries +- Whether the application is reporting database queries - Whether the application has a lot (or any) custom attributes added to each transaction - The error volume for the application - Whether the application is reporting distributed Tracing @@ -226,7 +226,7 @@ While we can assume that all calls to an application are needed to support the b ### Custom attributes -Any [custom attributes](https://docs.newrelic.com/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) added using a call to an APM API's [addCustomParameter](https://developer.newrelic.com/collect-data/custom-attributes/) will add an additional attribute to the Transaction payload. These are often useful, but as application logic and priorities changes the data can be come less valuable or even obsolute. +Any [custom attributes](https://docs.newrelic.com/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) added using a call to an APM API's [addCustomParameter](https://developer.newrelic.com/collect-data/custom-attributes/) will add an additional attribute to the Transaction payload. These are often useful, but as application logic and priorities changes the data can become less valuable or even obsolete. It is also true that Our Java agent will capture the following request.headers by default: @@ -302,7 +302,7 @@ distributed_tracing: { } ``` -Data volume will also vary based on whether you are using `infite tracing` or not. +Data volume will also vary based on whether you are using `infinite tracing` or not. Standard distributed tracing for APM agents (above) captures up to 10% of your traces, but if you want us to analyze all your data and find the most relevant traces, you can set up Infinite Tracing. This alternative to standard distributed tracing is available for all APM language agents except C SDK. @@ -437,7 +437,7 @@ See this document for [more details](https://docs.newrelic.com/docs/mobile-monit New Relic's [Infrastructure agent configuration file](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings/) contains a couple of powerful ways to control ingest volume. The most important is using sampling rates. There are several distinct sampling rate configurations that can be used: -The other is through custom process samle filters. +The other is through custom process sample filters. ### Sampling Rates @@ -454,7 +454,7 @@ There are a number of sampling rates that can be configured in infrastructure, b ### Process samples -Process samples can be the single most high volume source of data from the infrastructure agent. This is becuase it will send information about any running process on a host. They are disabled by default, however they can be enabled as follows: +Process samples can be the single most high volume source of data from the infrastructure agent. This is because it will send information about any running process on a host. They are disabled by default, however they can be enabled as follows: ``` enable_process_metrics: true @@ -466,7 +466,7 @@ By default, processes using low memory are excluded from being sampled. For more You can control how much data is sent to New Relic by configuring `include_matching_metrics`, which allows you to restrict the transmission of metric data based on the values of metric [attributes](https://docs.newrelic.com/docs/query-your-data/nrql-new-relic-query-language/nrql-query-tutorials/query-infrastructure-dimensional-metrics-nrql#naming-conventions). -You include metric data by defining literal or partial values for any of the attributes of the metric. For example, you can choose to send the host.process.cpuPercent of all processes whose process.name match the ^java regular expression. +You include metric data by defining literal or partial values for any of the attributes of the metric. For example, you can choose to send the host.process.cpuPercent of all processes whose process.name matches the ^java regular expression. In this example, we include process metrics using executable files and names: @@ -522,7 +522,7 @@ Default network interface filters for Windows: - Network interfaces that start with `Loop`, `isatap`, or `Local` -To override defaults inclue your own filter in the config file: +To override defaults include your own filter in the config file: ``` network_interface_filters: @@ -561,11 +561,11 @@ They are powerful and useful but if the data is not well organized or has become - Logs generated per cluster -It's not surprising that a complex and decentralized system like Kubernetes has the potentially to generate a lot of telemetry fast. There are a few good approaches to managing data ingest in Kubernetes. These will be very straighforward if you are using observability as code in your K8s deployments. +It's not surprising that a complex and decentralized system like Kubernetes has the potential to generate a lot of telemetry fast. There are a few good approaches to managing data ingest in Kubernetes. These will be very straightforward if you are using observability as code in your K8s deployments. ### Scrape Interval -Depending on your observability objectives you may consider adjusting the scrape interval. The default is 15s. Now the Kubernetes Cluster Explore only refreshes every 45s. If your primary use of the K8s data is to support the KCE visusalizations you may consider changing your scrape interval to 20s. That change from 15s to 20s can have a substantial impact. +Depending on your observability objectives you may consider adjusting the scrape interval. The default is 15s. Now the Kubernetes Cluster Explore only refreshes every 45s. If your primary use of the K8s data is to support the KCE visualizations you may consider changing your scrape interval to 20s. That change from 15s to 20s can have a substantial impact. ![Update Scrape Interval](images/oma-oe-dg-update-k8s-scrape-interval.png) @@ -621,7 +621,7 @@ New Relic's On Host Integrations (OHI for short) represent a diverse set of inte We'll use a few examples to demonstrate. -### [PostgreSQL Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/postgresql-monitoring-integration/#example-postgresSQL-collection-config) +### [PostgreSQL Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/postgresql-monitoring-integration/#example-postgresQL-collection-config) - Number of tables monitored @@ -678,7 +678,7 @@ The Kafka OHI configuration provides these adjustable settings that can help man - COLLECT_TOPIC_SIZE: Collect the metric Topic size. Options are `true` or `false`, defaults to `false`. - COLLECT_TOPIC_OFFSET:Collect the metric Topic offset. Options are `true` or `false`, defaults to `false`. -It should be noted that collection of topic level metrics especiallly offsets can be resource intensive to collect and can have an impact on data volume. It is very possible that a clusters ingest can increase by an order of magnitude simply by the addition of new Kafka topics to the cluster. +It should be noted that collection of topic level metrics especially offsets can be resource intensive to collect and can have an impact on data volume. It is very possible that a cluster's ingest can increase by an order of magnitude simply by the addition of new Kafka topics to the cluster. ### [MongoDB Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/mongodb-monitoring-integration) @@ -692,9 +692,9 @@ The MongoDB integration provides these adjustable settings that can help manage - interval: Default is 15s - METRICS: Set to `true` to collect only metrics - INVENTORY: Set to `true` to enable only inventory collection -- FILTERS: A JSON map of database names to an array of collection names. If empty, defaults to all databases and collections. +- FILTERS: A JSON map of database names to an array of collection names. If empty, it defaults to all databases and collections. -For any OHI you use it's important to be aware of parameters like `FILTERS` where the default is to collect metrics form all databases. This is an area where you can use your monitoring priorities to streamline collected data. +For any OHI you use it's important to be aware of parameters like `FILTERS` where the default is to collect metrics from all databases. This is an area where you can use your monitoring priorities to streamline collected data. *Example configuration with different intervals for METRIC and INVENTORY* @@ -739,7 +739,7 @@ The Elasticsearch integration provides these adjustable settings that can help m - METRICS: Set to `true` to collect only metrics - INVENTORY: Set to `true` to enable only inventory collection - COLLECT_INDICES: Signals whether to collect indices metrics or not. -- COLLECT_PRIMARIES: Signals whether to collect primaries metrics or not. +- COLLECT_PRIMARIES: Signals whether to collect primary metrics or not. - INDICES_REGEX: Filter which indices are collected. - MASTER_ONLY: Collect cluster metrics on the elected master only. @@ -787,7 +787,7 @@ The JMX integration provides these adjustable settings that can help manage data - METRICS: Set to `true` to collect only metrics - INVENTORY: Set to `true` to enable only inventory collection - METRIC_LIMIT: Number of metrics that can be collected per entity. If this limit is exceeded the entity will not be reported. A limit of 0 implies no limit. -- LOCAL_ENTITY: Collect all metrics on the local entity. Only use when monitoring localhost. +- LOCAL_ENTITY: Collect all metrics on the local entity. Only used when monitoring localhost. - COLLECTION_FILES: A comma-separated list of full file paths to the metric collection definition files. For on-host install, the default JVM metrics collection file is at /etc/newrelic-infra/integrations.d/jvm-metrics.yml. - COLLECTION_CONFIG: Configuration of the metrics collection as a JSON. @@ -809,7 +809,7 @@ COLLECTION_CONFIG={"collect":[{"domain":"Catalina","event_type":"TomcatSample"," ### Other On Host Integrations -There many other OHIs having configuration options that will help you optimize collection. +There are many other OHIs having configuration options that will help you optimize collection. Some commonly used ones are: - [NGINX](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/nginx/nginx-advanced-config) @@ -833,12 +833,12 @@ This is a good [starting point](https://docs.newrelic.com/docs/infrastructure/in - traps configured -This section focuses on New Relic's network performance monitoring which relies in the `ktranslate` agent from Kentik. This agent is quite sophisticated and it is important to fully understand the [advanced configuration docs](https://docs.newrelic.com/docs/network-performance-monitoring/advanced/advanced-config) before major optimization efforts. +This section focuses on New Relic's network performance monitoring which relies on the `ktranslate` agent from Kentik. This agent is quite sophisticated and it is important to fully understand the [advanced configuration docs](https://docs.newrelic.com/docs/network-performance-monitoring/advanced/advanced-config) before major optimization efforts. - mibs_enabled: Array of all active MIBs the ktranslate docker image will poll. This list is automatically generated during discovery if the discovery_add_mibs attribute is true. MIBs not listed here will not be polled on any device in the configuration file. You can specify a SNMP table directly in a MIB file using MIB-NAME.tableName syntax. Ex: HOST-RESOURCES-MIB.hrProcessorTable. - user_tags: key:value pair attributes to give more context to the device. Tags at this level will be applied to all devices in the configuration file. - devices: Section listing devices to be monitored for flow -- traps: configures IP and ports to be monited with SNMP traps (default is 127.0.0.1:1162) +- traps: configures IP and ports to be monitored with SNMP traps (default is 127.0.0.1:1162) - discovery: configures how endpoints can be discovered. Under this section the following parameters will do the most to increase or decrease scope: - cidrs: Array of target IP ranges in [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation). - ports: Array of target ports to scan during SNMP polling. @@ -863,7 +863,7 @@ Logs represent one of the most flexible sources of telemetry in that we are typi - New Relic Infra Agent (built-in Fluentbit) - Logstash -Forwarders generaly provide a fairly complete [routing workflow](https://docs.fluentd.org/configuration/routing-examples) that includes filtering, and transformation. +Forwarders generally provide a fairly complete [routing workflow](https://docs.fluentd.org/configuration/routing-examples) that includes filtering, and transformation. New Relic's infrastructure agent provides some very simple patterns for filtering unwanted logs. @@ -877,7 +877,7 @@ This field works in a way similar to grep -E in Unix systems. For example, for a pattern: WARN|ERROR ``` -If you have pre-written fluentd configurations for Fluentbit that do valuable filtering or parsing. It is possible to import those into our New Relic logging config. This is done by means of the `config_file` and `parsers` parameter. +If you have pre-written fluentd configurations for Fluentbit that do valuable filtering or parsing. It is possible to import those into our New Relic logging config. This is done by means of the `config_file` and `parsers` parameters. `config_file`: path to an existing Fluent Bit configuration file. Note that any overlapping source results in duplicate messages in New Relic Logs. @@ -937,7 +937,7 @@ see [nri-prometheus-latest.yaml](https://download.newrelic.com/infrastructure_ag ``` POMI will discover any Prometheus endpoint exposed at the node-level by default. This typically includes metrics coming from Kubelet and cAdvisor. -If you are running the New Relic Kubernetes Daemonset, it is import that you set require_scrape_enabled_label_for_nodes: true so that POMI does not collect duplicate metrics. +If you are running the New Relic Kubernetes Daemonset, it is important that you set require_scrape_enabled_label_for_nodes: true so that POMI does not collect duplicate metrics. The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here](https://github.com/newrelic/nri-kubernetes/blob/main/README.md). @@ -945,13 +945,13 @@ The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here] POMI will discover any Prometheus endpoint exposed at the node-level by default. This typically includes metrics coming from Kubelet and cAdvisor. -If you are running the New Relic Kubernetes Daemonset, it is import that you set require_scrape_enabled_label_for_nodes: true so that POMI does not collect duplicate metrics. +If you are running the New Relic Kubernetes Daemonset, it is important that you set require_scrape_enabled_label_for_nodes: true so that POMI does not collect duplicate metrics. The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here](https://github.com/newrelic/nri-kubernetes/blob/main/README.md). *POMI Config Parameters* ``` -# Whether k8s nodes needs to be labeled to be scraped or not. +# Whether k8s nodes need to be labeled to be scraped or not. # Defaults to false. require_scrape_enabled_label_for_nodes: false ``` @@ -1050,7 +1050,7 @@ topk(20, count by (__name__, job)({__name__=~".+"})) - Polling frequency (for polling based integration) -New Relic cloud integrations get data from cloud providers' APIs. Data is generally collected from monitoring APIs such as AWS CloudWatch, Azure Monitor, and GCP Stackdriver, and inventory metadata is collected from the specific services' APIs. More more our integrations get their data from streaming metrics that are pushed via a streaming service such as AWS Kinesis. +New Relic cloud integrations get data from cloud providers' APIs. Data is generally collected from monitoring APIs such as AWS CloudWatch, Azure Monitor, and GCP Stackdriver, and inventory metadata is collected from the specific services' APIs. Moreover, our integrations get their data from streaming metrics that are pushed via a streaming service such as AWS Kinesis. ### Polling API Based Integrations @@ -1145,7 +1145,7 @@ Drop filter rules help you accomplish several important goals: *A note of caution* -When creating drop rules, you are responsible for ensuring that the rules accurately identify and discard the data that meets the conditions that you have established. You are also responsible for monitoring the rule, as well as the data you disclose to New Relic. Always test and re-test your queries and after the drop rule was installed, make sure it works as intended. Creating a dashboard to monitor your data pre and post drop will help. +When creating drop rules, you are responsible for ensuring that the rules accurately identify and discard the data that meets the conditions that you have established. You are also responsible for monitoring the rule, as well as the data you disclose to New Relic. Always test and retest your queries and after the drop rule was installed, make sure it works as intended. Creating a dashboard to monitor your data pre and post drop will help. -In some cases we can economize on data where we have redundant coverage. For example in an environment where I have the AWS RDS integration running as well as the New Relic OHI I maybe be able to discard some overlapping metrics. +In some cases we can economize on data where we have redundant coverage. For example in an environment where I have the AWS RDS integration running as well as the New Relic OHI I may be able to discard some overlapping metrics. For quick exploration we can run a query like this: ``` @@ -1243,7 +1243,7 @@ FROM Metric select count(*) where metricName like 'aws.rds%' facet metricName li That will show us all metricNames matching the pattern. -We see from the results there is a high volume of metrics of the pattern `aws.rds.cpu%`. Let's drop those since we have other insrtrumentation for those. +We see from the results there is a high volume of metrics of the pattern `aws.rds.cpu%`. Let's drop those since we have other instrumentation for those. - Create the relevant query: 'FROM Metric select * where metricName like 'aws.rds.cpu%' facet metricName limit max since 1 day ago' - Make sure it finds the process samples we want to drop @@ -1276,11 +1276,12 @@ mutation { title="Other Events and Metrics" > -The preceding examples should show you all you need to know to use these techniques on any other event or metric in NRDB. If you can query it you can drop it. Reach out to New Relic if you questions about the precise way to structure a query for a drop rule. +The preceding examples should show you all you need to know to use these techniques on any other event or metric in NRDB. If you can query it you can drop it. Reach out to New Relic if you have questions about the precise way to structure a query for a drop rule. - \ No newline at end of file + + From af406479d03496e28a9eece41de584ae5f463682 Mon Sep 17 00:00:00 2001 From: Riley Starnes Date: Fri, 11 Mar 2022 08:58:33 -0800 Subject: [PATCH 19/25] Fix Spelling and Grammar I ran the guide through google doc's spelling and grammar checker to fix various issues. --- .../operational-efficiency/dg-coe.mdx | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-coe.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-coe.mdx index 7be30a1a5b5..429baa6cb2e 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-coe.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-coe.mdx @@ -22,7 +22,7 @@ Designate a few explicit roles and practices in order to ensure a level of conte ## Stakeholders and participants -Regardless of how your organization's teams are structured it is necessary to identify some individuals who will participate in the data governance process. Selection of the team can be ad hoc but it should include representation from a broad enough cross section of teams so that when priorities and decisions must be made you will have the right stakeholders. The team should have one individual who can be considered the overall observability manager. This may be the person who manages the New Relic account or an overriding team leader responsible for the sytems and infrastructure monitored by New Relic. +Regardless of how your organization's teams are structured it is necessary to identify some individuals who will participate in the data governance process. Selection of the team can be ad hoc but it should include representation from a broad enough cross section of teams so that when priorities and decisions must be made you will have the right stakeholders. The team should have one individual who can be considered the overall observability manager. This may be the person who manages the New Relic account or an overriding team leader responsible for the systems and infrastructure monitored by New Relic. ### Observability manager @@ -31,15 +31,15 @@ This is the go-to person to help resolve conflicts and to communicate with senio ### The governance team The *Observability manager* functions as the lead for this team. The members of this team bring in practical technical knowledge of the systems and services that are monitored in New Relic. They may be peers of the observability manager so collaboration is key. -If you have a pre-existing structure such as an Observability Centre of Excellence (OCoE) your governance team can be comprised primarily from the *OCoE Core Team*. +If you have a pre-existing structure such as an Observability Center of Excellence (OCoE) your governance team can be comprised primarily from the *OCoE Core Team*. -The primary responsibilites of a OCoE team generally are: +The primary responsibilities of a OCoE team generally are: - Maintain the relationship with New Relic. - Govern accounts and users. - Onboard new teams and individuals. -- Maintain the observability knowledge base. -- Promote of collaboration and sharing among teams. +- Maintain an observability knowledge base. +- Promote collaboration and sharing among teams. Incorporating data governance adds the following responsibilities: @@ -47,9 +47,9 @@ Incorporating data governance adds the following responsibilities: - Monitor data ingest baselines and respond to anomalies. - Draft and approve plans for data optimization/reduction as needed. - Participate in scheduled check-ins where baseline data is analyzed and compared to ingest targets. -- Make modifications to to ingest targets as needed. +- Make modifications to ingest targets as needed. -## Timelines and checkins +## Timelines and check ins Schedule data ingest governance meetings through the year to keep everyone up to date on data ingest volumes. Doing so makes data ingest governance predictable and easy to manage. @@ -74,6 +74,5 @@ Occasionally a system refactor or organizational change could lead to an unexpec - A new software deployment increases log volume by 3x. - A team enables a handful of new cloud integrations that unexpectedly increases metrics ingest by 200%. -- An acquisition of a new company leads to increase in overall telemetry volume. +- An acquisition of a new company leads to an increase in overall telemetry volume. - Peak business season activity combined with some pre-peak refactors results in a higher than expected custom events volume. - From c6c6dce786ce91b6b41986cddc2ac7013ec11005 Mon Sep 17 00:00:00 2001 From: Barb Bryant Date: Fri, 11 Mar 2022 09:09:10 -0800 Subject: [PATCH 20/25] style(Table format): hero edits --- .../postgresql/postgresql-advanced-config.mdx | 490 +++++++++--------- 1 file changed, 251 insertions(+), 239 deletions(-) diff --git a/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx b/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx index 1d777aac1a2..17674013302 100644 --- a/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx +++ b/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx @@ -9,260 +9,272 @@ metaDescription: New Relic's PostgreSQL monitoring integration's advanced config This integration is open source software. That means you can [browse its source code](https://github.com/newrelic/nri-nginx "Link opens in a new window.") and send improvements, or create your own fork and build it. -### PostgreSQL Instance Settings [#instance-settings] +### PostgreSQL instance settings [#instance-settings] -The PostgreSQL integration collects both Metrics(M) and Inventory(I) information. Check the **Applies To** column below to find which settings can be used for each specific collection: +The PostgreSQL integration collects both Metrics (**M**) and Inventory (**I**) information. The **Applies To** column in the following table indicates which settings can be used for each specific collection: - + - - - - - + + + + - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - - -{' '} - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SettingSetting DescriptionDefaultApplies To -
Applies To
- **HOSTNAME** - - The hostname for the PostgreSQL connection. - - localhost - - M/I -
**PORT**The port where PostgreSQL is running.5432M/I
**USERNAME**The user name for the PostgreSQL connection. **Required.**N/AM/I
**PASSWORD**The password for the PostgreSQL connection. **Required.**N/AM/I
- **COLLECTION_LIST** - - JSON array, a JSON object, or the string literal `ALL` that specifies the entities to be collected. The PostgreSQL user can only collect table and index metrics from tables it has `SELECT` permissions for. - - **Required except for PgBouncer.** - - [Examples](#example-postgresSQL-collection-config). - - N/A - - M -
**COLLECTION_IGNORE_DATABASE_LIST** - JSON array of database names that will be ignored for metrics collection. - Typically useful for cases where `COLLECTION_LIST` is set to `ALL` and some - databases need to be ignored. - '[]'M
**PGBOUNCER**Collect `pgbouncer` metrics.falseM
**ENABLE_SSL** - Determines if SSL is enabled. If `true`, `ssl_cert_location` and - `ssl_key_location` are required. - falseM/I
**TRUST_SERVER_CERTIFICATE** - If `true`, the server certificate is not verified for SSL. If `false`, the - server certificate identified in `ssl_root_cert_location` is verified. - falseM/I
**SSL_ROOT_CERT_LOCATION** - Absolute path to PEM-encoded root certificate file. Required if - `trust_server_certificate` is `false`. - N/AM/I
**SSL_CERT_LOCATION** - Absolute path to PEM-encoded client certificate file. Required if - `enable_ssl` is `true`. - N/AM/I
**SSL_KEY_LOCATION** - Absolute path to PEM-encoded client key file. Required if `enable_ssl` is - `true`. - N/AM/I
**TIMEOUT**maximum wait for connection, in seconds. Set to `0` for no timeout.10M/I
**DATABASE**The PostgreSQL database to connect to.postgresM/I
**CUSTOM_METRICS_QUERY** - A SQL query that required `columns metric_name`, `metric_type`, and - `metric_value.metric_type` can be `gauge`, `rate`, `delta`, or `attribute`. - Additional columns collected with the query are added to the metric set as - attributes. - N/AM
**CUSTOM_METRICS_CONFIG** - A path to a YAML file with a list of custom queries, along with their metric - type, database, and sample name overrides. See example for details. - N/AM
**COLLECT_DB_LOCK_METRICS** - Enable collecting database lock metrics, which can be performance intensive. - falseM
**COLLECT_BLOAT_METRICS**Enable tablespace bloat metrics, which can be performance intensive.trueM
**METRICS**Set to `true` to enable Metrics only collection.false
**INVENTORY**Set to `true` to enable Inventory only collection.false
+ `HOSTNAME` + + The hostname for the PostgreSQL connection. Default is localhost. + + M/I +
+ `PORT` + + The port where PostgreSQL is running. Default is 5432. + + M/I +
+ `USERNAME` + + The user name for the PostgreSQL connection. **Required.** + + M/I +
+ `PASSWORD` + + The password for the PostgreSQL connection. **Required.** + + M/I +
+ `COLLECTION_LIST` + + JSON array, a JSON object, or the string literal `ALL` that specifies the entities to be collected. The PostgreSQL user can only collect table and index metrics from tables it has `SELECT` permissions for. + + **Required except for `PgBouncer`.** + + [Examples](/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-integration#examples) + + M +
+ `COLLECTION_IGNORE_DATABASE_LIST` + + JSON array of database names that will be ignored for metrics collection. Typically useful for cases where `COLLECTION_LIST` is set to `ALL` and some databases need to be ignored. Default is `[]`. + + M +
+ `PGBOUNCER` + + Collect `pgbouncer` metrics. Default is `false`. + + M +
+ `ENABLE_SSL` + + Determines if SSL is enabled. If `true`, `ssl_cert_location` and `ssl_key_location` are required. Default is `false`. + + M/I +
+ `TRUST_SERVER_CERTIFICATE` + + If `true`, the server certificate is not verified for SSL. If `false`, the server certificate identified in `ssl_root_cert_location` is verified. Default is `false`. + + M/I +
+ `SSL_ROOT_CERT_LOCATION` + + Absolute path to PEM-encoded root certificate file. Required if `trust_server_certificate` is `false`. + + M/I +
+ `SSL_CERT_LOCATION` + + Absolute path to PEM-encoded client certificate file. Required if `enable_ssl` is `true`. + + M/I +
+ `SSL_KEY_LOCATION` + + Absolute path to PEM-encoded client key file. Required if `enable_ssl` is `true`. + + M/I +
+ `TIMEOUT` + + Maximum wait for connection, in seconds. Set to `0` for no timeout. Default is 10. + + M/I +
+ `DATABASE` + + The PostgreSQL database to connect to. Default is `postgres`. + + M/I +
+ `CUSTOM_METRICS_QUERY` + + The SQL query that requires `columns metric_name`, `metric_type`, and `metric_value.metric_type` can be `gauge`, `rate`, `delta`, or `attribute`. Additional columns collected with the query are added to the metric set as attributes. + + M +
+ `CUSTOM_METRICS_CONFIG` + + A path to a YAML file with a list of custom queries, along with their metric type, database, and sample name overrides. See the [example]() for details. + + M +
+ `COLLECT_DB_LOCK_METRICS` + + Enable collecting database lock metrics, which can be performance intensive. Default is `false`. + + M +
+ `COLLECT_BLOAT_METRICS` + + Enable tablespace bloat metrics, which can be performance intensive. Default is `true`. + + M +
+ `METRICS` + + Set to `true` to enable Metrics only collection. Default is `false`. +
+ `INVENTORY` + + Set to `true` to enable Inventory only collection. Default is `false`. +
The values for these settings can be defined in several ways: - Adding the value directly in the config file. This is the most common way. -- Replacing the values from environment variables using the `{{}}` notation. This requires infrastructure agent v1.14.0+. Read more [here](/docs/infrastructure/install-infrastructure-agent/configuration/configure-infrastructure-agent/#passthrough). -- Using Secrets management. Use this to protect sensible information such as passwords to be exposed in plain text on the configuration file. For more information, see [Secrets management](/docs/integrations/host-integrations/installation/secrets-management). +- Replacing the values from environment variables using the `{{}}` notation. This requires infrastructure agent v1.14.0 or higher. For more information, see our infrastructure monitoring documentation about [passthrough configuration]](/docs/infrastructure/install-infrastructure-agent/configuration/configure-infrastructure-agent/#passthrough). +- Using secrets management for sensitive data (such as passwords) without having to write them as plain text into the configuration files. For more information, see our documentation about [secrets management](/docs/infrastructure/host-integrations/installation/secrets-management/). + +### Labels and custom attributes [#labels] -### Labels and custom Attributes [#labels] +Environment variables can be used to control config settings, such as your license key, and are then passed through to the infrastructure agent. For instructions on how to use this feature, see [Configure the infrastructure agent](/docs/infrastructure/install-infrastructure-agent/configuration/configure-infrastructure-agent). -Environment variables can be used to control config settings, such as your license key, and are then passed through to the infrastructure agent. For instructions on how to use this feature, see [Configure the infrastructure agent](/docs/infrastructure/new-relic-infrastructure/configuration/configure-infrastructure-agent#passthrough). You can further decorate your metrics using labels. Labels allow you to add key/value pairs attributes to your metrics which you can then use to query, filter or group your metrics on. Our default sample config file includes examples of labels but, as they are not mandatory, you can remove, modify or add new ones of your choice: @@ -1166,4 +1178,4 @@ Here are some troubleshooting tips for the PostgreSQL integration: - If you have connection problems, make sure you can connect to the cluster from the same box with `psql`. - If you have problems collecting `PgBouncer` metrics, make sure you are connected to the instance through `PgBouncer`. Default port is `6432`. -- If you get the error message `Error creating list of entities to collect: pq: unsupported startup parameter: extra_float_digits`, set `ignore_startup_parameters = extra_float_digits` in the PgBouncer config file. +- If you get the error message `Error creating list of entities to collect: pq: unsupported startup parameter: extra_float_digits`, set `ignore_startup_parameters = extra_float_digits` in the `PgBouncer` config file. From 907084b2df7a3138e150e1917c8caf4dc70b095f Mon Sep 17 00:00:00 2001 From: Riley Starnes Date: Fri, 11 Mar 2022 09:58:11 -0800 Subject: [PATCH 21/25] Fix Spelling and Grammar I ran the guide through google doc's spelling and grammar checker to fix various issues. --- .../operational-efficiency/dg-baselining.mdx | 46 +++++++++---------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx index c37c8f4f0a8..b8c8c88861e 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining.mdx @@ -46,7 +46,7 @@ Understand exactly which groups within the org are contributing which types of d All billable telemetry is tracked in the `NrConsumption` and `NrMTDConsumption` event types. This guide focuses on how to query the `NrConsumption` event type, which provides more granular, real time data than `NrMTDConsumption`. The `NrConsumption` attribute `usageMetric` denotes the telemetry type. -Using `NrConsumption` you can ask questions like "How much browser data has each sub-account ingested in the last 30 days? How have the ingest changed since the previous 30 days?" +Using `NrConsumption` you can ask questions like "How much browser data has each sub-account ingested in the last 30 days? How has the ingest changed since the previous 30 days?" ``` FROM NrConsumption SELECT sum(GigabytesIngested) WHERE usageMetric = 'BrowserEventsBytes' SINCE 30 days AGO COMPARE WITH 30 days AGO FACET consumingAccountName @@ -59,7 +59,7 @@ Banking Platform, 75 GB, +2.9% Marketing Platform, 40 GB, -1.3% ``` -Below is a break down of the different `usageMetric` types, the constituent events (event types where the data is stored), and the type of agent or mechanism responsible for creating the data ingest. +Below is a breakdown of the different `usageMetric` types, the constituent events (event types where the data is stored), and the type of agent or mechanism responsible for creating the data ingest. *Billable Telemetry Breakdown* @@ -210,11 +210,11 @@ Below is a break down of the different `usageMetric` types, the constituent even Under New Relic's consumption model, telemetry data and users both contribute to your consumption. This guide is focused on maximizing the value of telemetry data. The mention of users in this section is to help you understand different options for balancing users and data. -There are three general types of usage plans. You usage plan may affect how you set ingest targets for your organization. +There are three general types of usage plans. Your usage plan may affect how you set ingest targets for your organization. ### *APoF* -If you have an `Annual Pool of Funds` (APoF) agreement you will likely have a monthly target budget for data ingest. For example you may have set a target of 5TB per day and 100 FSO users. In this type of plan data and users can be "traded off" but it's best to discuss this with other stakeholders in your organization to ensure you are getting the right mix for your observability goals. Although some customers will plan for variability in their consumption during the year let's assume for now our montly consumption budget is your APoF / 12. +If you have an `Annual Pool of Funds` (APoF) agreement you will likely have a monthly target budget for data ingest. For example you may have set a target of 5TB per day and 100 FSO users. In this type of plan data and users can be "traded off" but it's best to discuss this with other stakeholders in your organization to ensure you are getting the right mix for your observability goals. Although some customers will plan for variability in their consumption during the year, let's assume for now our monthly consumption budget is your APoF / 12. If you know the number of FSO and Core users you need you can use this formula: @@ -225,7 +225,7 @@ If you know the number of FSO and Core users you need you can use this formula: ### *Pay As You Go* -In a pay as you go plan you will not have a pre-determined yearly commit however, you will likely have an understood limit to your montly spend. In this model you would take the following to determine your target ingest: +In a pay as you go plan you will not have a predetermined yearly commit however, you will likely have an understood limit to your monthly spend. In this model you would take the following to determine your target ingest: `(monthly_target_spend - (num_fso_users*per_fso_cost) - (num_core_users*per_core_cost))/0.25` @@ -250,7 +250,7 @@ You can find more information on this topic in [Usage plans](/docs/licenses/lice > -Use the `rate` operator when you need to take a sample of data pulled from a certain time period and produce a given rate. For example take a daily sample of data and compute a 30 day rate based on that. +Use the `rate` operator when you need to take a sample of data pulled from a certain time period and produce a given rate. For example, take a daily sample of data and compute a 30 day rate based on that. *Compute rate based on a given sample of data* @@ -285,7 +285,7 @@ When it's important to constrain an ingest calculation to specific calendar mont SELECT sum(GigabytesIngested) AS 'Daily Ingest Rate (GB)' FROM NrConsumption WHERE productLine = 'DataPlatform' FACET monthOf(timestamp) LIMIT MAX SINCE 56 weeks AGO ``` -The resulting table shows fairly high variability. Note that things were fairly `hot` in `august` and September. Some of that is our organization seasonality but also was related to some increasing the breadth of our telemetry coverage. +The resulting table shows fairly high variability. Note that things were fairly `hot` in August and September. Some of that is our organization seasonality but also was related to some increasing the breadth of our telemetry coverage. |MONTH OF TIMESTAMP|GB INGESTED| |---|---| @@ -308,7 +308,7 @@ The resulting table shows fairly high variability. Note that things were fairly > -When you want to evaluate the amount of change in ingest volume or rate between one time period in another. This is important to know if you ingest is creeping up unexpetedly. +When you want to evaluate the amount of change in ingest volume or rate between one time period in another. This is important to know if your ingest is creeping up unexpectedly. *Simple Change Analysis* @@ -330,7 +330,7 @@ SELECT sum(GigabytesIngested) FROM NrConsumption WHERE productLine = 'DataPlatfo When you need to remove the effects of regular variability of ingest to see the broader pattern. -Telemetry is inherently noisy. Real world phenomena happen in spurts leaving with many random peaks and troughs in the signal. This is good in a way since it lets us view the full comlexity of a phenomenon. However when we are seeking to see trends we can be distracted by detail. NRQL provides a powerful to smoothing out any time series by cominging each data point with slightly older points This let's us focus on the overall temporal trend rather than one extreme `increase` or `decrease`. +Telemetry is inherently noisy. Real world phenomena happen in spurts leaving many random peaks and troughs in the signal. This is good in a way since it lets us view the full complexity of a phenomenon. However when we are seeking to see trends we can be distracted by detail. NRQL provides a powerful way to smoothing out any time series by combining each data point with slightly older points This let's us focus on the overall temporal trend rather than one extreme `increase` or `decrease`. Note the jaggedness of the raw timeseries for 1 day ingest rate: @@ -361,7 +361,7 @@ FROM NrConsumption SELECT rate(sum(GigabytesIngested), 1 day) WHERE productLine Use to estimate the statistical rate of change over a given time period. The rate of change is calculated using a linear least-squares regression to approximate the derivative -NRQL provides us some tools to assess the rate of change. This is useful since as we see in the previous example we had a very large increase over the past several months in browser metrics. This rate of change analysis uses the `derivative` operator and it gives us some confidence that the main growth happened back in early September. It seems as though our growth rate based on the 7 day derivative is somewhat negative so we may have reached a new plateau at the moment in BrowserEventsBytes ingest. +NRQL provides us with some tools to assess the rate of change. This is useful since as we see in the previous example we had a very large increase over the past several months in browser metrics. This rate of change analysis uses the `derivative` operator and it gives us some confidence that the main growth happened back in early September. It seems as though our growth rate based on the 7 day derivative is somewhat negative so we may have reached a new plateau at the moment in BrowserEventsBytes ingest. ``` SELECT derivative(sum(GigabytesIngested) , 7 day) FROM NrConsumption WHERE productLine = 'DataPlatform' and usageMetric = 'BrowserEventsBytes' LIMIT MAX SINCE 3 MONTHS AGO UNTIL THIS MONTH TIMESERIES 1 MONTH slide by 3 days COMPARE WITH 1 WEEK AGO @@ -369,7 +369,7 @@ SELECT derivative(sum(GigabytesIngested) , 7 day) FROM NrConsumption WHERE produ ![Rate of Growth Time Series](images/oma-oe-dg-rate-of-growth-time-series.png) -In this scenario the uptick was so blatant a simple time series of the rate will suffice. However the benefit of the deriviative is it can be more sensitive at assessing the relative quanty of growth and give us a sense of when it first started. This can be useful if we in the early stages of a major uptick. +In this scenario the uptick was so blatant a simple time series of the rate will suffice. However the benefit of the derivative is it can be more sensitive at assessing the relative quantity of growth and give us a sense of when it first started. This can be useful if we are in the early stages of a major uptick. Here is the simple plot of the SUM @@ -397,7 +397,7 @@ Use this operator whenever you need to estimate the ingest data footprint for a title="Ingest by application (APM|Browser|Mobile)" > -Run these queies in each sub-account or in a dashboard with account-specific charts. The queries estimate a 30 day rate based on 1 week of collection. +Run these queries in each sub-account or in a dashboard with account-specific charts. The queries estimate a 30 day rate based on 1 week of collection. *Estimate 30 day rate* @@ -464,7 +464,7 @@ FROM K8sClusterSample, K8sContainerSample,K8sDaemonsetSample, K8sDeploymentSampl id="ingest-by-process-sample" title="Process Samples" > - ProcessSample can be quite a high volume event. In this example we'll compute the 30 day ingest per commandline. + ProcessSample can be quite a high volume event. In this example we'll compute the 30 day ingest per command line. *Estimate 30 day rate by command name* @@ -487,7 +487,7 @@ FROM K8sClusterSample, K8sContainerSample,K8sDaemonsetSample, K8sDeploymentSampl [Add ingest target indicators to your dashboard](#add-target-indicators) [Generate a tabular 30 day ingest report](#generate-report) [Customize your report](#customize-report) -[Detect ingest anomalies](#detect-anamolies) +[Detect ingest anomalies](#detect-anomalies) [Install the entity breakdown dashboard (Optional)](#install-entity-breakdown-dashboard) [Install the cloud integration dashboard (optional)](#install-cloud-integration-dashboard) @@ -497,7 +497,7 @@ FROM K8sClusterSample, K8sContainerSample,K8sDaemonsetSample, K8sDeploymentSampl 3. Click `Install this quickstart` in the upper right portion of your browser window. 4. Select your top level master account or POA account in the account drop down. 5. Click `Done` since there is no agent to install. -6. When the quickstart is done installing open the `Data Governance Baseline` dashboard. +6. When the quickstart is done installing, open the `Data Governance Baseline` dashboard. That will bring you to the newly installed dashboard. @@ -507,7 +507,7 @@ The main overview tab shows a variety of charts including some powerful time ser ![Org Wide View](images/oma-dg-org-wide-dashboard-dark.png) -The second tab provides baseline report by sub-account and usage metric. +The second tab provides a baseline report by sub-account and usage metric. ![Org Wide Tabular](images/oma-dg-org-wide-dashboard-tabular-dark.png) @@ -529,7 +529,7 @@ Detail tabs include: - Pixie - `PixieBytes` -If you are using a POA account, be aware that it will not be included facets by `consumingAccountName`. If you install the dashboard into a parent account the consumption value of that parent account is the `sum` of data ingested directly to the parent account and data sent to any sub-accounts. +If you are using a POA account, be aware that it will not be included in facets by `consumingAccountName`. If you install the dashboard into a parent account the consumption value of that parent account is the `sum` of data ingested directly to the parent account and data sent to any sub-accounts. For example, when faceting on monthly ingest you may see something like: @@ -585,7 +585,7 @@ Below is an example of a sheet we imported into Google Sheets. The screenshot shows the table sorted by 30 day ingest total. -Feel free to adjust your timeline and some of the details as needed. For example, we chose to extract a *90 day derivative* to have some sense of change over the past few months. You could easily alter the time period of the derivative to suite your objectives. +Feel free to adjust your timeline and some of the details as needed. For example, we chose to extract a *90 day derivative* to have some sense of change over the past few months. You could easily alter the time period of the derivative to suit your objectives. ### Customize your report [#customize-report] @@ -594,7 +594,7 @@ Add useful columns to your report in order to facilitate other phases of data go - Notes: Note any growth anomalies and any relevant explanations for them. Indicate any major expected growth if foreseen. - Technical Contact: Name of the manager of a given sub-account or someone related to a specific telemetry type. -### Detect ingest anomalies [#detect-anamolies] +### Detect ingest anomalies [#detect-anomalies] #### Alert on ingest anomalies @@ -626,7 +626,7 @@ In a previous section you installed the ingest `baseline` dashboard that uses Nr 1. Go to the [same quickstart](https://onenr.io/0PoR8zpDYQG) you used for the baseline dashboard. 2. Click `Install this quickstart` in the upper right section of your browser window. -3. Don't install this instance of the dashboard into a POA account. Instead, install it into any account that contains APM, Browser, Mobile appliactions or K8s clusters [import dashboard function](/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/#dashboards-import). You can install this dashboard into multiple accounts. You can install it into a top-level parent account and modify the dashboard so you have account-specific charts all in one dashboard. +3. Don't install this instance of the dashboard into a POA account. Instead, install it into any account that contains APM, Browser, Mobile applications or K8s clusters [import dashboard function](/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards/#dashboards-import). You can install this dashboard into multiple accounts. You can install it into a top-level parent account and modify the dashboard so you have account-specific charts all in one dashboard. 4. Click `Done` since there is no agent to install. 6. When the quickstart is done installing open the `Data Governance Entity Breakdowns` dashboard. @@ -634,13 +634,13 @@ In a previous section you installed the ingest `baseline` dashboard that uses Nr You can refer back to [this section](/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining#understand-nr-consumption-metrics) to see exactly which event types are used in these breakdowns. -*NOTE* these queries consume more resources since they are not working from a pre-aggregated data source like NrConsumption. It may be necessary to adjust the time frames and take advantage of additional `where` clauses and the `limit` clause to make them performant in some of your environments. +*NOTE* These queries consume more resources since they are not working from a pre-aggregated data source like NrConsumption. It may be necessary to adjust the time frames and take advantage of additional `where` clauses and the `limit` clause to make them performant in some of your environments. ### Install the cloud integration dashboard (optional) [#install-cloud-integration-dashboard] -Cloud Integrations can often be a signficant source of data ingest growth. Without good visualizations it can be very difficult to pinpoint where the growth is coming from. This is partly because these integrations are so easy to configure and they are not part of an organizations normal CI/CD pipeline. They may also not be part of a formal configuration management system. +Cloud Integrations can often be a significant source of data ingest growth. Without good visualizations it can be very difficult to pinpoint where the growth is coming from. This is partly because these integrations are so easy to configure and they are not part of an organization's normal CI/CD pipeline. They may also not be part of a formal configuration management system. Fortunately this powerful set of dashboards can be installed [directly from New Relic I/O](https://onenr.io/0EPwJJO9Ow7). -Individual dashboards installed by this pakage include: +Individual dashboards installed by this package include: - AWS Integrations - Azure Integrations From 674c0943e4073c8374801d986fe9296eede2ed4d Mon Sep 17 00:00:00 2001 From: zuluecho9 Date: Fri, 11 Mar 2022 10:14:03 -0800 Subject: [PATCH 22/25] fix(opt data ingest): fix many style guide issues --- .../operational-efficiency/dg-optimizing.mdx | 329 +++++++++--------- 1 file changed, 161 insertions(+), 168 deletions(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx index 99d320de1b6..5a46aae8444 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx @@ -1,5 +1,5 @@ --- -title: Optimizing Data Ingest +title: Optimizing data ingest tags: - Observability maturity - Operational efficiency @@ -24,7 +24,7 @@ redirects: Optimizing is a very overloaded term. However in our context it means maximizing observability value of delivered data. In the process of optimizing ingest there can be real reductions in overall spend, but the goal starts from the standpoint of prioritizing different telemetry data and if needed to reduce the volume of certain data to meet certain budgetary guidelines. In this section we will focus on: -## Aligning to Value +## Aligning to value Consumption based observability platforms have revolutionized the way in which organizations can achieve maximum visibility into their cloud-based and on-prem platforms and services. Like with a migration to cloud compute consumption based SaaS platforms give business fine grained control over what they pay. This enables them to align costs more closely to identified business value. However this paradigm does require some additional thought into the value of collected telemetry in order to exploit the benefits. When executed properly this approach will allow us to move beyond purely reactive efforts to reduce and troubleshoot unexplained increases or low value telemetry. Adopting a value driven approach to optimization is a clear indicator of *observability maturity*. @@ -33,14 +33,12 @@ When executed properly this approach will allow us to move beyond purely reactiv This may be a good time to familiarize yourself with the [observability maturity principles](/docs/new-relic-solutions/observability-maturity/introduction/). This will make the concept of observability `value` drivers more concrete from an observability standpoint. - ## Process - One of the most important parts of the data governance framework is to align collected telemetry with *observability value drivers*. What we mean here is to ensure that we understand what the primary observability objective is when we configure new telemetry. @@ -48,20 +46,20 @@ When we introduce new telemetry we will want to understand what it delivers to o Objectives include: -- Meeting an Internal SLA -- Meeting an External SLA -- Supporting Feature Innovation (A/B Performance & Adoption Testing) -- Monitor Customer Experience -- Hold Vendors and Internal Service Providers to Their SLA -- Business Process Health Monitoring -- Other Compliance Requirements +- Meeting an internal SLA +- Meeting an external SLA +- Supporting feature innovation (A/B performance & adoption testing) +- Monitor customer experience +- Hold vendors and internal service providers to their SLA +- Business process health monitoring +- Other compliance requirements Alignment to these objectives are what allow us to make flexible and intuitive decisions about prioritizing one set of data over another and helping guide teams know where to start when instrumenting new platforms and services. In this section we'll make two core assumptions @@ -74,10 +72,10 @@ Use the following examples to help you visualize how you would assess your own t -An account is ingesting about 20% more than they had budgeted for. They have been asked by a manager to find some way to reduce consumption. Their most important value driver is `Uptime and Reliability` +An account is ingesting about 20% more than they had budgeted for. They have been asked by a manager to find some way to reduce consumption. Their most important value driver is `Uptime and reliability` ![Value Drivers Uptime](images/oma-oe-dg-value-driver-uptime.png) @@ -85,35 +83,35 @@ An account is ingesting about 20% more than they had budgeted for. They have be Their estate includes: -- APM (Dev, Staging, Prod) -- Distributed Tracing +- APM (dev, staging, prod) +- Distributed tracing - Browser -- Infrastructure Monitoring 100 hosts -- K8s Monitoring (Dev, Staging, Prod) -- Logs (Dev, Staging, Prod - Including Debug) +- Infrastructure monitoring 100 hosts +- K8s monitoring (dev, staging, prod) +- Logs (dev, staging, prod - including debug) - + - Omit debug logs (knowing they can be turned on if there is an issue) (saves 5%) -- Omit several K8s state merics which are not required to display the Kubernetes Cluster Explore (saves 10%) -- Drop some custom Browser events they were collecting back when they were doing a lot of A/B testing of new features (saves 10%) +- Omit several K8s state metrics which are not required to display the Kubernetes cluster explorer (saves 10%) +- Drop some custom browser monitoring events they were collecting back when they were doing a lot of A/B testing of new features (saves 10%) -After executing those changes the team is 5% below their budget and has freed up some space to do a NPM pilot. Their manager is satisfied they are not losing any significant `Uptime and Reliability` observability. +After executing those changes the team is 5% below their budget and has freed up some space to do a NPM pilot. Their manager is satisfied they are not losing any significant `Uptime and reliability` observability. - + - 5% under their original budget -- headroom created for an NPM pilot which servers Uptime and Reliability Objectives -- Minimal loss of Uptime and Reliability observability +- headroom created for an NPM pilot which servers uptime and reliability objectives +- Minimal loss of uptime and reliability observability -A team responsible for a new user facing platform with an emphasis on Mobile and Browser is running 50% over budget. They will need to right size their ingest, but they are adamant about not sacrificing any `Customer Experience` observability. +A team responsible for a new user facing platform with an emphasis on mobile monitoring and browser monitoring is running 50% over budget. They will need to right size their ingest, but they are adamant about not sacrificing any `Customer Experience` observability. ![Value Drivers Uptime](images/oma-oe-dg-value-driver-customer.png) @@ -124,20 +122,20 @@ Their estate includes: - Mobile - Browser - APM -- Distributed Tracing -- Infrastructure on 30 Hosts Including Process Samples +- Distributed tracing +- Infrastructure on 30 hosts, including process samples - Serverless monitoring for some backend asynchronous processes - Logs from their serverless functions - Various cloud integrations - + - Omit the serverless logs (they are basically redundant to what they get from their Lambda integration) - Decrease the process sample rate on their hosts to every one minute - Drop process sample data in DEV environments - Turn off EC2 integration which is highly redundant with other infrastructure monitoring provided by the New Relic infra agent. - + - 5% over their original budget - Sufficient to get them trough peak season - No loss of customer experience observability @@ -148,7 +146,7 @@ After executing the changes they are now just 5% over their original budget, but A team is in the process of refactoring a large Python monolith to four microservices. The monolith shares much infrastructure with the new architecture including a customer database, and a cache layer. They are 70% over budget and have two months to go before they can officially decommission the monolith. @@ -159,20 +157,20 @@ A team is in the process of refactoring a large Python monolith to four microser Their estate includes: -- K8s Monitoring (microservices) -- New Relic Host Monitoring (monolith) +- K8s monitoring (microservices) +- New Relic host monitoring (monolith) - APM (microservices and host monitoring) -- Distributed Tracing (microservices and host monitoring) +- Distributed tracing (microservices and host monitoring) - Postgresql (shared) - Redis(shared) - MSSQL (future DB for the microservice architecture) -- Load Balancer Logging (microservices and host monitoring) +- Load balancer logging (microservices and host monitoring) - + - Configure load balancer logging to only monitor 5xx response codes (monolith) - Cust sample rate on ProcessSample, StorageSample, and NetworkSample to 60s for hosts running the monolith - Disable MSSQL monitoring since currently the new architecture does not use it. -- Disable Distributed Tracing for the monolith as it's far less useful then it will be for the microservice architecture. +- Disable distributed tracing for the monolith as it's far less useful then it will be for the microservice architecture. @@ -187,7 +185,7 @@ Their estate includes: At this stage we assume you've given some good thought to all of the kinds of telemetry in your account(s) and how they relate to your value drivers. This section will provide detailed technical instructions and examples on how to reduce a variety of telemetry types. @@ -196,16 +194,16 @@ We will break them up into three different approaches: - + - Monitored transactions - Error activity - Custom events @@ -218,9 +216,9 @@ The volume of data generated by the APM agent will be determined by several fact - Whether the application is reporting database queries - Whether the application has a lot (or any) custom attributes added to each transaction - The error volume for the application -- Whether the application is reporting distributed Tracing +- Whether the application is reporting distributed tracing -### Managing Volume +### Managing volume While we can assume that all calls to an application are needed to support the business, it is possible that we could be more thrifty in our overall architecture. In an extreme case we may have a user profile microservice that is called every 10 seconds by its clients. This helps reduce latency if some user information is updated by other clients. However, one lever we have is reducing the frequency of calls to this service to for example every minute. @@ -242,13 +240,13 @@ Developers will also use `addCustomParameter` to capture additional (potentially For an example of the rich configuration that is available in relation to APM see our [Java agent documentation](https://docs.newrelic.com/docs/apm/agents/java-agent/attributes/java-agent-attributes/#requestparams) -### Error Events +### Error events It is possible to determine how errors will be handled by APM. This can reduce the volume of data in some cases. For example there may be a high volume, but harmless error that cannot be removed at the present time. We have the ability to `collect`, `ignore`, or `mark as expected`. This document covers [all of the details](https://docs.newrelic.com/docs/apm/agents/manage-apm-agents/agent-data/manage-errors-apm-collect-ignore-or-mark-expected/). -### Database Queries +### Database queries One highly variable aspect of APM instances is the number of database calls and what configurations we have set. We have a fair amount of control on how verbose database query monitoring is. These queries will show up in the transaction traces page. @@ -260,7 +258,7 @@ Common database query setting changes include: More details can be found [here](https://docs.newrelic.com/docs/apm/transactions/transaction-traces/transaction-traces-database-queries-page/#settings). -### Transaction Traces +### Transaction traces - Number of connected services @@ -282,8 +280,7 @@ Transaction trace settings available using server-side configuration will differ - Thread profiler - Cross application tracing - -### Distributed Tracing +### Distributed tracing Distributed tracing configuration will have some language-specific differences. @@ -316,16 +313,16 @@ The main parameters that could drive a small increase in monthly ingest are: - -- Page Loads -- Ajax Calls + +- Page loads +- Ajax calls - Error activity -In [agent version 1211](https://docs.newrelic.com/docs/release-notes/new-relic-browser-release-notes/browser-agent-release-notes/) or higher, all network requests made by a page are recorded as AjaxRequest events. You can use the deny list configuration options in the Application settings page to filter which requests record events. Regardless of this filter, all network requests are captured as metrics and available in the AJAX page. +In [agent version 1211](https://docs.newrelic.com/docs/release-notes/new-relic-browser-release-notes/browser-agent-release-notes/) or higher, all network requests made by a page are recorded as AjaxRequest events. You can use the deny list configuration options in the **Application settings** page to filter which requests record events. Regardless of this filter, all network requests are captured as metrics and available in the AJAX page. ### Using the deny list @@ -347,15 +344,13 @@ To update the deny list of URLs your application will filter from creating event Go to [one.newrelic.com](http://one.newrelic.com/), and click Browser. Select an app. On the left navigation, click *App settings*. -Under *Ajax Request Deny List*, add the filters you would like to apply to your app. +Under *Ajax request deny list*, add the filters you would like to apply to your app. Select *Save application settings* to update the agent configuration. Redeploy the browser agent (either restarting the associated APM agent or updating the copy/paste browser installation). ### Validating - - ``` FROM AjaxRequest SELECT * WHERE requestUrl LIKE `%example.com%` ``` @@ -364,10 +359,10 @@ FROM AjaxRequest SELECT * WHERE requestUrl LIKE `%example.com%` - + - Monthly active users - Crash events - Number of events per user @@ -375,7 +370,7 @@ FROM AjaxRequest SELECT * WHERE requestUrl LIKE `%example.com%` ### Android -[Feature Flags](https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-android/android-sdk-api/android-agent-configuration-feature-flags/) +[Feature flags](https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-android/android-sdk-api/android-agent-configuration-feature-flags/) All settings, including the call to invoke the agent, are called in the onCreate method of the MainActivity class. To change settings, call the setting in one of two ways (if the setting supports it): @@ -384,10 +379,10 @@ NewRelic.disableFeature(FeatureFlag.DefaultInteractions); NewRelic.enableFeature(FeatureFlag.CrashReporting); NewRelic.withApplicationToken().start(this.getApplication()); ``` -[Analytics Settings](Enable or disable collection of event data. These events are reported to Insights and used in the Crash analysis page.) -Enable or disable collection of event data. These events are reported to Insights and used in the Crash analysis page. +[Analytics Settings](Enable or disable collection of event data. These events are reported to the New Relic database and used in the **Crash analysis** page.) +Enable or disable collection of event data. These events are reported to the New Relic database and used in the **Crash analysis** page. -It's also possible to configure [agent logging](Enable or disable collection of event data. These events are reported to Insights and used in the Crash analysis page.) to be more or less verbose. +It's also possible to configure [agent logging](Enable or disable collection of event data. These events are reported to New Relic and used in the **Crash analysis** page.) to be more or less verbose. ### iOS @@ -395,13 +390,13 @@ Like with Android New Relic's iOS configuration allows to enable and disable fea The following feature flags can be configured: -#### Crash and Error Reporting +#### Crash and error reporting - NRFeatureFlag_CrashReporting - NRFeatureFlag_HandleExceptionEvents - NRFeatureFlag_CrashReporting -#### Distributed Tracing +#### Distributed tracing - NRFeatureFlag_DistributedTracing @@ -411,7 +406,7 @@ The following feature flags can be configured: - NRFeatureFlag_InteractionTracing - NRFeatureFlag_SwiftInteractionTracing -#### Network Feature Flags +#### Network feature flags - NRFeatureFlag_ExperimentalNetworkInstrumentation - NRFeatureFlag_NSURLSessionInstrumentation @@ -419,27 +414,28 @@ The following feature flags can be configured: - NRFeatureFlag_RequestErrorEvents - NRFeatureFlag_HttpResponseBodyCapture -See this document for [more details](https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-ios/ios-sdk-api/ios-agent-configuration-feature-flags/). +See this document for [more details](/docs/mobile-monitoring/new-relic-mobile-ios/ios-sdk-api/ios-agent-configuration-feature-flags/). - + - Hosts and containers monitored -- Sampling Rates for Core Events -- Process Sample Configurations +- Sampling rates for core events +- Process sample configurations - Custom attributes -- Number and type of OHIs installed +- Number and type of on-host integrations installed - Log forwarding configuration -New Relic's [Infrastructure agent configuration file](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings/) contains a couple of powerful ways to control ingest volume. The most important is using sampling rates. There are several distinct sampling rate configurations that can be used: +New Relic's [infrastructure agent configuration file](/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings/) contains a couple of powerful ways to control ingest volume. The most important is using sampling rates. There are several distinct sampling rate configurations that can be used: + The other is through custom process sample filters. -### Sampling Rates +### Sampling rates There are a number of sampling rates that can be configured in infrastructure, but these are the most commonly used. @@ -492,9 +488,9 @@ env: - regex "\\System32\\svchost" ``` -### Network Interface filter +### Network interface filter - + - Number of network interfaces monitored @@ -552,10 +548,10 @@ They are powerful and useful but if the data is not well organized or has become - + - Number of `pods` and `containers` monitored - Frequency and number of kube state metrics collected - Logs generated per cluster @@ -563,25 +559,25 @@ They are powerful and useful but if the data is not well organized or has become It's not surprising that a complex and decentralized system like Kubernetes has the potential to generate a lot of telemetry fast. There are a few good approaches to managing data ingest in Kubernetes. These will be very straightforward if you are using observability as code in your K8s deployments. -### Scrape Interval +### Scrape interval -Depending on your observability objectives you may consider adjusting the scrape interval. The default is 15s. Now the Kubernetes Cluster Explore only refreshes every 45s. If your primary use of the K8s data is to support the KCE visualizations you may consider changing your scrape interval to 20s. That change from 15s to 20s can have a substantial impact. +Depending on your observability objectives you may consider adjusting the scrape interval. The default is 15s. Now the Kubernetes cluster explorer only refreshes every 45s. If your primary use of the K8s data is to support the KCE visualizations you may consider changing your scrape interval to 20s. That change from 15s to 20s can have a substantial impact. ![Update Scrape Interval](images/oma-oe-dg-update-k8s-scrape-interval.png) -### Kube State Metrics +### Kube state metrics -The Kubernetes Cluster Explorer requires only the following kube state metrics (KSM): +The Kubernetes cluster explorer requires only the following kube state metrics (KSM): ``` -Container Data -Cluster Data -Node Data -Pod Data -Volume Data -API Server Data* -Controller Manager Data* -ETCD Data* -Scheduler Data* +Container data +Cluster data +Node data +Pod data +Volume data +API server data* +Controller manager data* +ETCD data* +Scheduler data* *Not collected in a managed Kubernetes environment (EKS, GKE, AKS, etc.) @@ -591,16 +587,16 @@ Scheduler Data* You may consider disabling some of the following: ``` -DaemonSet Data -Deployment Data -Endpoint Data -Namespace Data -ReplicaSet Data** -Service Data -StatefulSet Data +DaemonSet data +Deployment data +Endpoint data +Namespace data +ReplicaSet data** +Service data +StatefulSet data ``` -*Updating the Manifest* +*Updating the manifest* ![Kube State Metrics - Manifest](images/oma-oe-dg-update-k8s-kube-state-metrics.png) *Updating the Helm chart* @@ -610,25 +606,25 @@ StatefulSet Data -New Relic's On Host Integrations (OHI for short) represent a diverse set of integrations for third party services such as Postgresql, MySQL, Kafka, RabbitMQ etc. It's not possible to provide every optimization technique in the scope of this document, but we can provide these generally applicable techniques: +New Relic's on-host integrations (OHI for short) represent a diverse set of integrations for third party services such as Postgresql, MySQL, Kafka, RabbitMQ etc. It's not possible to provide every optimization technique in the scope of this document, but we can provide these generally applicable techniques: - Manage sampling rate - Manage those parts of the config that can increase or decrease breadth of collection - Manage those parts of the config that allow for custom queries -- Manage the Infra Agents custom attributes since they will be applied to all OHI data. +- Manage the infrastructure agents' custom attributes because they'll be applied to all on-host integration data. We'll use a few examples to demonstrate. -### [PostgreSQL Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/postgresql-monitoring-integration/#example-postgresQL-collection-config) +### [PostgreSQL integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/postgresql-monitoring-integration/#example-postgresQL-collection-config) - + - Number of tables monitored - Number of indices monitored -The PostgreSQL OHI configuration provides these adjustable settings that can help manage data volume: +The PostgreSQL on-host integration configuration provides these adjustable settings that can help manage data volume: - interval: Default is 15s - COLLECTION_LIST: list of tables to monitor (use ALL to monitor ALL) @@ -639,7 +635,7 @@ The PostgreSQL OHI configuration provides these adjustable settings that can hel - INVENTORY: Set to `true` to enable only inventory collection - CUSTOM_METRICS_CONFIG: Config file containing custom collection queries -*Sample Config* +*Sample config* ``` integrations: @@ -661,14 +657,14 @@ integrations: inventory_source: config/postgresql ``` -### [Kafka Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/kafka-monitoring-integration/) +### [Kafka integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/kafka-monitoring-integration/) - + - Number of brokers in cluster - Number of topics in cluster -The Kafka OHI configuration provides these adjustable settings that can help manage data volume: +The Kafka on-host integration configuration provides these adjustable settings that can help manage data volume: - interval: Default is 15s - TOPIC_MODE: Determines how many topics we collect. Options are `all`, `none`, `list`, or `regex`. @@ -683,7 +679,7 @@ It should be noted that collection of topic level metrics especially offsets can ### [MongoDB Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/mongodb-monitoring-integration) - + - Number of databases monitored @@ -694,7 +690,7 @@ The MongoDB integration provides these adjustable settings that can help manage - INVENTORY: Set to `true` to enable only inventory collection - FILTERS: A JSON map of database names to an array of collection names. If empty, it defaults to all databases and collections. -For any OHI you use it's important to be aware of parameters like `FILTERS` where the default is to collect metrics from all databases. This is an area where you can use your monitoring priorities to streamline collected data. +For any on-host integration you use it's important to be aware of parameters like `FILTERS` where the default is to collect metrics from all databases. This is an area where you can use your monitoring priorities to streamline collected data. *Example configuration with different intervals for METRIC and INVENTORY* @@ -726,9 +722,9 @@ integrations: inventory_source: config/mongodb ``` -### [Elasticsearch Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch-monitoring-integration) +### [Elasticsearch integration](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch-monitoring-integration) - + - Number of nodes in cluster - Number of indices in cluster @@ -773,9 +769,9 @@ integrations: inventory_source: config/elasticsearch ``` -### [JMX Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/jmx-monitoring-integration) +### [JMX integration](/docs/infrastructure/host-integrations/host-integrations-list/jmx-monitoring-integration) - + - Metrics listed in COLLECTION_CONFIGS @@ -807,10 +803,9 @@ Omitting any one entry from that config such as `NonHeapMemoryUsage.Init` will h COLLECTION_CONFIG={"collect":[{"domain":"Catalina","event_type":"TomcatSample","beans":[{"query":"type=UtilityExecutor","attributes":["completedTaskCount"]}]}]} ``` -### Other On Host Integrations +### Other on-host integrations -There are many other OHIs having configuration options that will help you optimize collection. -Some commonly used ones are: +There are many other on-host integrations with configuration options that will help you optimize collection. Some commonly used ones are: - [NGINX](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/nginx/nginx-advanced-config) - [MySQL](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/mySQL/mysql-integration) @@ -824,9 +819,9 @@ This is a good [starting point](https://docs.newrelic.com/docs/infrastructure/in - + - Monitored devices driven by: - hard configured devices - CIDR scope in discovery section @@ -848,10 +843,10 @@ This section focuses on New Relic's network performance monitoring which relies - + - Logs forwarded - Average size of forward log records @@ -860,7 +855,7 @@ Logs represent one of the most flexible sources of telemetry in that we are typi - Fluentd - Fluentbit -- New Relic Infra Agent (built-in Fluentbit) +- New Relic infrastructure agent (built-in Fluentbit) - Logstash Forwarders generally provide a fairly complete [routing workflow](https://docs.fluentd.org/configuration/routing-examples) that includes filtering, and transformation. @@ -887,25 +882,25 @@ If you have pre-written fluentd configurations for Fluentbit that do valuable fi Learning how to inject attributes (or tags) into your logs in your data pipeline and to perform transformations can help with downstream feature dropping using New Relic drop rules. By augmenting your logs with metadata about the source, we can make centralized and easily reversible decisions about what to drop on the backend. Some fairly obvious attributes that should be present on your logs in one form or another are: - Team -- Environment (Dev/Stage/Prod) +- Environment (dev/stage/prod) - Application -- Data Center -- Log Level +- Data center +- Log level Below are some detailed routing and filtering resources: - [Common filter and routing patterns in Fluentd](https://docs.fluentd.org/configuration/routing-examples) - [Fluentbit data pipeline](https://docs.fluentbit.io/manual/concepts/data-pipeline) -- [Forwarding logs with New Relic Infrastructure agent](https://docs.newrelic.com/docs/logs/forward-logs/forward-your-logs-using-infrastructure-agent/) +- [Forwarding logs with New Relic infrastructure agent](https://docs.newrelic.com/docs/logs/forward-logs/forward-your-logs-using-infrastructure-agent/) - + - Number of metrics exported from apps - Number of metrics transferred via remote write or POMI @@ -914,20 +909,20 @@ New Relic provides two primary options for sending Prometheus metrics to NRDB. ### Option 1: [Prometheus Remote Write](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) -Prometheus Server scrape config options are [fully documented here](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config). These scrape configs determine which metrics are collected by Prometheus Server and by configuring the [remote_write](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) parameter, the collected metrics can be written to NRDB via the New Relic Metrics API. +Prometheus server scrape config options are [fully documented here](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config). These scrape configs determine which metrics are collected by the Prometheus server and by configuring the [remote_write](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) parameter, the collected metrics can be written to NRDB via the New Relic Metric API. -### Option 2: [Prometheus OpenMetrics Integration (POMI)](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-openmetrics/install-update-or-uninstall-your-prometheus-openmetrics-integration) +### Option 2: [Prometheus OpenMetrics integration (POMI)](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-openmetrics/install-update-or-uninstall-your-prometheus-openmetrics-integration) -POMI is a standalone integration that scrapes metrics from both dynamically discovered and static Prometheus endpoints. POMI then sends this data to NRDB via the New Relic Metrics API. This integration is ideal for customers not currently running Prometheus Server. +POMI is a standalone integration that scrapes metrics from both dynamically discovered and static Prometheus endpoints. POMI then sends this data to NRDB via the New Relic Metric API. This integration is ideal for customers not currently running Prometheus Server. -#### POMI: Scrape Label +#### POMI: scrape label POMI will discover any Prometheus endpoint containing the label or annotation “prometheus.io/scrape=true” by default. Depending upon what is deployed in the cluster, this can be a large number of endpoints and thus, a large number of metrics ingested. It is suggested that the scrape_enabled_label parameter be modified to something custom (e.g. “newrelic/scrape”) and that the Prometheus endpoints be selectively labeled or annotated when data ingest is of utmost concern. see [nri-prometheus-latest.yaml](https://download.newrelic.com/infrastructure_agent/integrations/kubernetes/nri-prometheus-latest.yaml) for the latest reference config. -*POMI Config Parameter* +*POMI config parameter* ``` # Label used to identify scrapable targets. @@ -941,14 +936,14 @@ If you are running the New Relic Kubernetes Daemonset, it is important that you The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here](https://github.com/newrelic/nri-kubernetes/blob/main/README.md). -#### POMI: Scrape Label for Nodes +#### POMI: scrape label for nodes POMI will discover any Prometheus endpoint exposed at the node-level by default. This typically includes metrics coming from Kubelet and cAdvisor. If you are running the New Relic Kubernetes Daemonset, it is important that you set require_scrape_enabled_label_for_nodes: true so that POMI does not collect duplicate metrics. The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here](https://github.com/newrelic/nri-kubernetes/blob/main/README.md). -*POMI Config Parameters* +*POMI config parameters* ``` # Whether k8s nodes need to be labeled to be scraped or not. @@ -958,13 +953,13 @@ The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here] #### POMI: Co-existing with *nri-kubernetes* -New Relic’s [Kubernetes Integration](https://docs.newrelic.com/docs/integrations/kubernetes-integration/get-started/introduction-kubernetes-integration) collects a [number of metrics](https://docs.newrelic.com/docs/integrations/kubernetes-integration/understand-use-data/find-use-your-kubernetes-data#metrics) OOTB, however, it does not collect every possible metric available from a Kubernetes Cluster. +New Relic’s [Kubernetes integration](/docs/integrations/kubernetes-integration/get-started/introduction-kubernetes-integration) collects a [number of metrics](/docs/integrations/kubernetes-integration/understand-use-data/find-use-your-kubernetes-data#metrics) OOTB, however, it does not collect every possible metric available from a Kubernetes Cluster. In the POMI config, you’ll see a section similar to this which will *disable* metric collection for a subset of metrics that the New Relic Kubernetes integration is already collecting from *Kube State Metrics*. It’s also very important to set `require_scrape_enabled_label_for_node: true` so that Kubelet and cAdvisor metrics are not duplicated. -*POMI Config Parameters* +*POMI config parameters* ``` transformations: @@ -985,18 +980,18 @@ transformations: ``` -#### POMI: Request/Limit Settings +#### POMI: request/limit settings -When running POMI, it’s recommended to apply the following [Resource limits](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/) for clusters generating approximately 500k DPM: +When running POMI, it’s recommended to apply the following [resource limits](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/) for clusters generating approximately 500k DPM: -- CPU Limit: 1 core (1000m) -- Memory Limit: 1Gb 1024 (1G) +- CPU limit: 1 core (1000m) +- Memory limit: 1Gb 1024 (1G) -The Resource request for CPU and Memory should be set at something reasonable so that POMI receives enough resources from the cluster. Setting this to something extremely low (e.g. cpu: 50m) can result in cluster resources being consumed by “noisy neighbors”. +The resource request for CPU and Memory should be set at something reasonable so that POMI receives enough resources from the cluster. Setting this to something extremely low (e.g. cpu: 50m) can result in cluster resources being consumed by “noisy neighbors”. An example query for determining the DPM (post-ingest) can be found [here](coming_soon). -*POMI Config Parameter* +*POMI config parameter* ``` … @@ -1017,23 +1012,23 @@ spec: … ``` -### POMI: Estimating DPM and Cardinality +### POMI: estimating DPM and cardinality -Although cardinality is not associated directly with billable per GB ingest, NR1 does maintain certain rate limits on cardinality and data points per minute. Being able to visualize cardinality and DPM from a Prometheus cluster can be very important. +Although cardinality is not associated directly with billable per GB ingest, New Relic does maintain certain rate limits on cardinality and data points per minute. Being able to visualize cardinality and DPM from a Prometheus cluster can be very important. Trial and paid accounts receive a 1M DPM and 1M cardinality limit for trial purposes, but you can request up to 15M DPM and 15M cardinality for your account. To request changes to your metric rate limits, contact your New Relic account representative. View this [doc](https://docs.newrelic.com/docs/data-apis/ingest-apis/metric-api/metric-api-limits-restricted-attributes/) for infor. -If you’re already running Prometheus Server, you can run DPM and Cardinality estimates there prior to enabling POMI or remote_write. +If you’re already running Prometheus Server, you can run DPM and cardinality estimates there prior to enabling POMI or remote_write. -*Data Points Per Minute (DPM)* +*Data points per minute (DPM)* rate(prometheus_tsdb_head_samples_appended_total[10m]) * 60 -*Top 20 Metrics (Highest Cardinality)* +*Top 20 metrics (highest cardinality)* topk(20, count by (__name__, job)({__name__=~".+"})) @@ -1042,17 +1037,17 @@ topk(20, count by (__name__, job)({__name__=~".+"})) - + - Number of metrics exported per integration - Polling frequency (for polling based integration) New Relic cloud integrations get data from cloud providers' APIs. Data is generally collected from monitoring APIs such as AWS CloudWatch, Azure Monitor, and GCP Stackdriver, and inventory metadata is collected from the specific services' APIs. Moreover, our integrations get their data from streaming metrics that are pushed via a streaming service such as AWS Kinesis. -### Polling API Based Integrations +### Polling API-based integrations If you want to report more or less data from your cloud integrations, or if you need to control the use of the cloud providers' APIs to prevent reaching rate and throttling limits in your cloud account, you can change the configuration settings to modify the amount of data they report. The two main controls are: @@ -1121,10 +1116,6 @@ The following policy allows the user to publish metrics in any namespace except } ``` - - - - @@ -1132,7 +1123,7 @@ The following policy allows the user to publish metrics in any namespace except *If you can query it you can drop it* @@ -1153,8 +1144,9 @@ When creating drop rules, you are responsible for ensuring that the rules accura title="Logs" > -All New Relic drop rules are implemented by the same backend data model and API. However New Relic Log Management provides a powerful UI that makes it very easy to create and monitor drop rules. -In our previous section on prioritizing telemetry we ran through some exercises to show ways in which we could deprecate certain data. Let's revisit this example: +All New Relic drop rules are implemented by the same backend data model and API. However New Relic log management provides a powerful UI that makes it very easy to create and monitor drop rules. + +In our previous section on prioritizing telemetry we ran through some exercises to show ways in which we could deprecate certain data. Let's revisit this example: ``` Omit debug logs (knowing they can be turned on if there is an issue) (saves 5%) @@ -1168,7 +1160,7 @@ Omit debug logs (knowing they can be turned on if there is an issue) (saves 5%) - Under *Manage Data* click *Drop filters* and create and enable a filter named 'Drop Debug Logs' - Verify the rule works -#### Method 2: [GraphQL API](https://docs.newrelic.com/docs/data-apis/manage-data/drop-data-using-nerdgraph/) +#### Method 2: [Our NerdGraph API](/docs/data-apis/manage-data/drop-data-using-nerdgraph/) - Create the relevant NRQL query: `SELECT count(*) FROM Log WHERE `level` = 'DEBUG'` - Make sure it finds the logs we want to drop @@ -1197,7 +1189,7 @@ mutation { Let's implement the recommendation: `Drop process sample data in DEV environments`. @@ -1206,7 +1198,7 @@ Let's implement the recommendation: `Drop process sample data in DEV environment - Make sure it finds the process samples we want to drop - Check for other variations on `env` such as `ENV` and `Environment` - Check for various of `DEV` such as `Dev` and `Development` -- Use the NerGraph API to execute the following statement and make sure it works: +- Use our NerdGraph API to execute the following statement and make sure it works: ``` mutation { @@ -1231,10 +1223,11 @@ mutation { -In some cases we can economize on data where we have redundant coverage. For example in an environment where I have the AWS RDS integration running as well as the New Relic OHI I may be able to discard some overlapping metrics. +In some cases we can economize on data where we have redundant coverage. For example: in an environment where you have the AWS RDS integration running as well as the New Relic on-host integration you may be able to discard some overlapping metrics. + For quick exploration we can run a query like this: ``` @@ -1247,7 +1240,7 @@ We see from the results there is a high volume of metrics of the pattern `aws.rd - Create the relevant query: 'FROM Metric select * where metricName like 'aws.rds.cpu%' facet metricName limit max since 1 day ago' - Make sure it finds the process samples we want to drop -- Use the NerGraph API to execute the following statement and make sure it works: +- Use the NerdGraph API to execute the following statement and make sure it works: ``` mutation { @@ -1273,7 +1266,7 @@ mutation { The preceding examples should show you all you need to know to use these techniques on any other event or metric in NRDB. If you can query it you can drop it. Reach out to New Relic if you have questions about the precise way to structure a query for a drop rule. From c0d66d40f9ba9a44e10da570efb62bfb96b79a35 Mon Sep 17 00:00:00 2001 From: Barb Bryant Date: Fri, 11 Mar 2022 10:30:31 -0800 Subject: [PATCH 23/25] fix(xrefs): Updated links --- .../postgresql/postgresql-advanced-config.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx b/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx index 17674013302..c79499db85a 100644 --- a/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx +++ b/src/content/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-advanced-config.mdx @@ -212,7 +212,7 @@ The PostgreSQL integration collects both Metrics (**M**) and Inventory (**I**) i `CUSTOM_METRICS_CONFIG` - A path to a YAML file with a list of custom queries, along with their metric type, database, and sample name overrides. See the [example]() for details. + A path to a YAML file with a list of custom queries, along with their metric type, database, and sample name overrides. See the [examples](/docs/infrastructure/host-integrations/host-integrations-list/postgresql/postgresql-integration#examples) for details. M @@ -268,7 +268,7 @@ The PostgreSQL integration collects both Metrics (**M**) and Inventory (**I**) i The values for these settings can be defined in several ways: - Adding the value directly in the config file. This is the most common way. -- Replacing the values from environment variables using the `{{}}` notation. This requires infrastructure agent v1.14.0 or higher. For more information, see our infrastructure monitoring documentation about [passthrough configuration]](/docs/infrastructure/install-infrastructure-agent/configuration/configure-infrastructure-agent/#passthrough). +- Replacing the values from environment variables using the `{{}}` notation. This requires infrastructure agent v1.14.0 or higher. For more information, see our infrastructure monitoring documentation about [passthrough configuration](/docs/infrastructure/install-infrastructure-agent/configuration/configure-infrastructure-agent/#passthrough). - Using secrets management for sensitive data (such as passwords) without having to write them as plain text into the configuration files. For more information, see our documentation about [secrets management](/docs/infrastructure/host-integrations/installation/secrets-management/). ### Labels and custom attributes [#labels] From 7cfa9d8cef2e11b7bacbf1f99d83bd885ffa6fea Mon Sep 17 00:00:00 2001 From: ZuluEcho9 Date: Fri, 11 Mar 2022 10:32:58 -0800 Subject: [PATCH 24/25] fix(hitrust): fixing broken links --- .../certificates-standards-regulations/hipaa.mdx | 2 +- .../certificates-standards-regulations/iso.mdx | 2 +- .../certificates-standards-regulations/soc2.mdx | 2 +- .../certificates-standards-regulations/tisax.mdx | 2 +- .../security-privacy/compliance/data-encryption.mdx | 2 +- .../compliance/fedramp-compliant-endpoints.mdx | 1 - .../compliance/hipaa-readiness-new-relic.mdx | 2 +- .../security/security-privacy/compliance/hitrust.mdx | 10 +++++----- .../regulatory-audits-new-relic-services.mdx | 4 ++-- src/nav/new-relic-security.yml | 2 +- 10 files changed, 14 insertions(+), 15 deletions(-) diff --git a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/hipaa.mdx b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/hipaa.mdx index 46fd8493744..753ae2d5ac5 100644 --- a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/hipaa.mdx +++ b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/hipaa.mdx @@ -29,7 +29,7 @@ The following applies to the New Relic One platform: - Last Updated + Last updated diff --git a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/iso.mdx b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/iso.mdx index ed9415b359c..c0f3086d276 100644 --- a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/iso.mdx +++ b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/iso.mdx @@ -25,7 +25,7 @@ The following applies to the New Relic One platform: - Last Updated + Last updated diff --git a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/soc2.mdx b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/soc2.mdx index 6fa4f2bde20..27bd8a5a046 100644 --- a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/soc2.mdx +++ b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/soc2.mdx @@ -25,7 +25,7 @@ The following applies to the New Relic One platform: - Last Updated + Last updated diff --git a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/tisax.mdx b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/tisax.mdx index 9b5c312410e..75dc2b787b8 100644 --- a/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/tisax.mdx +++ b/src/content/docs/security/security-privacy/compliance/certificates-standards-regulations/tisax.mdx @@ -28,7 +28,7 @@ The following applies to the New Relic One platform: - Last Updated + Last updated diff --git a/src/content/docs/security/security-privacy/compliance/data-encryption.mdx b/src/content/docs/security/security-privacy/compliance/data-encryption.mdx index 4c2f1bbd159..a1a29ae9d32 100644 --- a/src/content/docs/security/security-privacy/compliance/data-encryption.mdx +++ b/src/content/docs/security/security-privacy/compliance/data-encryption.mdx @@ -4,7 +4,7 @@ tags: - Security - Security and Privacy - Compliance -metaDescription: 'New Relic''s data encryption methods, including who gets it, what data is encrypted, and how it works with data at rest or in transit.' +metaDescription: "New Relic's data encryption methods, including who gets it, what data is encrypted, and how it works with data at rest or in transit." redirects: - /docs/security/new-relic-security/data-privacy/data-encryption - /docs/security/new-relic-security/compliance/data-encryption diff --git a/src/content/docs/security/security-privacy/compliance/fedramp-compliant-endpoints.mdx b/src/content/docs/security/security-privacy/compliance/fedramp-compliant-endpoints.mdx index b19e8b8dea0..665756f098f 100644 --- a/src/content/docs/security/security-privacy/compliance/fedramp-compliant-endpoints.mdx +++ b/src/content/docs/security/security-privacy/compliance/fedramp-compliant-endpoints.mdx @@ -31,7 +31,6 @@ New Relic customers must meet all of the following requirements for New Relic’ 5. **Authorized services and features**: Customer must use only FedRAMP audited and authorized New Relic services and features (see below). - ## Overview of data sources [#overview] There are multiple ways to get data into New Relic. This doc has two sections: diff --git a/src/content/docs/security/security-privacy/compliance/hipaa-readiness-new-relic.mdx b/src/content/docs/security/security-privacy/compliance/hipaa-readiness-new-relic.mdx index 958c46b76f5..dccf363fad5 100644 --- a/src/content/docs/security/security-privacy/compliance/hipaa-readiness-new-relic.mdx +++ b/src/content/docs/security/security-privacy/compliance/hipaa-readiness-new-relic.mdx @@ -16,7 +16,7 @@ Before you request New Relic’s Business Associate Agreement ("BAA"), we want t - Our application performance monitoring and data analytics solutions are intended for use cases with non-sensitive timing and metric data, which you control by your deployment and configuration choices. - Additional information is available in our BAA FAQs located in our [HIPAA BAA FAQ](https://newrelic.com/termsandconditions/hipaabaaFAQ). -## Acknowledgements and Requirements [#acknowledgements-requirements] +## Acknowledgements and requirements [#acknowledgements-requirements] ### New Relic's role [#new-relics-role] diff --git a/src/content/docs/security/security-privacy/compliance/hitrust.mdx b/src/content/docs/security/security-privacy/compliance/hitrust.mdx index e610bcd5515..46b41de006d 100644 --- a/src/content/docs/security/security-privacy/compliance/hitrust.mdx +++ b/src/content/docs/security/security-privacy/compliance/hitrust.mdx @@ -23,7 +23,7 @@ Not all [New Relic One](/docs/new-relic-one/use-new-relic-one/get-started/introd - Last Updated + Last updated @@ -39,7 +39,7 @@ Not all [New Relic One](/docs/new-relic-one/use-new-relic-one/get-started/introd - HITRUST Certificate + HITRUST certificate @@ -51,7 +51,7 @@ Not all [New Relic One](/docs/new-relic-one/use-new-relic-one/get-started/introd - New Relic One Platform + New Relic One platform @@ -65,7 +65,7 @@ HITRUST doesn't provide certification for the following services: - Last Updated + Last updated @@ -103,7 +103,7 @@ HITRUST doesn't provide certification for the following services: - Log Patterns + Log patterns diff --git a/src/content/docs/security/security-privacy/compliance/regulatory-audits-new-relic-services.mdx b/src/content/docs/security/security-privacy/compliance/regulatory-audits-new-relic-services.mdx index 884db1968e9..40243a1c818 100644 --- a/src/content/docs/security/security-privacy/compliance/regulatory-audits-new-relic-services.mdx +++ b/src/content/docs/security/security-privacy/compliance/regulatory-audits-new-relic-services.mdx @@ -1,5 +1,5 @@ --- -title: Certifications, Standards and Regulations +title: Certifications, standards, and regulations tags: - Security - Security and Privacy @@ -20,7 +20,7 @@ For detailed information, see the documentation on the specific certifications, * [FedRAMP Moderate](/docs/security/security-privacy/compliance/certificates-standards-regulations/fedramp-moderate/) * [HIPAA enabled capabilities](/docs/security/security-privacy/compliance/certificates-standards-regulations/hipaa/) -* [HITRUST compliance](/docs/security/security-privacy/compliance/certificates-standards-regulations/hitrust/) +* [HITRUST compliance](/docs/security/security-privacy/compliance/hitrust) * [ISO 27001](/docs/security/security-privacy/compliance/certificates-standards-regulations/iso/) * [SOC 2](/docs/security/security-privacy/compliance/certificates-standards-regulations/soc2/) * [TISAX](/docs/security/security-privacy/compliance/certificates-standards-regulations/tisax) diff --git a/src/nav/new-relic-security.yml b/src/nav/new-relic-security.yml index eb9e0bb9e9b..7bbbf5105c1 100644 --- a/src/nav/new-relic-security.yml +++ b/src/nav/new-relic-security.yml @@ -30,7 +30,7 @@ pages: - title: HIPAA path: /docs/security/security-privacy/compliance/certificates-standards-regulations/hipaa - title: HITRUST CSF - path: /docs/security/security-privacy/compliance/certificates-standards-regulations/hitrust + path: /docs/security/security-privacy/compliance/hitrust - title: ISO path: /docs/security/security-privacy/compliance/certificates-standards-regulations/iso - title: SOC 2 From 7e46ca23f5fadd0f7a554cc3d5b5226f4612ebd6 Mon Sep 17 00:00:00 2001 From: Bradley Camacho <42678939+bradleycamacho@users.noreply.github.com> Date: Fri, 11 Mar 2022 10:35:34 -0800 Subject: [PATCH 25/25] Make doc links relative --- .../operational-efficiency/dg-optimizing.mdx | 78 +++++++++---------- 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx index 0a4b82ec191..b83282de1fb 100644 --- a/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx +++ b/src/content/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-optimizing.mdx @@ -67,7 +67,7 @@ Alignment to these objectives are what allow us to make flexible and intuitive d In this section we'll make two core assumptions - We have the tools and techniques from the [Baselining](/docs/new-relic-solutions/observability-maturity/operational-efficiency/dg-baselining) section to have a proper accounting of where our ingset comes from. -- We have a good understanding of the [OM Value Drivers](https://docs.newrelic.com/docs/new-relic-solutions/observability-maturity/introduction/) as presented in New Relic's OMA framework. This will be crucial in applying a value and a priority to groups of telemetry +- We have a good understanding of the [OM Value Drivers](/docs/new-relic-solutions/observability-maturity/introduction/) as presented in New Relic's OMA framework. This will be crucial in applying a value and a priority to groups of telemetry Use the following examples to help you visualize how you would assess your own telemetry ingest and make the sometimes hard decisions that are needed to get within budget. Although each of these examples tries to focus on a value driver, we believe instrumentation will always be able to serve more than one value driver. This is the hardest part of data ingest governance and we don't pretend that ambiguity doesn't exist. @@ -226,7 +226,7 @@ While we can assume that all calls to an application are needed to support the b ### Custom attributes -Any [custom attributes](https://docs.newrelic.com/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) added using a call to an APM API's [addCustomParameter](https://developer.newrelic.com/collect-data/custom-attributes/) will add an additional attribute to the Transaction payload. These are often useful, but as application logic and priorities changes the data can be come less valuable or even obsolute. +Any [custom attributes](/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) added using a call to an APM API's [addCustomParameter](https://developer.newrelic.com/collect-data/custom-attributes/) will add an additional attribute to the Transaction payload. These are often useful, but as application logic and priorities changes the data can be come less valuable or even obsolute. It is also true that Our Java agent will capture the following request.headers by default: @@ -239,14 +239,14 @@ Our Java agent will capture the following request.headers by default: Developers will also use `addCustomParameter` to capture additional (potentially more verbose headers). -For an example of the rich configuration that is available in relation to APM see our [Java agent documentation](https://docs.newrelic.com/docs/apm/agents/java-agent/attributes/java-agent-attributes/#requestparams) +For an example of the rich configuration that is available in relation to APM see our [Java agent documentation](/docs/apm/agents/java-agent/attributes/java-agent-attributes/#requestparams) ### Error Events It is possible to determine how errors will be handled by APM. This can reduce the volume of data in some cases. For example there may be a high volume, but harmless error that cannot be removed at the present time. -We have the ability to `collect`, `ignore`, or `mark as expected`. This document covers [all of the details](https://docs.newrelic.com/docs/apm/agents/manage-apm-agents/agent-data/manage-errors-apm-collect-ignore-or-mark-expected/). +We have the ability to `collect`, `ignore`, or `mark as expected`. This document covers [all of the details](/docs/apm/agents/manage-apm-agents/agent-data/manage-errors-apm-collect-ignore-or-mark-expected/). ### Database Queries @@ -254,11 +254,11 @@ One highly variable aspect of APM instances is the number of database calls and Common database query setting changes include: -- [Collecting raw query data instead of obfuscated or turning off query collection](https://docs.newrelic.com/docs/apm/transactions/transaction-traces/configure-transaction-traces#record-sql) +- [Collecting raw query data instead of obfuscated or turning off query collection](/docs/apm/transactions/transaction-traces/configure-transaction-traces#record-sql) - Changing the stack trace threshold - Turning on query explain plan collection -More details can be found [here](https://docs.newrelic.com/docs/apm/transactions/transaction-traces/transaction-traces-database-queries-page/#settings). +More details can be found [here](docs/apm/transactions/transaction-traces/transaction-traces-database-queries-page/#settings). ### Transaction Traces @@ -267,8 +267,8 @@ More details can be found [here](https://docs.newrelic.com/docs/apm/transactions - Number of monitored method calls per connected services -In APM, [transaction traces](https://docs.newrelic.com/docs/apm/transactions/transaction-traces/transaction-traces) record in-depth details about your application's transactions and database calls. You can edit the default settings for transaction traces. -This is also highly configurable using [this guide](https://docs.newrelic.com/docs/apm/transactions/transaction-traces/configure-transaction-traces) +In APM, [transaction traces](/docs/apm/transactions/transaction-traces/transaction-traces) record in-depth details about your application's transactions and database calls. You can edit the default settings for transaction traces. +This is also highly configurable using [this guide](/docs/apm/transactions/transaction-traces/configure-transaction-traces) The level and mode of configurability will be language-specific in many cases. Transaction trace settings available using server-side configuration will differ depending on the New Relic agent you use. The UI includes descriptions of each. Settings in the UI may include: @@ -325,7 +325,7 @@ The main parameters that could drive a small increase in monthly ingest are: - Error activity -In [agent version 1211](https://docs.newrelic.com/docs/release-notes/new-relic-browser-release-notes/browser-agent-release-notes/) or higher, all network requests made by a page are recorded as AjaxRequest events. You can use the deny list configuration options in the Application settings page to filter which requests record events. Regardless of this filter, all network requests are captured as metrics and available in the AJAX page. +In [agent version 1211](/docs/release-notes/new-relic-browser-release-notes/browser-agent-release-notes/) or higher, all network requests made by a page are recorded as AjaxRequest events. You can use the deny list configuration options in the Application settings page to filter which requests record events. Regardless of this filter, all network requests are captured as metrics and available in the AJAX page. ### Using the deny list @@ -375,7 +375,7 @@ FROM AjaxRequest SELECT * WHERE requestUrl LIKE `%example.com%` ### Android -[Feature Flags](https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-android/android-sdk-api/android-agent-configuration-feature-flags/) +[Feature Flags](/docs/mobile-monitoring/new-relic-mobile-android/android-sdk-api/android-agent-configuration-feature-flags/) All settings, including the call to invoke the agent, are called in the onCreate method of the MainActivity class. To change settings, call the setting in one of two ways (if the setting supports it): @@ -419,7 +419,7 @@ The following feature flags can be configured: - NRFeatureFlag_RequestErrorEvents - NRFeatureFlag_HttpResponseBodyCapture -See this document for [more details](https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-ios/ios-sdk-api/ios-agent-configuration-feature-flags/). +See this document for [more details](/docs/mobile-monitoring/new-relic-mobile-ios/ios-sdk-api/ios-agent-configuration-feature-flags/). -New Relic's [Infrastructure agent configuration file](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings/) contains a couple of powerful ways to control ingest volume. The most important is using sampling rates. There are several distinct sampling rate configurations that can be used: +New Relic's [Infrastructure agent configuration file](/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings/) contains a couple of powerful ways to control ingest volume. The most important is using sampling rates. There are several distinct sampling rate configurations that can be used: The other is through custom process samle filters. ### Sampling Rates @@ -464,7 +464,7 @@ This has the same effect as setting `metrics_process_sample_rate` to -1. By default, processes using low memory are excluded from being sampled. For more information, see `disable-zero-mem-process-filter`. -You can control how much data is sent to New Relic by configuring `include_matching_metrics`, which allows you to restrict the transmission of metric data based on the values of metric [attributes](https://docs.newrelic.com/docs/query-your-data/nrql-new-relic-query-language/nrql-query-tutorials/query-infrastructure-dimensional-metrics-nrql#naming-conventions). +You can control how much data is sent to New Relic by configuring `include_matching_metrics`, which allows you to restrict the transmission of metric data based on the values of metric [attributes](/docs/query-your-data/nrql-new-relic-query-language/nrql-query-tutorials/query-infrastructure-dimensional-metrics-nrql#naming-conventions). You include metric data by defining literal or partial values for any of the attributes of the metric. For example, you can choose to send the host.process.cpuPercent of all processes whose process.name match the ^java regular expression. @@ -535,7 +535,7 @@ network_interface_filters: ### Custom attributes -[Custom attributes](https://docs.newrelic.com/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine.) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine. +[Custom attributes](/docs/data-apis/custom-data/custom-events/collect-custom-attributes/) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine.) are key-value pairs (similar to tags in other tools) used to annotate the data from the Infrastructure agent. You can use this metadata to build filter sets, group your results, and annotate your data. For example, you might indicate a machine's environment (staging or production), the service a machine hosts (login service, for example), or the team responsible for that machine. Example of custom attributes from `newrelic.yml` @@ -621,7 +621,7 @@ New Relic's On Host Integrations (OHI for short) represent a diverse set of inte We'll use a few examples to demonstrate. -### [PostgreSQL Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/postgresql-monitoring-integration/#example-postgresSQL-collection-config) +### [PostgreSQL Integration](/docs/infrastructure/host-integrations/host-integrations-list/postgresql-monitoring-integration/#example-postgresSQL-collection-config) - Number of tables monitored @@ -661,7 +661,7 @@ integrations: inventory_source: config/postgresql ``` -### [Kafka Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/kafka-monitoring-integration/) +### [Kafka Integration](/docs/infrastructure/host-integrations/host-integrations-list/kafka-monitoring-integration/) - Number of brokers in cluster @@ -681,7 +681,7 @@ The Kafka OHI configuration provides these adjustable settings that can help man It should be noted that collection of topic level metrics especiallly offsets can be resource intensive to collect and can have an impact on data volume. It is very possible that a clusters ingest can increase by an order of magnitude simply by the addition of new Kafka topics to the cluster. -### [MongoDB Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/mongodb-monitoring-integration) +### [MongoDB Integration](/docs/infrastructure/host-integrations/host-integrations-list/mongodb-monitoring-integration) - Number of databases monitored @@ -726,7 +726,7 @@ integrations: inventory_source: config/mongodb ``` -### [Elasticsearch Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch-monitoring-integration) +### [Elasticsearch Integration](/docs/infrastructure/host-integrations/host-integrations-list/elasticsearch-monitoring-integration) - Number of nodes in cluster @@ -773,7 +773,7 @@ integrations: inventory_source: config/elasticsearch ``` -### [JMX Integration](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/jmx-monitoring-integration) +### [JMX Integration](/docs/infrastructure/host-integrations/host-integrations-list/jmx-monitoring-integration) - Metrics listed in COLLECTION_CONFIGS @@ -812,13 +812,13 @@ COLLECTION_CONFIG={"collect":[{"domain":"Catalina","event_type":"TomcatSample"," There many other OHIs having configuration options that will help you optimize collection. Some commonly used ones are: -- [NGINX](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/nginx/nginx-advanced-config) -- [MySQL](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/mySQL/mysql-integration) -- [Redis](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/redis-monitoring-integration) -- [Apache](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/apache-monitoring-integration) -- [RabbitMQ](https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/rabbitmq-monitoring-integration) +- [NGINX](/docs/infrastructure/host-integrations/host-integrations-list/nginx/nginx-advanced-config) +- [MySQL](/docs/infrastructure/host-integrations/host-integrations-list/mySQL/mysql-integration) +- [Redis](/docs/infrastructure/host-integrations/host-integrations-list/redis-monitoring-integration) +- [Apache](/docs/infrastructure/host-integrations/host-integrations-list/apache-monitoring-integration) +- [RabbitMQ](/docs/infrastructure/host-integrations/host-integrations-list/rabbitmq-monitoring-integration) -This is a good [starting point](https://docs.newrelic.com/docs/infrastructure/infrastructure-integrations/get-started/introduction-infrastructure-integrations#on-host) to learn more. +This is a good [starting point](/docs/infrastructure/infrastructure-integrations/get-started/introduction-infrastructure-integrations#on-host) to learn more. @@ -833,7 +833,7 @@ This is a good [starting point](https://docs.newrelic.com/docs/infrastructure/in - traps configured -This section focuses on New Relic's network performance monitoring which relies in the `ktranslate` agent from Kentik. This agent is quite sophisticated and it is important to fully understand the [advanced configuration docs](https://docs.newrelic.com/docs/network-performance-monitoring/advanced/advanced-config) before major optimization efforts. +This section focuses on New Relic's network performance monitoring which relies in the `ktranslate` agent from Kentik. This agent is quite sophisticated and it is important to fully understand the [advanced configuration docs](/docs/network-performance-monitoring/advanced/advanced-config) before major optimization efforts. - mibs_enabled: Array of all active MIBs the ktranslate docker image will poll. This list is automatically generated during discovery if the discovery_add_mibs attribute is true. MIBs not listed here will not be polled on any device in the configuration file. You can specify a SNMP table directly in a MIB file using MIB-NAME.tableName syntax. Ex: HOST-RESOURCES-MIB.hrProcessorTable. - user_tags: key:value pair attributes to give more context to the device. Tags at this level will be applied to all devices in the configuration file. @@ -896,7 +896,7 @@ Below are some detailed routing and filtering resources: - [Common filter and routing patterns in Fluentd](https://docs.fluentd.org/configuration/routing-examples) - [Fluentbit data pipeline](https://docs.fluentbit.io/manual/concepts/data-pipeline) -- [Forwarding logs with New Relic Infrastructure agent](https://docs.newrelic.com/docs/logs/forward-logs/forward-your-logs-using-infrastructure-agent/) +- [Forwarding logs with New Relic Infrastructure agent](/docs/logs/forward-logs/forward-your-logs-using-infrastructure-agent/) @@ -912,11 +912,11 @@ Below are some detailed routing and filtering resources: New Relic provides two primary options for sending Prometheus metrics to NRDB. The best practices for managing metric ingest in this document will be focused primarily on Option 2, the Prometheus OpenMetrics Integration (POMI), as this component was created by New Relic. -### Option 1: [Prometheus Remote Write](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) +### Option 1: [Prometheus Remote Write](/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) -Prometheus Server scrape config options are [fully documented here](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config). These scrape configs determine which metrics are collected by Prometheus Server and by configuring the [remote_write](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) parameter, the collected metrics can be written to NRDB via the New Relic Metrics API. +Prometheus Server scrape config options are [fully documented here](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config). These scrape configs determine which metrics are collected by Prometheus Server and by configuring the [remote_write](/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration) parameter, the collected metrics can be written to NRDB via the New Relic Metrics API. -### Option 2: [Prometheus OpenMetrics Integration (POMI)](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-openmetrics/install-update-or-uninstall-your-prometheus-openmetrics-integration) +### Option 2: [Prometheus OpenMetrics Integration (POMI)](/docs/integrations/prometheus-integrations/install-configure-openmetrics/install-update-or-uninstall-your-prometheus-openmetrics-integration) POMI is a standalone integration that scrapes metrics from both dynamically discovered and static Prometheus endpoints. POMI then sends this data to NRDB via the New Relic Metrics API. This integration is ideal for customers not currently running Prometheus Server. @@ -958,7 +958,7 @@ The endpoints targeted by the New Relic Kubernetes Daemonset are outlined [here] #### POMI: Co-existing with *nri-kubernetes* -New Relic’s [Kubernetes Integration](https://docs.newrelic.com/docs/integrations/kubernetes-integration/get-started/introduction-kubernetes-integration) collects a [number of metrics](https://docs.newrelic.com/docs/integrations/kubernetes-integration/understand-use-data/find-use-your-kubernetes-data#metrics) OOTB, however, it does not collect every possible metric available from a Kubernetes Cluster. +New Relic’s [Kubernetes Integration](/docs/integrations/kubernetes-integration/get-started/introduction-kubernetes-integration) collects a [number of metrics](/docs/integrations/kubernetes-integration/understand-use-data/find-use-your-kubernetes-data#metrics) OOTB, however, it does not collect every possible metric available from a Kubernetes Cluster. In the POMI config, you’ll see a section similar to this which will *disable* metric collection for a subset of metrics that the New Relic Kubernetes integration is already collecting from *Kube State Metrics*. @@ -1022,7 +1022,7 @@ spec: Although cardinality is not associated directly with billable per GB ingest, NR1 does maintain certain rate limits on cardinality and data points per minute. Being able to visualize cardinality and DPM from a Prometheus cluster can be very important. -Trial and paid accounts receive a 1M DPM and 1M cardinality limit for trial purposes, but you can request up to 15M DPM and 15M cardinality for your account. To request changes to your metric rate limits, contact your New Relic account representative. View this [doc](https://docs.newrelic.com/docs/data-apis/ingest-apis/metric-api/metric-api-limits-restricted-attributes/) for infor. +Trial and paid accounts receive a 1M DPM and 1M cardinality limit for trial purposes, but you can request up to 15M DPM and 15M cardinality for your account. To request changes to your metric rate limits, contact your New Relic account representative. View this [doc](/docs/data-apis/ingest-apis/metric-api/metric-api-limits-restricted-attributes/) for infor. If you’re already running Prometheus Server, you can run DPM and Cardinality estimates there prior to enabling POMI or remote_write. @@ -1056,8 +1056,8 @@ New Relic cloud integrations get data from cloud providers' APIs. Data is genera If you want to report more or less data from your cloud integrations, or if you need to control the use of the cloud providers' APIs to prevent reaching rate and throttling limits in your cloud account, you can change the configuration settings to modify the amount of data they report. The two main controls are: -- [Change polling frequency](https://docs.newrelic.com/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/#polling) -- [Change what data is reported](https://docs.newrelic.com/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/#filter-data) +- [Change polling frequency](/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/#polling) +- [Change what data is reported](/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/#filter-data) Examples of business reasons for wanting to change your polling frequency include: @@ -1068,11 +1068,11 @@ Examples of business reasons for wanting to change your polling frequency includ Changing the configuration settings for your integrations may impact alert conditions and chart trends. -View [this doc](https://docs.newrelic.com/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/) for more details. +View [this doc](/docs/infrastructure/infrastructure-integrations/cloud-integrations/configure-polling-frequency-data-collection-cloud-integrations/) for more details. ### *Streaming* or *Pushed* Metrics -More and more cloud integrations are offering the option of having the data pushed via a [streaming service](https://docs.newrelic.com/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/). This has proven to cut down on latency drastically. One issue some users have observed is that it's not as easy to control volume since sampling rate cannot be configured. +More and more cloud integrations are offering the option of having the data pushed via a [streaming service](/docs/infrastructure/amazon-integrations/aws-integrations-list/aws-metric-stream/). This has proven to cut down on latency drastically. One issue some users have observed is that it's not as easy to control volume since sampling rate cannot be configured. *Drop rules* will be well covered in the next section. They are the primary way of filtering out streaming metrics that are too high volume. However there are some things that can be done on the cloud provider side to limit the stream volume somewhat. @@ -1160,7 +1160,7 @@ In our previous section on prioritizing telemetry we ran through some exercises Omit debug logs (knowning they can be turned on if there is an issue) (saves 5%) ``` -#### Method 1: [Log UI](https://docs.newrelic.com/docs/logs/ui-data/drop-data-drop-filter-rules/) +#### Method 1: [Log UI](/docs/logs/ui-data/drop-data-drop-filter-rules/) - Identify the logs we care about using a filter in the Log UI: `level: DEBUG` - Make sure it finds the logs we want to drop @@ -1168,7 +1168,7 @@ Omit debug logs (knowning they can be turned on if there is an issue) (saves 5%) - Under *Manage Data* click *Drop filters* and create and enable a filter named 'Drop Debug Logs' - Verify the rule works -#### Method 2: [GraphQL API](https://docs.newrelic.com/docs/data-apis/manage-data/drop-data-using-nerdgraph/) +#### Method 2: [GraphQL API](/docs/data-apis/manage-data/drop-data-using-nerdgraph/) - Create the relevant NRQL query: `SELECT count(*) FROM Log WHERE `level` = 'DEBUG'` - Make sure it finds the logs we want to drop @@ -1283,4 +1283,4 @@ The preceding examples should show you all you need to know to use these techniq - \ No newline at end of file +