From 15fcc9e1665f0e6d9f6ccf32fea0a955f7300f49 Mon Sep 17 00:00:00 2001 From: tonypowa Date: Fri, 26 Apr 2024 13:42:56 +0200 Subject: [PATCH 01/13] docs / alerting / fundamentals / templates --- .../_index.md} | 2 +- .../templates/template-labels-annotations.md | 202 ++++++++++++++++++ 2 files changed, 203 insertions(+), 1 deletion(-) rename docs/sources/alerting/fundamentals/notifications/{message-templating.md => templates/_index.md} (98%) create mode 100644 docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md diff --git a/docs/sources/alerting/fundamentals/notifications/message-templating.md b/docs/sources/alerting/fundamentals/notifications/templates/_index.md similarity index 98% rename from docs/sources/alerting/fundamentals/notifications/message-templating.md rename to docs/sources/alerting/fundamentals/notifications/templates/_index.md index 6f93872912991..36781e78e7972 100644 --- a/docs/sources/alerting/fundamentals/notifications/message-templating.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/_index.md @@ -3,7 +3,7 @@ aliases: - ../../contact-points/message-templating/ # /docs/grafana//alerting/contact-points/message-templating/ - ../../alert-rules/message-templating/ # /docs/grafana//alerting/alert-rules/message-templating/ - ../../unified-alerting/message-templating/ # /docs/grafana//alerting/unified-alerting/message-templating/ -canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/message-templating/ +canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/ description: Learn about templates keywords: - grafana diff --git a/docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md b/docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md new file mode 100644 index 0000000000000..0f6464987d959 --- /dev/null +++ b/docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md @@ -0,0 +1,202 @@ +--- +canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/template-labels-annotations + +description: Template labels and annotations to customize alert notifications +keywords: + - grafana + - alerting + - labels + - annotations + - templating +labels: + products: + - cloud + - enterprise + - oss +menuTitle: Template labels and annotations +title: Template labels and annotations +weight: 300 +--- + +# Labels and annotations + +Labels and annotations contain information about an alert. Labels are used to differentiate an alert from all other alerts, while annotations are used to add additional information to an existing alert. + +When creating alert rules, you can also [template labels and annotations](#template-labels) to optimize and customize your alerts. + +## Labels + +**Labels** are unique identifiers of an alert. You can use them for searching, silencing, and routing notifications. + +Examples of labels are `server=server1` or `team=backend`. Each alert rule can have more than one label and the complete set of labels for an alert rule is called its label set. It is this label set that identifies the alert. + +For example, an alert rule might have the label set `{alertname="High CPU usage",server="server1"}` while another alert rule might have the label set `{alertname="High CPU usage",server="server2"}`. These are two separate alert rules because although their `alertname` labels are the same, their `server` labels are different. + +Labels are a fundamental component of alerting: + +- The complete set of labels for an alert is what uniquely identifies an alert within Grafana alerts. +- The alerting UI shows labels for every alert instance generated during evaluation of that rule. +- Contact points can access labels to send notification messages that contain specific alert information. +- The Alertmanager uses labels to match alerts for silences and alert groups in notification policies. + +Note that two alert rules cannot have the same labels. If two alert rules have the same labels such as `foo=bar,bar=baz` and `foo=bar,bar=baz` then one of the alerts will be discarded. + +### How label matching works + +Use labels and label matchers to link alert rules to notification policies and silences. This allows for a flexible way to manage your alert instances, specify which policy should handle them, and which alerts to silence. + +A label matchers consists of 3 distinct parts, the **label**, the **value** and the **operator**. + +- The **Label** field is the name of the label to match. It must exactly match the label name. + +- The **Value** field matches against the corresponding value for the specified **Label** name. How it matches depends on the **Operator** value. + +- The **Operator** field is the operator to match against the label value. The available operators are: + + | Operator | Description | + | -------- | -------------------------------------------------- | + | `=` | Select labels that are exactly equal to the value. | + | `!=` | Select labels that are not equal to the value. | + | `=~` | Select labels that regex-match the value. | + | `!~` | Select labels that do not regex-match the value. | + +If you are using multiple label matchers, they are combined using the AND logical operator. This means that all matchers must match in order to link a rule to a policy. + +{{< collapse title="Label matching example" >}} + +If you define the following set of labels for your alert: + +`{ foo=bar, baz=qux, id=12 }` + +then: + +- A label matcher defined as `foo=bar` matches this alert rule. +- A label matcher defined as `foo!=bar` does _not_ match this alert rule. +- A label matcher defined as `id=~[0-9]+` matches this alert rule. +- A label matcher defined as `baz!~[0-9]+` matches this alert rule. +- Two label matchers defined as `foo=bar` and `id=~[0-9]+` match this alert rule. + +**Exclude labels** + +You can also write label matchers to exclude labels. + +Here is an example that shows how to exclude the label `Team`. You can choose between any of the values below to exclude labels. + +| Label | Operator | Value | +| ------ | -------- | ----- | +| `team` | `=` | `""` | +| `team` | `!~` | `.+` | +| `team` | `=~` | `^$` | + +{{< /collapse >}} + +## Label types + +An alert's label set can contain three types of labels: + +- Data source query labels. For example, if you are monitoring temperature readings and each time series for these readings has a `sensor_id`, and a `location` label. These labels are used to provide additional context or dimensions to the metric data, helping to differentiate between different time series. + +- Labels that are automatically added by Grafana (i.e. alertname and grafana_folder). These are Grafana reserved labels. + +- Labels that you define yourself to help filter data in your alert rules. + You can also template labels. For example in your alert rule, you could add a label that uses templating to create more dynamic and customizable alerting. E.g. `environment` `=` `{{ your text/template }}`. + +{{}} +Ensure the label set for an alert does not have two or more labels with the same name. If a label has the same name as a label from the data source then it will replace that label. However, should a label have the same name as a reserved label then the label will be omitted from the alert. +{{}} + +{{< collapse title="Key format" >}} + +Grafana's built-in Alertmanager supports both Unicode label keys and values. If you are using an external Prometheus Alertmanager, label keys must be compatible with their [data model](https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels). +This means that label keys must only contain **ASCII letters**, **numbers**, as well as **underscores** and match the regex `[a-zA-Z_][a-zA-Z0-9_]*`. +Any invalid characters will be removed or replaced by the Grafana alerting engine before being sent to the external Alertmanager according to the following rules: + +- `Whitespace` will be removed. +- `ASCII characters` will be replaced with `_`. +- `All other characters` will be replaced with their lower-case hex representation. If this is the first character it will be prefixed with `_`. + +Example: A label key/value pair `Alert! 🔔="🔥"` will become `Alert_0x1f514="🔥"`. + +If multiple label keys are sanitized to the same value, the duplicates will have a short hash of the original label appended as a suffix. + +{{< /collapse >}} + +### Reserved labels + +Reserved labels can be used in the same way as manually configured labels. The current list of available reserved labels are: + +| Label | Description | +| -------------- | ----------------------------------------- | +| alert_name | The name of the alert rule. | +| grafana_folder | Title of the folder containing the alert. | + +Labels prefixed with `grafana_` are reserved by Grafana for special use. To stop Grafana Alerting from adding a reserved label, you can disable it via the `disabled_labels` option in [unified_alerting.reserved_labels](/docs/grafana//setup-grafana/configure-grafana#unified_alertingreserved_labels) configuration. + + +### Template labels + +Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). + +{{ }} +Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one +{{ }} + +Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. + +{{ }} +In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. +{{ }} + +Here are some commonly used built-in [variables][variables-label-annotation] to interact with the name and value of labels in Grafana alerting: + +- The `$labels` variable, which contains all labels from the query. + + For example, let's say you have an alert rule that triggers when the CPU usage exceeds a certain threshold. You want to create annotations that provide additional context when this alert is triggered, such as including the specific server that experienced the high CPU usage. + + + The host {{ index $labels "instance" }} has exceeded 80% CPU usage for the last 5 minutes + + The outcome of this template would print: + + The host instance 1 has exceeded 80% CPU usage for the last 5 minutes + +- The `$value` variable, which is a string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule. + + In the context of the previous example, $value variable would write something like this: + + CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ $value }} + + The outcome of this template would print: + + CPU usage for instance1 has exceeded 80% for the last 5 minutes: [ var='A' labels={instance=instance1} value=81.234 ] + +- The `$values` variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (i.e. the id that identifies the query or expression. By default the Red ID of the query is “A”). + + Given an alert with the labels instance=server1 and an instant query with the value 81.2345, would write like this: + + + CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ index $values "A" }} + + And it would print: + + CPU usage for instance1 has exceeded 80% for the last 5 minutes: 81.2345 + +## Annotations + +Both labels and annotations have the same structure: a set of named values; however their intended uses are different. The purpose of annotations is to add additional information to existing alerts. + +There are a number of suggested annotations in Grafana such as `description`, `summary`, `runbook_url`, `dashboardUId` and `panelId`. Like labels, annotations must have a name, and their value can contain a combination of text and template code that is evaluated when an alert is fired. + +Here is an example of templating an annotation in the context of an alert rule. The text/template is added into the Add annotations section. + + CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes + +The outcome of this template would print + + CPU usage for Instance 1 has exceeded 80% for the last 5 minutes + + +{{% docs/reference %}} +[variables-label-annotation]: "/docs/grafana/ -> /docs/grafana//alerting/alerting-rules/templating-labels-annotations" +[variables-label-annotation]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/templating-labels-annotations" +{{% /docs/reference %}} \ No newline at end of file From 85c5d086ac694f49d60a2f5d72c962a2e9d3ae32 Mon Sep 17 00:00:00 2001 From: tonypowa Date: Fri, 26 Apr 2024 13:54:16 +0200 Subject: [PATCH 02/13] renamed and adjusted front matter --- .../{template-labels-annotations.md => annotation-label.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename docs/sources/alerting/fundamentals/notifications/templates/{template-labels-annotations.md => annotation-label.md} (98%) diff --git a/docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md similarity index 98% rename from docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md rename to docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md index 0f6464987d959..91e3c5ca6c745 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates/template-labels-annotations.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md @@ -1,7 +1,7 @@ --- canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/template-labels-annotations -description: Template labels and annotations to customize alert notifications +description: Learn how to use annotations and labels to store key information about alerts and customize alert notifications keywords: - grafana - alerting @@ -13,8 +13,8 @@ labels: - cloud - enterprise - oss -menuTitle: Template labels and annotations -title: Template labels and annotations +menuTitle: labels and annotations +title: Labels and annotations weight: 300 --- From 5045b14ee788b7aec72574e59dae781000fe240b Mon Sep 17 00:00:00 2001 From: tonypowa Date: Fri, 26 Apr 2024 14:53:43 +0200 Subject: [PATCH 03/13] pretty --- .../templates/annotation-label.md | 22 ++++++++----------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md index 91e3c5ca6c745..a200cc3ea4102 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md @@ -132,7 +132,6 @@ Reserved labels can be used in the same way as manually configured labels. The c Labels prefixed with `grafana_` are reserved by Grafana for special use. To stop Grafana Alerting from adding a reserved label, you can disable it via the `disabled_labels` option in [unified_alerting.reserved_labels](/docs/grafana//setup-grafana/configure-grafana#unified_alertingreserved_labels) configuration. - ### Template labels Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). @@ -141,43 +140,41 @@ Label templates are applied in the alert rule itself (i.e. in the Configure labe Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one {{ }} -Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. +Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. {{ }} -In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. +In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. {{ }} Here are some commonly used built-in [variables][variables-label-annotation] to interact with the name and value of labels in Grafana alerting: - The `$labels` variable, which contains all labels from the query. - For example, let's say you have an alert rule that triggers when the CPU usage exceeds a certain threshold. You want to create annotations that provide additional context when this alert is triggered, such as including the specific server that experienced the high CPU usage. - + For example, let's say you have an alert rule that triggers when the CPU usage exceeds a certain threshold. You want to create annotations that provide additional context when this alert is triggered, such as including the specific server that experienced the high CPU usage. The host {{ index $labels "instance" }} has exceeded 80% CPU usage for the last 5 minutes - The outcome of this template would print: + The outcome of this template would print: The host instance 1 has exceeded 80% CPU usage for the last 5 minutes - The `$value` variable, which is a string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule. - In the context of the previous example, $value variable would write something like this: + In the context of the previous example, $value variable would write something like this: CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ $value }} - The outcome of this template would print: + The outcome of this template would print: CPU usage for instance1 has exceeded 80% for the last 5 minutes: [ var='A' labels={instance=instance1} value=81.234 ] - The `$values` variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (i.e. the id that identifies the query or expression. By default the Red ID of the query is “A”). - Given an alert with the labels instance=server1 and an instant query with the value 81.2345, would write like this: - + Given an alert with the labels instance=server1 and an instant query with the value 81.2345, would write like this: CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ index $values "A" }} - And it would print: + And it would print: CPU usage for instance1 has exceeded 80% for the last 5 minutes: 81.2345 @@ -195,8 +192,7 @@ The outcome of this template would print CPU usage for Instance 1 has exceeded 80% for the last 5 minutes - {{% docs/reference %}} [variables-label-annotation]: "/docs/grafana/ -> /docs/grafana//alerting/alerting-rules/templating-labels-annotations" [variables-label-annotation]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/templating-labels-annotations" -{{% /docs/reference %}} \ No newline at end of file +{{% /docs/reference %}} From b4a3a93d07c0182beb8ac9e4f3254739c7c3af50 Mon Sep 17 00:00:00 2001 From: tonypowa Date: Fri, 26 Apr 2024 15:02:06 +0200 Subject: [PATCH 04/13] frontmatter --- .../fundamentals/notifications/templates/_index.md | 8 ++++---- .../notifications/templates/annotation-label.md | 5 ++--- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates/_index.md b/docs/sources/alerting/fundamentals/notifications/templates/_index.md index 36781e78e7972..61ee7a19b690b 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates/_index.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/_index.md @@ -1,8 +1,8 @@ --- -aliases: - - ../../contact-points/message-templating/ # /docs/grafana//alerting/contact-points/message-templating/ - - ../../alert-rules/message-templating/ # /docs/grafana//alerting/alert-rules/message-templating/ - - ../../unified-alerting/message-templating/ # /docs/grafana//alerting/unified-alerting/message-templating/ +# aliases: +# - ../../contact-points/message-templating/ # /docs/grafana//alerting/contact-points/message-templating/ +# - ../../alert-rules/message-templating/ # /docs/grafana//alerting/alert-rules/message-templating/ +# - ../../unified-alerting/message-templating/ # /docs/grafana//alerting/unified-alerting/message-templating/ canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/ description: Learn about templates keywords: diff --git a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md index a200cc3ea4102..2cf5378d59ca8 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md @@ -1,6 +1,5 @@ --- -canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/template-labels-annotations - +canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/annotation-label description: Learn how to use annotations and labels to store key information about alerts and customize alert notifications keywords: - grafana @@ -13,7 +12,7 @@ labels: - cloud - enterprise - oss -menuTitle: labels and annotations +menuTitle: Labels and annotations title: Labels and annotations weight: 300 --- From e39d4c8efd93b737b68bbc201e1c990d5b2b784a Mon Sep 17 00:00:00 2001 From: tonypowa Date: Fri, 26 Apr 2024 15:13:37 +0200 Subject: [PATCH 05/13] admonition fix --- .../notifications/templates/annotation-label.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md index 2cf5378d59ca8..c6422ead34036 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md @@ -135,15 +135,15 @@ Labels prefixed with `grafana_` are reserved by Grafana for special use. To stop Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). -{{ }} +{} Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one -{{ }} +{} Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. -{{ }} +{} In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. -{{ }} +{} Here are some commonly used built-in [variables][variables-label-annotation] to interact with the name and value of labels in Grafana alerting: From aad7b134c66d5e11ab4603312c48038cc0f8b663 Mon Sep 17 00:00:00 2001 From: tonypowa Date: Fri, 26 Apr 2024 15:15:46 +0200 Subject: [PATCH 06/13] admo --- .../notifications/templates/annotation-label.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md index c6422ead34036..07226eb7c1793 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md +++ b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md @@ -135,15 +135,15 @@ Labels prefixed with `grafana_` are reserved by Grafana for special use. To stop Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). -{} -Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one -{} +{{}} +Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one. +{{}} Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. -{} +{{}} In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. -{} +{{}} Here are some commonly used built-in [variables][variables-label-annotation] to interact with the name and value of labels in Grafana alerting: From ffe623b950893ca710753b9c6a82c5c354912070 Mon Sep 17 00:00:00 2001 From: tonypowa Date: Tue, 30 Apr 2024 10:11:33 +0200 Subject: [PATCH 07/13] restructuring --- .../fundamentals/notifications/templates.md | 159 ++++++++++++++ .../notifications/templates/_index.md | 51 ----- .../templates/annotation-label.md | 197 ------------------ 3 files changed, 159 insertions(+), 248 deletions(-) create mode 100644 docs/sources/alerting/fundamentals/notifications/templates.md delete mode 100644 docs/sources/alerting/fundamentals/notifications/templates/_index.md delete mode 100644 docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md new file mode 100644 index 0000000000000..1207ad9d1dd8e --- /dev/null +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -0,0 +1,159 @@ +--- +aliases: + - ../../contact-points/message-templating/ # /docs/grafana//alerting/contact-points/message-templating/ + - ../../alert-rules/message-templating/ # /docs/grafana//alerting/alert-rules/message-templating/ + - ../../unified-alerting/message-templating/ # /docs/grafana//alerting/unified-alerting/message-templating/ +canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/ +description: Learn about templates +keywords: + - grafana + - alerting + - guide + - contact point + - templating +labels: + products: + - cloud + - enterprise + - oss +title: Templates +weight: 114 +--- + +# Templates + +Use templating to customize, format, and reuse alert notification messages. Create more flexible and informative alert notification messages by incorporating dynamic content, such as metric values, labels, and other contextual information. + +In Grafana, there are two ways to template your alert notification messages: + +1. Labels and annotations + + - Template labels and annotations in alert rules. + - Labels and annotations contain information about an alert. + - Labels are used to differentiate an alert from all other alerts, while annotations are used to add additional information to an existing alert. + +2. Notification templates + + - Template notifications in contact points. + - Add notification templates to contact points for reuse and consistent messaging in your notifications. + - Use notification templates to change the title, message, and format of the message in your notifications. + +This diagram illustrates the entire process of templating, from the creation of labels and annotations in alert rules or notification templates in contact points, to what they look like when exported and applied in your alert notification messages. + +{{< figure src="/media/docs/alerting/grafana-templating-diagram-2.jpg" max-width="1200px" caption="How Templating works" >}} + +In this diagram: + +- **Monitored Application**: A web server, database, or any other service generating metrics. For example, it could be an NGINX server providing metrics about request rates, response times, and so on. +- **Prometheus**: Prometheus collects metrics from the monitored application. For example, it might scrape metrics from the NGINX server, including labels like instance (the server hostname) and job (the service name). +- **Grafana**: Grafana queries Prometheus to retrieve metrics data. For example, you might create an alert rule to monitor NGINX request rates over time, and template labels or annotations based on the instance label. +- **Alertmanager**: Part of the Prometheus ecosystem, Alertmanager handles alert notifications. For example, if the request rate exceeds a certain threshold on a particular NGINX server, Alertmanager can send an alert notification to, for example, Slack or email, including the server name and the exceeded threshold (the instance label will be interpolated, and the actual server name will appear in the alert notification). +- **Alert notification**: When an alert rule condition is met, Alertmanager sends a notification to various channels such as Slack, Grafana OnCall, etc. These notifications can include information from the labels associated with the alerting rule. For example, if an alert triggers due to high CPU usage on a specific server, the notification message can include details like server name (instance label), disk usage percentage, and the threshold that was exceeded. + +## Labels and annotations + +Labels and annotations contain information about an alert. Labels are used to differentiate an alert from all other alerts, while annotations are used to add additional information to an existing alert. + +### Template labels + +Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). + +{{}} +Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one. +{{}} + +Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. + +{{}} +In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. +{{}} + +Here are some commonly used built-in [variables][variables-label-annotation] to interact with the name and value of labels in Grafana alerting: + +- The `$labels` variable, which contains all labels from the query. + + For example, let's say you have an alert rule that triggers when the CPU usage exceeds a certain threshold. You want to create annotations that provide additional context when this alert is triggered, such as including the specific server that experienced the high CPU usage. + + The host {{ index $labels "instance" }} has exceeded 80% CPU usage for the last 5 minutes + + The outcome of this template would print: + + The host instance 1 has exceeded 80% CPU usage for the last 5 minutes + +- The `$value` variable, which is a string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule. + + In the context of the previous example, $value variable would write something like this: + + CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ $value }} + + The outcome of this template would print: + + CPU usage for instance1 has exceeded 80% for the last 5 minutes: [ var='A' labels={instance=instance1} value=81.234 ] + +- The `$values` variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (i.e. the id that identifies the query or expression. By default the Red ID of the query is “A”). + + Given an alert with the labels instance=server1 and an instant query with the value 81.2345, would write like this: + + CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ index $values "A" }} + + And it would print: + + CPU usage for instance1 has exceeded 80% for the last 5 minutes: 81.2345 + +### Template annotations + +Both labels and annotations have the same structure: a set of named values; however their intended uses are different. The purpose of annotations is to add additional information to existing alerts. + +There are a number of suggested annotations in Grafana such as `description`, `summary`, `runbook_url`, `dashboardUId` and `panelId`. Like labels, annotations must have a name, and their value can contain a combination of text and template code that is evaluated when an alert is fired. + +Here is an example of templating an annotation in the context of an alert rule. The text/template is added into the Add annotations section. + + CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes + +The outcome of this template would print + + CPU usage for Instance 1 has exceeded 80% for the last 5 minutes + +### Notification templates + +**Notification templates** represent the alternative approach to templating designed for reusing templates. Notifications are messages to inform users about events or conditions triggered by alerts. You can create reusable notification templates to customize the content and format of alert notifications. Variables, labels, or other context-specific details can be added to the templates to dynamically insert information like metric values. + +Here is the example of a notification template: + +```go +{ define "alerts.message" -}} +{{ if .Alerts.Firing -}} +{{ len .Alerts.Firing }} firing alert(s) +{{ template "alerts.summarize" .Alerts.Firing }} +{{- end }} +{{- if .Alerts.Resolved -}} +{{ len .Alerts.Resolved }} resolved alert(s) +{{ template "alerts.summarize" .Alerts.Resolved }} +{{- end }} +{{- end }} + +{{ define "alerts.summarize" -}} +{{ range . -}} +- {{ index .Annotations "summary" }} +{{ end }} +{{ end }} +``` + +The message you would receive in your contact point: + + 1 firing alert(s) + - The database server db1 has exceeded 75% of available disk space. Disk space used is 76%, please resize the disk size within the next 24 hours + + 1 resolved alert(s) + - The web server web1 has been responding to 5% of HTTP requests with 5xx errors for the last 5 minutes + +Once the template is created, you need to make reference to it in your **Contact point** (in the Optional [contact point] settings) . + +{{ }} +It's not recommended to include individual alert information within notification templates. Instead, it's more effective to incorporate such details within the rule using labels and annotations. +{{ }} + +{{% docs/reference %}} +[variables-label-annotation]: "/docs/grafana/ -> /docs/grafana//alerting/alerting-rules/templating-labels-annotations" +[variables-label-annotation]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/templating-labels-annotations" +{{% /docs/reference %}} diff --git a/docs/sources/alerting/fundamentals/notifications/templates/_index.md b/docs/sources/alerting/fundamentals/notifications/templates/_index.md deleted file mode 100644 index 61ee7a19b690b..0000000000000 --- a/docs/sources/alerting/fundamentals/notifications/templates/_index.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -# aliases: -# - ../../contact-points/message-templating/ # /docs/grafana//alerting/contact-points/message-templating/ -# - ../../alert-rules/message-templating/ # /docs/grafana//alerting/alert-rules/message-templating/ -# - ../../unified-alerting/message-templating/ # /docs/grafana//alerting/unified-alerting/message-templating/ -canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/ -description: Learn about templates -keywords: - - grafana - - alerting - - guide - - contact point - - templating -labels: - products: - - cloud - - enterprise - - oss -title: Templates -weight: 114 ---- - -## Templates - -Use templating to customize, format, and reuse alert notification messages. Create more flexible and informative alert notification messages by incorporating dynamic content, such as metric values, labels, and other contextual information. - -In Grafana, there are two ways to template your alert notification messages: - -1. Labels and annotations - - - Template labels and annotations in alert rules. - - Labels and annotations contain information about an alert. - - Labels are used to differentiate an alert from all other alerts, while annotations are used to add additional information to an existing alert. - -2. Notification templates - - - Template notifications in contact points. - - Add notification templates to contact points for reuse and consistent messaging in your notifications. - - Use notification templates to change the title, message, and format of the message in your notifications. - -This diagram illustrates the entire process of templating, from the creation of labels and annotations in alert rules or notification templates in contact points, to what they look like when exported and applied in your alert notification messages. - -{{< figure src="/media/docs/alerting/grafana-templating-diagram-2.jpg" max-width="1200px" caption="How Templating works" >}} - -In this diagram: - -- **Monitored Application**: A web server, database, or any other service generating metrics. For example, it could be an NGINX server providing metrics about request rates, response times, and so on. -- **Prometheus**: Prometheus collects metrics from the monitored application. For example, it might scrape metrics from the NGINX server, including labels like instance (the server hostname) and job (the service name). -- **Grafana**: Grafana queries Prometheus to retrieve metrics data. For example, you might create an alert rule to monitor NGINX request rates over time, and template labels or annotations based on the instance label. -- **Alertmanager**: Part of the Prometheus ecosystem, Alertmanager handles alert notifications. For example, if the request rate exceeds a certain threshold on a particular NGINX server, Alertmanager can send an alert notification to, for example, Slack or email, including the server name and the exceeded threshold (the instance label will be interpolated, and the actual server name will appear in the alert notification). -- **Alert notification**: When an alert rule condition is met, Alertmanager sends a notification to various channels such as Slack, Grafana OnCall, etc. These notifications can include information from the labels associated with the alerting rule. For example, if an alert triggers due to high CPU usage on a specific server, the notification message can include details like server name (instance label), disk usage percentage, and the threshold that was exceeded. diff --git a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md b/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md deleted file mode 100644 index 07226eb7c1793..0000000000000 --- a/docs/sources/alerting/fundamentals/notifications/templates/annotation-label.md +++ /dev/null @@ -1,197 +0,0 @@ ---- -canonical: https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/templates/annotation-label -description: Learn how to use annotations and labels to store key information about alerts and customize alert notifications -keywords: - - grafana - - alerting - - labels - - annotations - - templating -labels: - products: - - cloud - - enterprise - - oss -menuTitle: Labels and annotations -title: Labels and annotations -weight: 300 ---- - -# Labels and annotations - -Labels and annotations contain information about an alert. Labels are used to differentiate an alert from all other alerts, while annotations are used to add additional information to an existing alert. - -When creating alert rules, you can also [template labels and annotations](#template-labels) to optimize and customize your alerts. - -## Labels - -**Labels** are unique identifiers of an alert. You can use them for searching, silencing, and routing notifications. - -Examples of labels are `server=server1` or `team=backend`. Each alert rule can have more than one label and the complete set of labels for an alert rule is called its label set. It is this label set that identifies the alert. - -For example, an alert rule might have the label set `{alertname="High CPU usage",server="server1"}` while another alert rule might have the label set `{alertname="High CPU usage",server="server2"}`. These are two separate alert rules because although their `alertname` labels are the same, their `server` labels are different. - -Labels are a fundamental component of alerting: - -- The complete set of labels for an alert is what uniquely identifies an alert within Grafana alerts. -- The alerting UI shows labels for every alert instance generated during evaluation of that rule. -- Contact points can access labels to send notification messages that contain specific alert information. -- The Alertmanager uses labels to match alerts for silences and alert groups in notification policies. - -Note that two alert rules cannot have the same labels. If two alert rules have the same labels such as `foo=bar,bar=baz` and `foo=bar,bar=baz` then one of the alerts will be discarded. - -### How label matching works - -Use labels and label matchers to link alert rules to notification policies and silences. This allows for a flexible way to manage your alert instances, specify which policy should handle them, and which alerts to silence. - -A label matchers consists of 3 distinct parts, the **label**, the **value** and the **operator**. - -- The **Label** field is the name of the label to match. It must exactly match the label name. - -- The **Value** field matches against the corresponding value for the specified **Label** name. How it matches depends on the **Operator** value. - -- The **Operator** field is the operator to match against the label value. The available operators are: - - | Operator | Description | - | -------- | -------------------------------------------------- | - | `=` | Select labels that are exactly equal to the value. | - | `!=` | Select labels that are not equal to the value. | - | `=~` | Select labels that regex-match the value. | - | `!~` | Select labels that do not regex-match the value. | - -If you are using multiple label matchers, they are combined using the AND logical operator. This means that all matchers must match in order to link a rule to a policy. - -{{< collapse title="Label matching example" >}} - -If you define the following set of labels for your alert: - -`{ foo=bar, baz=qux, id=12 }` - -then: - -- A label matcher defined as `foo=bar` matches this alert rule. -- A label matcher defined as `foo!=bar` does _not_ match this alert rule. -- A label matcher defined as `id=~[0-9]+` matches this alert rule. -- A label matcher defined as `baz!~[0-9]+` matches this alert rule. -- Two label matchers defined as `foo=bar` and `id=~[0-9]+` match this alert rule. - -**Exclude labels** - -You can also write label matchers to exclude labels. - -Here is an example that shows how to exclude the label `Team`. You can choose between any of the values below to exclude labels. - -| Label | Operator | Value | -| ------ | -------- | ----- | -| `team` | `=` | `""` | -| `team` | `!~` | `.+` | -| `team` | `=~` | `^$` | - -{{< /collapse >}} - -## Label types - -An alert's label set can contain three types of labels: - -- Data source query labels. For example, if you are monitoring temperature readings and each time series for these readings has a `sensor_id`, and a `location` label. These labels are used to provide additional context or dimensions to the metric data, helping to differentiate between different time series. - -- Labels that are automatically added by Grafana (i.e. alertname and grafana_folder). These are Grafana reserved labels. - -- Labels that you define yourself to help filter data in your alert rules. - You can also template labels. For example in your alert rule, you could add a label that uses templating to create more dynamic and customizable alerting. E.g. `environment` `=` `{{ your text/template }}`. - -{{}} -Ensure the label set for an alert does not have two or more labels with the same name. If a label has the same name as a label from the data source then it will replace that label. However, should a label have the same name as a reserved label then the label will be omitted from the alert. -{{}} - -{{< collapse title="Key format" >}} - -Grafana's built-in Alertmanager supports both Unicode label keys and values. If you are using an external Prometheus Alertmanager, label keys must be compatible with their [data model](https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels). -This means that label keys must only contain **ASCII letters**, **numbers**, as well as **underscores** and match the regex `[a-zA-Z_][a-zA-Z0-9_]*`. -Any invalid characters will be removed or replaced by the Grafana alerting engine before being sent to the external Alertmanager according to the following rules: - -- `Whitespace` will be removed. -- `ASCII characters` will be replaced with `_`. -- `All other characters` will be replaced with their lower-case hex representation. If this is the first character it will be prefixed with `_`. - -Example: A label key/value pair `Alert! 🔔="🔥"` will become `Alert_0x1f514="🔥"`. - -If multiple label keys are sanitized to the same value, the duplicates will have a short hash of the original label appended as a suffix. - -{{< /collapse >}} - -### Reserved labels - -Reserved labels can be used in the same way as manually configured labels. The current list of available reserved labels are: - -| Label | Description | -| -------------- | ----------------------------------------- | -| alert_name | The name of the alert rule. | -| grafana_folder | Title of the folder containing the alert. | - -Labels prefixed with `grafana_` are reserved by Grafana for special use. To stop Grafana Alerting from adding a reserved label, you can disable it via the `disabled_labels` option in [unified_alerting.reserved_labels](/docs/grafana//setup-grafana/configure-grafana#unified_alertingreserved_labels) configuration. - -### Template labels - -Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). - -{{}} -Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one. -{{}} - -Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. - -{{}} -In Grafana templating, the $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation within alert rule definitions. -{{}} - -Here are some commonly used built-in [variables][variables-label-annotation] to interact with the name and value of labels in Grafana alerting: - -- The `$labels` variable, which contains all labels from the query. - - For example, let's say you have an alert rule that triggers when the CPU usage exceeds a certain threshold. You want to create annotations that provide additional context when this alert is triggered, such as including the specific server that experienced the high CPU usage. - - The host {{ index $labels "instance" }} has exceeded 80% CPU usage for the last 5 minutes - - The outcome of this template would print: - - The host instance 1 has exceeded 80% CPU usage for the last 5 minutes - -- The `$value` variable, which is a string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule. - - In the context of the previous example, $value variable would write something like this: - - CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ $value }} - - The outcome of this template would print: - - CPU usage for instance1 has exceeded 80% for the last 5 minutes: [ var='A' labels={instance=instance1} value=81.234 ] - -- The `$values` variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (i.e. the id that identifies the query or expression. By default the Red ID of the query is “A”). - - Given an alert with the labels instance=server1 and an instant query with the value 81.2345, would write like this: - - CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes: {{ index $values "A" }} - - And it would print: - - CPU usage for instance1 has exceeded 80% for the last 5 minutes: 81.2345 - -## Annotations - -Both labels and annotations have the same structure: a set of named values; however their intended uses are different. The purpose of annotations is to add additional information to existing alerts. - -There are a number of suggested annotations in Grafana such as `description`, `summary`, `runbook_url`, `dashboardUId` and `panelId`. Like labels, annotations must have a name, and their value can contain a combination of text and template code that is evaluated when an alert is fired. - -Here is an example of templating an annotation in the context of an alert rule. The text/template is added into the Add annotations section. - - CPU usage for {{ index $labels "instance" }} has exceeded 80% for the last 5 minutes - -The outcome of this template would print - - CPU usage for Instance 1 has exceeded 80% for the last 5 minutes - -{{% docs/reference %}} -[variables-label-annotation]: "/docs/grafana/ -> /docs/grafana//alerting/alerting-rules/templating-labels-annotations" -[variables-label-annotation]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/templating-labels-annotations" -{{% /docs/reference %}} From d98b03bc4de6f5d323ebd62dd399a9c0a96cd80c Mon Sep 17 00:00:00 2001 From: antonio <45235678+tonypowa@users.noreply.github.com> Date: Tue, 30 Apr 2024 11:50:31 +0200 Subject: [PATCH 08/13] Update docs/sources/alerting/fundamentals/notifications/templates.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --- docs/sources/alerting/fundamentals/notifications/templates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md index 1207ad9d1dd8e..688de3a030f58 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates.md +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -139,7 +139,7 @@ Here is the example of a notification template: {{ end }} ``` -The message you would receive in your contact point: +This is the message you would receive in your contact point: 1 firing alert(s) - The database server db1 has exceeded 75% of available disk space. Disk space used is 76%, please resize the disk size within the next 24 hours From 0e20f44550eeb792e10814d57a7029e0a830f552 Mon Sep 17 00:00:00 2001 From: antonio <45235678+tonypowa@users.noreply.github.com> Date: Tue, 30 Apr 2024 11:50:37 +0200 Subject: [PATCH 09/13] Update docs/sources/alerting/fundamentals/notifications/templates.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --- docs/sources/alerting/fundamentals/notifications/templates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md index 688de3a030f58..8d316bc078ae7 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates.md +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -118,7 +118,7 @@ The outcome of this template would print **Notification templates** represent the alternative approach to templating designed for reusing templates. Notifications are messages to inform users about events or conditions triggered by alerts. You can create reusable notification templates to customize the content and format of alert notifications. Variables, labels, or other context-specific details can be added to the templates to dynamically insert information like metric values. -Here is the example of a notification template: +Here is an example of a notification template: ```go { define "alerts.message" -}} From 2fa0153c6bd136b92d3842e6f0f8599db27f7cf8 Mon Sep 17 00:00:00 2001 From: antonio <45235678+tonypowa@users.noreply.github.com> Date: Tue, 30 Apr 2024 11:50:45 +0200 Subject: [PATCH 10/13] Update docs/sources/alerting/fundamentals/notifications/templates.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --- docs/sources/alerting/fundamentals/notifications/templates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md index 8d316bc078ae7..c1806b9b45039 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates.md +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -114,7 +114,7 @@ The outcome of this template would print CPU usage for Instance 1 has exceeded 80% for the last 5 minutes -### Notification templates +### Template notifications **Notification templates** represent the alternative approach to templating designed for reusing templates. Notifications are messages to inform users about events or conditions triggered by alerts. You can create reusable notification templates to customize the content and format of alert notifications. Variables, labels, or other context-specific details can be added to the templates to dynamically insert information like metric values. From 051c3b5f7cd039984e0efb898979cc6fca2a3701 Mon Sep 17 00:00:00 2001 From: antonio <45235678+tonypowa@users.noreply.github.com> Date: Tue, 30 Apr 2024 11:50:54 +0200 Subject: [PATCH 11/13] Update docs/sources/alerting/fundamentals/notifications/templates.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --- docs/sources/alerting/fundamentals/notifications/templates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md index c1806b9b45039..9746c15c99e7a 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates.md +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -116,7 +116,7 @@ The outcome of this template would print ### Template notifications -**Notification templates** represent the alternative approach to templating designed for reusing templates. Notifications are messages to inform users about events or conditions triggered by alerts. You can create reusable notification templates to customize the content and format of alert notifications. Variables, labels, or other context-specific details can be added to the templates to dynamically insert information like metric values. +Notification templates represent the alternative approach to templating designed for reusing templates. Notifications are messages to inform users about events or conditions triggered by alerts. You can create reusable notification templates to customize the content and format of alert notifications. Variables, labels, or other context-specific details can be added to the templates to dynamically insert information like metric values. Here is an example of a notification template: From fe32b9976848a4f228326c3f2a08453cc88387b5 Mon Sep 17 00:00:00 2001 From: antonio <45235678+tonypowa@users.noreply.github.com> Date: Tue, 30 Apr 2024 11:51:00 +0200 Subject: [PATCH 12/13] Update docs/sources/alerting/fundamentals/notifications/templates.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --- docs/sources/alerting/fundamentals/notifications/templates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md index 9746c15c99e7a..d081f07040994 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates.md +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -59,7 +59,7 @@ Labels and annotations contain information about an alert. Labels are used to di Label templates are applied in the alert rule itself (i.e. in the Configure labels and notifications section of an alert). {{}} -Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that, it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it might cause Grafana to create many alerts when you actually only need one. +Think about templating labels when you need to improve or change how alerts are uniquely identified. This is especially helpful if the labels you get from your query aren't detailed enough. Keep in mind that it's better to keep long sentences for summaries and descriptions. Also, avoid using the query's value in labels because it may result in the creation of many alerts when you actually only need one. {{}} Templating can be applied by using variables and functions. These variables can represent dynamic values retrieved from your data queries. From e9e71326b2a6db7f447c262e246f7393f1f52fa5 Mon Sep 17 00:00:00 2001 From: tonypowa Date: Tue, 30 Apr 2024 12:00:48 +0200 Subject: [PATCH 13/13] amended admonition --- docs/sources/alerting/fundamentals/notifications/templates.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sources/alerting/fundamentals/notifications/templates.md b/docs/sources/alerting/fundamentals/notifications/templates.md index d081f07040994..dacd9a5310ff3 100644 --- a/docs/sources/alerting/fundamentals/notifications/templates.md +++ b/docs/sources/alerting/fundamentals/notifications/templates.md @@ -149,9 +149,9 @@ This is the message you would receive in your contact point: Once the template is created, you need to make reference to it in your **Contact point** (in the Optional [contact point] settings) . -{{ }} +{{}} It's not recommended to include individual alert information within notification templates. Instead, it's more effective to incorporate such details within the rule using labels and annotations. -{{ }} +{{}} {{% docs/reference %}} [variables-label-annotation]: "/docs/grafana/ -> /docs/grafana//alerting/alerting-rules/templating-labels-annotations"