Skip to content

Commit

Permalink
docs: move federated rule groups documentation to its own section
Browse files Browse the repository at this point in the history
Signed-off-by: Miguel Ángel Ortuño <ortuman@gmail.com>
  • Loading branch information
ortuman committed May 30, 2022
1 parent 3ace949 commit 55eb13c
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 49 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,35 @@ The ruler then notifies Alertmanagers of any **FIRING** (`firing`) alerts.
Configure the addresses of Alertmanagers with the `-ruler.alertmanager-url` flag, which supports the DNS service discovery format.
For more information about DNS service discovery, refer to [Supported discovery modes]({{< relref "../../../configuring/about-dns-service-discovery.md" >}}).

## Federated rule groups

A federated rule group is a rule group with a non-empty `source_tenants`.

The `source_tenants` field allows aggregating data from multiple tenants while evaluating a rule group. The expressions
of each rule in the group will be evaluated against the data of all tenants in `source_tenants`. If `source_tenants` is
empty or omitted, then the tenant under which the group is created will be treated as the `source_tenant`.

Federated rule groups are skipped during evaluation by default. This feature depends on
the cross-tenant query federation feature. To enable federated rules
set `-ruler.tenant-federation.enabled=true` and `-tenant-federation.enabled=true` CLI flags (or their respective YAML
config options).

During evaluation query limits applied to single tenants are also applied to each query in the rule group. For example,
if `tenant-a` has a federated rule group with `source_tenants: [tenant-b, tenant-c]`, then query limits for `tenant-b`
and `tenant-c` will be applied. If any of these limits is exceeded, the whole evaluation will fail. No partial results
will be saved. The same "no partial results" guarantee applies to queries failing for other reasons (e.g. ingester
unavailability).

The time series used during evaluation of federated rules will have the `__tenant_id__` label, similar to how it is
present on series returned with cross-tenant query federation.

> **Note**: Federated rule groups allow data from multiple source tenants to be written into a single
> destination tenant. This makes the existing separation of tenants' data less clear. For example, `tenant-a` has a
> federated rule group that aggregates over `tenant-b`'s data (e.g. `sum(metric_b)`) and writes the result back
> into `tenant-a`'s storage (e.g. as metric `sum:metric_b`). Now part of `tenant-b`'s data is copied to `tenant-a` (albeit
> aggregated). Have this in mind when configuring the access control layer in front of mimir and when enabling federated
> rules via `-ruler.tenant-federation.enabled`.
## Sharding

The ruler supports multi-tenancy and horizontal scalability.
Expand Down
48 changes: 0 additions & 48 deletions docs/sources/operators-guide/reference-http-api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -705,54 +705,6 @@ rules:
severity: warning
```
#### Federated rule groups
A federated rule groups is a rule group with a non-empty `source_tenants`.

The `source_tenants` field allows aggregating data from multiple tenants while evaluating a rule group. The expressions
of each rule in the group will be evaluated against the data of all tenants in `source_tenants`. If `source_tenants` is
empty or omitted, then the tenant under which the group is created will be treated as the `source_tenant`.

Federated rule groups are skipped during evaluation by default. This feature depends on
the cross-tenant query federation feature. To enable federated rules
set `-ruler.tenant-federation.enabled=true` and `-tenant-federation.enabled=true` CLI flags (or their respective YAML
config options).

During evaluation query limits applied to single tenants are also applied to each query in the rule group. For example,
if `tenant-a` has a federated rule group with `source_tenants: [tenant-b, tenant-c]`, then query limits for `tenant-b`
and `tenant-c` will be applied. If any of these limits is exceeded, the whole evaluation will fail. No partial results
will be saved. The same "no partial results" guarantee applies to queries failing for other reasons (e.g. ingester
unavailability).

The time series used during evaluation of federated rules will have the `__tenant_id__` label, similar to how it is
present on series returned with cross-tenant query federation.

**Considerations:** Federated rule groups allow data from multiple source tenants to be written into a single
destination tenant. This makes the existing separation of tenants' data less clear. For example, `tenant-a` has a
federated rule group that aggregates over `tenant-b`'s data (e.g. `sum(metric_b)`) and writes the result back
into `tenant-a`'s storage (e.g. as metric `sum:metric_b`). Now part of `tenant-b`'s data is copied to `tenant-a` (albeit
aggregated). Have this in mind when configuring the access control layer in front of mimir and when enabling federated
rules via `-ruler.tenant-federation.enabled`.

#### Example "federated rules group" request body

```yaml
name: <string>
interval: <duration;optional>
source_tenants:
- <string>
rules:
- record: <string>
expr: <string>
- alert: <string>
expr: <string>
for: <duration>
annotations:
<annotation_name>: <string>
labels:
<label_name>: <string>
```

### Delete rule group
```
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/release-notes/v2.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ These features and enhancements distinguish Grafana Mimir from Cortex 1.10.0:

- **Query sharding for improved query speeds**: Grafana Mimir introduces query sharding to accelerate the execution of high-cardinality or CPU-intensive queries. Query sharding distributes the execution of a single query across multiple machines, to significantly reduce query execution time. We have seen speedups of 10 to 30x in our Grafana Cloud Metrics clusters. Refer to [query sharding]({{< relref "../operators-guide/architecture/query-sharding/index.md" >}}) to learn more.

- **Federated rule groups**: Grafana Mimir makes it possible to write alerting and recording rules that use metrics data from multiple tenants. For example, a user can now create a recording rule that adds metricA from tenantA to metricB from tenantB and writes the result to tenantC. This feature is experimental. For more information on how to use it, refer to [federated rule groups]({{< relref "../operators-guide/reference-http-api/index.md#federated-rule-groups" >}}).
- **Federated rule groups**: Grafana Mimir makes it possible to write alerting and recording rules that use metrics data from multiple tenants. For example, a user can now create a recording rule that adds metricA from tenantA to metricB from tenantB and writes the result to tenantC. This feature is experimental. For more information on how to use it, refer to [federated rule groups]({{< relref "../operators-guide/architecture/components/ruler/index.md#federated-rule-groups" >}}).

- **Understand your metrics cardinality**: Grafana Mimir adds two API endpoints to help users identify high-cardinality metrics. The [`label_names` endpoint]({{< relref "../operators-guide/reference-http-api/index.md#label-names-cardinality" >}}) takes a metric name and returns all label names applied to that metric, as well as the count of values for each label name. When run without a metric name, it returns the highest-cardinality label names. The [`label_values` endpoint]({{< relref "../operators-guide/reference-http-api/index.md#label-values-cardinality" >}}) returns the highest-cardinality metrics, and can be used to get a count of how many series have a given label-value pair applied. The new [custom tracker feature]({{< relref "../operators-guide/configuring/configuring-custom-trackers.md" >}}) takes this one step further by allowing you to track the count of active series over time that match a specific label matcher.

Expand Down

0 comments on commit 55eb13c

Please sign in to comment.