Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 2.3 to main #2995

Merged
merged 19 commits into from
Sep 21, 2022
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 39 additions & 38 deletions CHANGELOG.md

Large diffs are not rendered by default.

36 changes: 1 addition & 35 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,7 @@ mixin-serve: ## Runs Grafana loading the mixin dashboards compiled at operations
@./operations/mimir-mixin-tools/serve/run.sh

mixin-screenshots: ## Generates mixin dashboards screenshots.
@find docs/sources/operators-guide/monitoring-grafana-mimir/dashboards -name '*.png' -delete
@find docs/sources/operators-guide/monitor-grafana-mimir/dashboards -name '*.png' -delete
@./operations/mimir-mixin-tools/screenshots/run.sh

check-jsonnet-manifests: ## Check the jsonnet manifests.
Expand Down Expand Up @@ -580,40 +580,6 @@ integration-tests: cmd/mimir/$(UPTODATE)
web-serve:
cd website && hugo --config config.toml --minify -v server

# Generate binaries for a Mimir release
dist: dist/$(UPTODATE)

dist/$(UPTODATE):
rm -fr ./dist
mkdir -p ./dist
# Build binaries for various architectures and operating systems. Only
# mimirtool supports Windows for now. Also darwin/386 is not a valid
# architecture.
for os in linux darwin windows; do \
for arch in 386 amd64 arm64; do \
suffix="" ; \
if [ "$$os" = "windows" ]; then \
suffix=".exe" ; \
fi; \
if [ "$$os" = "darwin" ] && [ "$$arch" = "386" ]; then \
continue; \
fi; \
echo "Building mimirtool for $$os/$$arch"; \
GOOS=$$os GOARCH=$$arch CGO_ENABLED=0 go build $(GO_FLAGS) -o ./dist/mimirtool-$$os-$$arch$$suffix ./cmd/mimirtool; \
sha256sum ./dist/mimirtool-$$os-$$arch$$suffix | cut -d ' ' -f 1 > ./dist/mimirtool-$$os-$$arch$$suffix-sha-256; \
if [ "$$os" = "windows" ]; then \
continue; \
fi; \
echo "Building Mimir for $$os/$$arch"; \
GOOS=$$os GOARCH=$$arch CGO_ENABLED=0 go build $(GO_FLAGS) -o ./dist/mimir-$$os-$$arch$$suffix ./cmd/mimir; \
sha256sum ./dist/mimir-$$os-$$arch$$suffix | cut -d ' ' -f 1 > ./dist/mimir-$$os-$$arch$$suffix-sha-256; \
echo "Building query-tee for $$os/$$arch"; \
GOOS=$$os GOARCH=$$arch CGO_ENABLED=0 go build $(GO_FLAGS) -o ./dist/query-tee-$$os-$$arch$$suffix ./cmd/query-tee; \
sha256sum ./dist/query-tee-$$os-$$arch$$suffix | cut -d ' ' -f 1 > ./dist/query-tee-$$os-$$arch$$suffix-sha-256; \
done; \
done; \
touch $@

# Generate packages for a Mimir release.
FPM_OPTS := fpm -s dir -v $(VERSION) -n mimir -f \
--license "AGPL 3.0" \
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 30 additions & 19 deletions docs/sources/release-notes/v2.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,46 +9,57 @@ weight: 300

Grafana Labs is excited to announce version 2.3 of Grafana Mimir, the most scalable, most performant open source time series database in the world.

The highlights that follow include the top features, enhancements, and bugfixes in this release. If you are upgrading from Grafana Mimir 2.2, there is upgrade-related information as well.
For the complete list of changes, see the [Changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md).
The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the [changelog](https://github.com/grafana/mimir/blob/main/CHANGELOG.md).

**Note**: If you are upgrading from Grafana Mimir 2.2, review the [list of important changes](#important-changes) that follow.

## Features and enhancements

- **Ingest metrics in OpenTelemetry format**:
This release of Grafana Mimir introduces experimental support for ingesting metrics from the [OpenTelemetry Collector's](https://opentelemetry.io/docs/collector/) [`otlphttp` exporter](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/otlphttpexporter/README.md). This adds a second ingestion option for users of the OTel Collector; Mimir was already compatible with the [`prometheusremotewrite` exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/prometheusremotewriteexporter). For more information, please see [Configure OTel Collector]({{< relref "../operators-guide/configure/configure-otel-collector.md">}}).

- **Increased instant query performance**:
Grafana Mimir now supports splitting instant queries by time. This allows it to better parallelize execution of instant queries and therefore return results faster. At present, splitting is only supported for a subset of instant queries, which means not all instant queries will see a speedup. This feature is being released as experimental and is disabled by default. It can be enabled by setting `-query-frontend.split-instant-queries-by-interval`.

- **Tenant federation for metadata queries**:
Users with [tenant federation enabled]({{< relref "../operators-guide/secure/authentication-and-authorization/#grafana-mimir-authentication-and-authorization">}}) could previously issue instant queries, range queries, and exemplar queries to multiple tenants at once and receive a single aggregated result. With Grafana Mimir 2.3, we've added tenant federation support to the `/api/v1/metadata` endpoint as well.
Users with [tenant federation enabled]({{< relref "../operators-guide/secure/authentication-and-authorization/#grafana-mimir-authentication-and-authorization">}}) could already issue instant queries, range queries, and exemplar queries to multiple tenants at once and receive a single aggregated result. With Grafana Mimir 2.3, we've added tenant federation support to the `/api/v1/metadata` endpoint as well.

- **Simpler object storage configuration**:
Users can now configure block, alertmanager, and ruler storage all at once with the `common` YAML config option key (or `-common.storage.*` CLI flags). By centralizing your object storage configuration in one place, this enhancement makes configuration faster and less error prone. Users can still individually configure storage for each of these components if they desire. For more information, see the [Common Configurations]({{< relref "../operators-guide/configure/about-configurations/#common-configurations">}}).
Users can now configure block, alertmanager, and ruler storage all at once with the `common` YAML config option key (or `-common.storage.*` CLI flags). By centralizing your object storage configuration in one place, this enhancement makes configuration faster and less error prone. Users may still individually configure storage for each of these components if they desire. For more information, see the [Common Configurations]({{< relref "../operators-guide/configure/about-configurations/#common-configurations">}}).

- **DEB and RPM packages for Mimir**:
Starting with version 2.3, we're publishing deb and rpm files for Grafana Mimir, which will make installing and running it on Debian or RedHat-based linux systems much easier. Thank you to community contributor [wilfriedroset](https://github.com/wilfriedroset) for your work to implement this!
- **.deb and .rpm packages for Mimir**:
Starting with version 2.3, we're publishing .deb and .rpm files for Grafana Mimir, which will make installing and running it on Debian or RedHat-based linux systems much easier. Thank you to community contributor [wilfriedroset](https://github.com/wilfriedroset) for your work to implement this!

- **Import historic data to Grafana Mimir**:
Users can now backfill time series data from their existing Prometheus or Cortex installation into Mimir using `mimirtool`, making it possible to migrate to Grafana Mimir without losing your existing metrics data. This support is still considered experimental and does not work for data stored in Thanos yet. To learn more about this feature, please see [mimirtool backfill]({{< relref "../operators-guide/tools/mimirtool/#backfill">}}) and [Configure TSDB block upload]({{< relref "../operators-guide/configure/configure-tsdb-block-upload/">}})
- **Import historic data**:
Users can now backfill time series data from their existing Prometheus or Cortex installation into Mimir using `mimirtool`, making it possible to migrate to Grafana Mimir without losing your existing metrics data. This support is still considered experimental and does not yet work for data stored in Thanos. To learn more about this feature, please see [`mimirtool backfill`]({{< relref "../operators-guide/tools/mimirtool/#backfill">}}) and [Configure TSDB block upload]({{< relref "../operators-guide/configure/configure-tsdb-block-upload/">}})

- **Increased instant query performance**:
Grafana Mimir now supports splitting instant queries by time. This allows it to better parallelize execution of instant queries and therefore return results faster. At present, splitting is only supported for a subset of instant queries, which means not all instant queries will see a speedup. This feature is currently experimental and is disabled by default. It can be enabled with the `split_instant_queries_by_interval` YAML config option in the `limits` section (or the CLI flag `-query-frontend.split-instant-queries-by-interval`).

- **New Helm chart minor release**: The Mimir Helm chart is the best way to install Mimir on Kubernetes. As part of the Mimir 2.3 release, we’re also releasing version 3.1 of the Mimir Helm chart. Notable enhancements follow. For the full list of changes, see the [Helm chart changelog](https://github.com/grafana/mimir/tree/main/operations/helm/charts/mimir-distributed/CHANGELOG.md).
- We've upgraded the MinIO subchart dependency from a [deprecated chart](https://helm.min.io/) to the [supported one](https://charts.min.io/). This creates a **breaking change** in how the administrator password is set. However, as the built-in MinIO is not a recommended object store for production use cases, this change did not warrant a new major version of the Mimir Helm chart.
- The backfill API endpoints for importing historic time series data are now exposed on the Nginx gateway.
- Nginx now sets the value of the `X-Scope-OrgID` header equal to the value of Mimir's `no_auth_tenant` parameter by default. The previous release had set the value of `X-Scope-OrgID` to `anonymous` by default which [complicated the process of migrating to Mimir](https://github.com/grafana/mimir/issues/2563).
- Memberlist now uses DNS service-discovery by default, which should decrease startup time for large Mimir clusters.
### Helm chart improvements

## Upgrade considerations
The Mimir Helm chart is the best way to install Mimir on Kubernetes. As part of the Mimir 2.3 release, we’re also releasing version 3.1 of the Mimir Helm chart.

Notable enhancements follow. For the full list of changes, see the [Helm chart changelog](https://github.com/grafana/mimir/tree/main/operations/helm/charts/mimir-distributed/CHANGELOG.md).

- We've upgraded the MinIO subchart dependency from a [deprecated chart](https://helm.min.io/) to the [supported one](https://charts.min.io/). This creates a **breaking change** in how the administrator password is set. However, as the built-in MinIO is not a recommended object store for production use cases, this change did not warrant a new major version of the Mimir Helm chart.
- [Query sharding](https://grafana.com/docs/mimir/latest/operators-guide/architecture/query-sharding/) is now enabled by default which should give you better performance on high cardinality metrics queries.
- To compensate for the increased number of queries generated by query sharding, the [query scheduler](https://grafana.com/docs/mimir/latest/operators-guide/architecture/components/query-scheduler/) component is now enabled by default.
- The backfill API endpoints for importing historic time series data are now exposed on the Nginx gateway.
- Nginx now sets the value of the `X-Scope-OrgID` header equal to the value of Mimir's `no_auth_tenant` parameter by default. The previous release had set the value of `X-Scope-OrgID` to `anonymous` by default which [complicated the process of migrating to Mimir](https://github.com/grafana/mimir/issues/2563).
- Memberlist now uses DNS service-discovery by default, which decreases startup time for large Mimir clusters.

## Important changes

In Grafana Mimir 2.3 we have removed the following previously deprecated configuration options:

- The `extend_writes` parameter in the distributor YAML configuration and `-distributor.extend-writes` CLI flag have been removed.
- The `active_series_custom_trackers` parameter has been removed from the YAML configuration. It had already been moved to the runtime configuration. See [#1188](https://github.com/grafana/mimir/pull/1188) for details.
- The `blocks-storage.tsdb.isolation-enabled` parameter in the YAML configuration and `-blocks-storage.tsdb.isolation-enabled` CLI flag have been removed.

With Grafana Mimir 2.3 we have also updated the default value for the CLI flag `-distributor.ha-tracker.max-clusters` to `100` to provide Denial-of-Service protection. Previously `-distributor.ha-tracker.max-clusters` was unlimited by default which could allow a tenant with HA Dedupe enabled to overload the HA tracker with `__cluster__` label values that could cause the HA Dedupe database to fail.

With Grafana Mimir 2.3 we have also updated the default value for `-distributor.ha-tracker.max-clusters` to `100` to provide Denial-of-Service protection. Previously `-distributor.ha-tracker.max-clusters` was unlimited by default which could allow a tenant with HA Dedupe enabled to overload the HA tracker with `__cluster__` label values that could cause the HA Dedupe database to fail.
Also, as [noted above](#helm-chart-improvements), the administrator password for Helm chart deployments using the built-in MinIO is now set differently.

## Bug fixes

- [PR 2447](https://github.com/grafana/mimir/pull/2447): Fix incorrect mapping of http status codes `429` to `500` when the request queue is full in the query-frontend. This corrects behavior in the query-frontend where a `429 "Too Many Outstanding Requests"` error (a retriable error) from a querier was incorrectly returned as a `500` system error (an unretriable error).
- [PR 2447](https://github.com/grafana/mimir/pull/2447): Fix incorrect mapping of http status codes `429` to `500` when the request queue is full in the query-frontend. This corrects behavior in the query-frontend where a _retryable_ `429 "Too Many Outstanding Requests"` error from a querier was incorrectly returned as an _unretryable_ `500` system error.
- [PR 2505](https://github.com/grafana/mimir/pull/2505): The Memberlist key-value (KV) store now tries to "fast-join" the cluster to avoid serving an empty KV store. This fix addresses the confusing "empty ring" error response and the error log message "ring doesn't exist in KV store yet" emitted by services when there are other members present in the ring when a service starts. Those using other key-value store options (e.g., consul, etcd) are not impacted by this bug.
- [PR 2289](https://github.com/grafana/mimir/pull/2289): The "List Prometheus rules" API endpoint of the Mimir Ruler component is no longer blocked while rules are being synced. This means users can now list rules while syncing larger rule sets.
1 change: 1 addition & 0 deletions operations/helm/charts/mimir-distributed/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ Entries should include a reference to the Pull Request that introduced the chang
* [ENHANCEMENT] Support autoscaling/v2 HorizontalPodAutoscaler for nginx autoscaling. This is used when deploying on Kubernetes >= 1.25. #2848
* [ENHANCEMENT] Add podAntiAffinity to sizing plans (small.yaml, large.yaml, capped-small.yaml, capped-large.yaml). #2906


## 3.1.0

* [CHANGE] **breaking change** Update minio deprecated helm chart (<https://helm.min.io/>) to the supported chart's version (<https://charts.min.io/>). #2427
Expand Down
2 changes: 1 addition & 1 deletion operations/helm/charts/mimir-distributed/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
version: 3.1.0-weekly.203
appVersion: r203
description: "Grafana Mimir"
home: https://grafana.com/docs/mimir/v2.2.x/
home: https://grafana.com/docs/mimir/v2.3.x/
icon: https://grafana.com/static/img/logos/logo-mimir.svg
kubeVersion: ^1.20.0-0
name: mimir-distributed
Expand Down
Loading