Make ingesters return gRPC errors only #7151
Conversation
Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>
duricanikolic
force-pushed
the
yuri/grpc-errors
branch
from
f5ee4fa
to
79b193f
Compare
There was a problem hiding this comment.
Thanks for your work on this @duricanikolic :)
Should we also move this flag in about-versioning.md from experimental to deprecated?
Good point. Thank you |
Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>
Signed-off-by: Yuri Nikolic <durica.nikolic@grafana.com>
| @@ -23,7 +23,7 @@ | |||
| * [CHANGE] All: set `-server.report-grpc-codes-in-instrumentation-label-enabled` to `true` by default, which enables reporting gRPC status codes as `status_code` labels in the `cortex_request_duration_seconds` metric. #7144 | |||
| * [CHANGE] Distributor: report gRPC status codes as `status_code` labels in the `cortex_ingester_client_request_duration_seconds` metric by default. #7144 | |||
| * [CHANGE] Distributor: CLI flag `-ingester.client.report-grpc-codes-in-instrumentation-label-enabled` has been deprecated, and its default value is set to `true`. #7144 | |||
| * [CHANGE] Ingester: CLI flag `-ingester.return-only-grpc-errors` has been deprecated, and its default value is set to `true`. #7151 | |||
| * [CHANGE] Ingester: CLI flag `-ingester.return-only-grpc-errors` has been deprecated, and its default value is set to `true`. To ensure backwards compatibility, during a migration from a version prior to 2.11.0 to 2.12 or later, `-ingester.return-only-grpc-errors` should be set to `false`. Once all the components are migrated, the flag can be removed. #7151 | |||
There was a problem hiding this comment.
Since the side effects aren't that bad (misinterpreted error codes only, right?), should we mention the tradeoff of not doing a proper migration too? I think this might help some users just decide to not do a migration.
This may be on the longer side for a changelog entry, but i think that it will be useful information in the release notes and we have the context now, so might as well do it now.
What this PR does
This PR deprecates the
-ingester.ingester.return-only-grpc-errorsCLI flag, and sets its default value totrue. This CLI flag will be removed in Mimir 2.14.0, but the default behaviour of Mimir will be as the CLI flag were set totrue.As a direct consequence of this PR, ingesters will always return only gRPC errors. The support for HTTP status codes is still there for backwards compatibility, but will be removed in Mimir 2.14.0, when the
-ingester.ingester.return-only-grpc-errorsCLI flag is removed.Remark about backwards compatibility: if migrating from a version prior to 2.11.0 to a version 2.12.0 or later,
-ingester.ingester.return-only-grpc-errorsshould be set tofalse. Once both distributor and ingester are running on a version 2.12.0 or later, the flag can be removed, so that the Mimir's default behaviour will be used.Which issue(s) this PR fixes or relates to
Part of #6008
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX].about-versioning.mdupdated with experimental features.