Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable protobuf query result payload format by default for rule evaluation path. #4709

Merged
merged 2 commits into from
Apr 12, 2023
Merged

Enable protobuf query result payload format by default for rule evaluation path. #4709

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1289791
Enable protobuf query result payload format by default for rule evalu…
charleskorn Apr 12, 2023
5a73041
Add changelog entry and update experimental features list in docs.
charleskorn Apr 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
* [CHANGE] Ingester: the configuration parameter `-blocks-storage.tsdb.max-tsdb-opening-concurrency-on-startup` has been deprecated and will be removed in Mimir 2.10. #4445
* [CHANGE] Query-frontend: Cached results now contain timestamp which allows Mimir to check if cached results are still valid based on current TTL configured for tenant. Results cached by previous Mimir version are used until they expire from cache, which can take up to 7 days. If you need to use per-tenant TTL sooner, please flush results cache manually. #4439
* [CHANGE] Ingester: the `cortex_ingester_tsdb_wal_replay_duration_seconds` metrics has been removed. #4465
* [CHANGE] Query-frontend: use protobuf internal query result payload format by default. This feature is no longer considered experimental. #4557
* [CHANGE] Query-frontend and ruler: use protobuf internal query result payload format by default. This feature is no longer considered experimental. #4557 #4709
* [CHANGE] Ruler: reject creating federated rule groups while tenant federation is disabled. Previously the rule groups would be silently dropped during bucket sync. #4555
* [CHANGE] Compactor: the `/api/v1/upload/block/{block}/finish` endpoint now returns a `429` status code when the compactor has reached the limit specified by `-compactor.max-block-upload-validation-concurrency`. #4598
* [CHANGE] Store-gateway: cache key format for expanded postings has changed. This will invalidate the expanded postings in the index cache when deployed. #4667
Expand Down
5 changes: 2 additions & 3 deletions cmd/mimir/config-descriptor.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10087,10 +10087,9 @@
"required": false,
"desc": "Format to use when retrieving query results from query-frontends. Supported values: json, protobuf",
"fieldValue": null,
"fieldDefaultValue": "json",
"fieldDefaultValue": "protobuf",
"fieldFlag": "ruler.query-frontend.query-result-response-format",
"fieldType": "string",
"fieldCategory": "experimental"
"fieldType": "string"
}
],
"fieldValue": null,
Expand Down
2 changes: 1 addition & 1 deletion cmd/mimir/help-all.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2116,7 +2116,7 @@ Usage of ./cmd/mimir/mimir:
-ruler.query-frontend.grpc-client-config.tls-server-name string
Override the expected name on the server certificate.
-ruler.query-frontend.query-result-response-format string
[experimental] Format to use when retrieving query results from query-frontends. Supported values: json, protobuf (default "json")
Format to use when retrieving query results from query-frontends. Supported values: json, protobuf (default "protobuf")
-ruler.query-stats-enabled
Report the wall time for ruler queries to complete as a per-tenant metric and as an info level log message.
-ruler.recording-rules-evaluation-enabled
Expand Down
2 changes: 2 additions & 0 deletions cmd/mimir/help.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -583,6 +583,8 @@ Usage of ./cmd/mimir/mimir:
Maximum number of rules per rule group per-tenant. 0 to disable. (default 20)
-ruler.query-frontend.address string
GRPC listen address of the query-frontend(s). Must be a DNS address (prefixed with dns:///) to enable client side load balancing.
-ruler.query-frontend.query-result-response-format string
Format to use when retrieving query results from query-frontends. Supported values: json, protobuf (default "protobuf")
-ruler.ring.consul.hostname string
Hostname and port of Consul. (default "localhost:8500")
-ruler.ring.etcd.endpoints string
Expand Down
2 changes: 0 additions & 2 deletions docs/sources/mimir/configure/about-versioning.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,6 @@ The following features are currently experimental:
- `-max-separate-metrics-groups-per-user`
- Overrides-exporter
- Peer discovery / tenant sharding for overrides exporters (`-overrides-exporter.ring.enabled`)
- Protobuf internal query result payload format for rule evaluation (`-ruler.query-frontend.query-result-response-format=protobuf`)
- Note that using the protobuf format for the query path (`-query-frontend.query-result-response-format=protobuf`) is not considered experimental
- Per-tenant Results cache TTL (`-query-frontend.results-cache-ttl`, `-query-frontend.results-cache-ttl-for-out-of-order-time-window`)
- Fetching TLS secrets from Vault for various clients (`-vault.enabled`)

Expand Down
6 changes: 3 additions & 3 deletions docs/sources/mimir/references/configuration-parameters/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1590,10 +1590,10 @@ query_frontend:
# ruler.query-frontend.grpc-client-config
[grpc_client_config: <grpc_client>]

# (experimental) Format to use when retrieving query results from
# query-frontends. Supported values: json, protobuf
# Format to use when retrieving query results from query-frontends. Supported
# values: json, protobuf
# CLI flag: -ruler.query-frontend.query-result-response-format
[query_result_response_format: <string> | default = "json"]
[query_result_response_format: <string> | default = "protobuf"]

tenant_federation:
# Enable rule groups to query against multiple tenants. The tenant IDs
Expand Down
4 changes: 2 additions & 2 deletions pkg/ruler/remotequerier.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ type QueryFrontendConfig struct {
// GRPCClientConfig contains gRPC specific config options.
GRPCClientConfig grpcclient.Config `yaml:"grpc_client_config" doc:"description=Configures the gRPC client used to communicate between the rulers and query-frontends."`

QueryResultResponseFormat string `yaml:"query_result_response_format" category:"experimental"`
QueryResultResponseFormat string `yaml:"query_result_response_format"`
}

func (c *QueryFrontendConfig) RegisterFlags(f *flag.FlagSet) {
Expand All @@ -73,7 +73,7 @@ func (c *QueryFrontendConfig) RegisterFlags(f *flag.FlagSet) {

c.GRPCClientConfig.RegisterFlagsWithPrefix("ruler.query-frontend.grpc-client-config", f)

f.StringVar(&c.QueryResultResponseFormat, "ruler.query-frontend.query-result-response-format", formatJSON, fmt.Sprintf("Format to use when retrieving query results from query-frontends. Supported values: %s", strings.Join(allFormats, ", ")))
f.StringVar(&c.QueryResultResponseFormat, "ruler.query-frontend.query-result-response-format", formatProtobuf, fmt.Sprintf("Format to use when retrieving query results from query-frontends. Supported values: %s", strings.Join(allFormats, ", ")))
}

func (c *QueryFrontendConfig) Validate() error {
Expand Down