Skip to content

Commit

Permalink
Query|Receiver: Do not log full request on ProxyStore by default (tha…
Browse files Browse the repository at this point in the history
…nos-io#7305)

* Query|Receiver|Store: Do not log full request on ProxyStore by default

We had a problem on our production where a sudden increase in requests with long matchers was putting our receivers under a lot of pressure.
Upon checking profiles we saw that the problem was calls to Log()

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* Adding changelog

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

---------

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>
  • Loading branch information
pedro-stanaka authored and jnyi committed Jun 3, 2024
1 parent 5a34a49 commit 0a3291f
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ We use *breaking :warning:* to mark changes that are not backward compatible (re
- [#7123](https://github.com/thanos-io/thanos/pull/7123) Rule: Change default Alertmanager API version to v2.
- [#7223](https://github.com/thanos-io/thanos/pull/7223) Automatic detection of memory limits and configure GOMEMLIMIT to match.
- [#7283](https://github.com/thanos-io/thanos/pull/7283) Compact: *breaking :warning:* Replace group with resolution in compact downsample metrics to avoid cardinality explosion with large numbers of groups.
- [#7305](https://github.com/thanos-io/thanos/pull/7305) Query|Receiver: Do not log full request on ProxyStore by default.

### Removed

Expand Down
5 changes: 4 additions & 1 deletion pkg/store/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,10 @@ func (s *ProxyStore) TSDBInfos() []infopb.TSDBInfo {
func (s *ProxyStore) Series(originalRequest *storepb.SeriesRequest, srv storepb.Store_SeriesServer) error {
// TODO(bwplotka): This should be part of request logger, otherwise it does not make much sense. Also, could be
// tiggered by tracing span to reduce cognitive load.
reqLogger := log.With(s.logger, "component", "proxy", "request", originalRequest.String())
reqLogger := log.With(s.logger, "component", "proxy")
if s.debugLogging {
reqLogger = log.With(reqLogger, "request", originalRequest.String())
}

match, matchers, err := matchesExternalLabels(originalRequest.Matchers, s.selectorLabels)
if err != nil {
Expand Down

0 comments on commit 0a3291f

Please sign in to comment.