Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support frontend V2 with query scheduler. #4071

Merged
merged 22 commits into from
Aug 6, 2021

Conversation

jeschkies
Copy link
Contributor

@jeschkies jeschkies commented Jul 28, 2021

What this PR does / why we need it:
This enables the configuration of a query scheduler to scale out queries further. The query frontend and querier can be passed a scheduler-address now. The scheduler itself can be the Cortex process.

See https://cortexmetrics.io/docs/operations/scaling-query-frontend/ for more details.

Checklist

  • Documentation added
  • Tests updated (see Docker compose file)

@jeschkies jeschkies changed the title Support Cortex Frontend V2. Support frontend V2 with query scheduler. Jul 28, 2021
@pull-request-size pull-request-size bot added size/M and removed size/S labels Jul 28, 2021
@pull-request-size pull-request-size bot added size/L and removed size/M labels Jul 29, 2021
grpc_client_config:
max_send_msg_size: 1.048576e+08
parallelism: 6
scheduler_address: query-scheduler:9009
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether this is a duplicate of -querier.scheduler-address in the Docker compose file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-querier.scheduler-address takes precedence.

@jeschkies jeschkies requested review from cyriltovena and owen-d and removed request for cyriltovena July 30, 2021 15:23
@pull-request-size pull-request-size bot added size/L and removed size/M labels Aug 2, 2021
@jeschkies jeschkies marked this pull request as ready for review August 2, 2021 13:59
@jeschkies jeschkies requested review from KMiller-Grafana and a team as code owners August 2, 2021 13:59
Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good I think we should also integrate a new module for the query-scheduler so that we only use Loki docker image.

I'd love to see also a follow up PR to update our jsonnet.

@jeschkies
Copy link
Contributor Author

I'd love to see also a follow up PR to update our jsonnet.

@cyriltovena, do you mean for production or for examples?

@cyriltovena
Copy link
Contributor

I'd love to see also a follow up PR to update our jsonnet.

@cyriltovena, do you mean for production or for examples?

In the production folder

Copy link
Contributor

@cyriltovena cyriltovena left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Just one small change is required.

pkg/validation/limits.go Show resolved Hide resolved
@cyriltovena cyriltovena merged commit c3aaebe into grafana:main Aug 6, 2021
@jeschkies jeschkies deleted the karsten/frontendv2 branch August 6, 2021 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants