Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add celery worker cpu request #226

Closed
wants to merge 1 commit into from
Closed

Add celery worker cpu request #226

wants to merge 1 commit into from

Conversation

yakkomajuri
Copy link
Contributor

Description

Friendly reminder that I'm way out of my depth with Helm charts.

Essentially, as part of the special migrations work, I'd like to bump Celery specs a bit.

There are 2 ways to do this:

  • Set min replicas to 2
  • Bump CPU request

Here I'm going with a CPU request bump as Celery will run n worker processes where n == n cores. Our default deployments seem to be giving us 2 cores, and adding an extra one ensures Celery can have the same throughput while we hijack a worker for special migrations.

I guess this is a breaking change? Maybe the replicas approach is better?

Just sparking discussion here. I'd rather have a sensible default than write on the release notes that people should bump celery replicas on upgrades.

Type of change

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

How has this been tested?

It hasn't

Checklist

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

@guidoiaquinti
Copy link
Contributor

👋 Hey Yakko, welcome back in this repo!

I'm 100% into adding resource limits (see #159) but as we didn't do this before we should consider this a breaking change. See also the considerations already done in the other PR.

@tiina303 tiina303 marked this pull request as draft February 10, 2022 17:26
@yakkomajuri
Copy link
Contributor Author

@guidoiaquinti what would we need to get this over the line? can we batch this with the next breaking release?

@guidoiaquinti
Copy link
Contributor

@guidoiaquinti what would we need to get this over the line? can we batch this with the next breaking release?

I think we should make this change as part of #159.

@hazzadous
Copy link
Contributor

I going to close this to reduce the noise in this repo, if it's really important feel free to open again.

@hazzadous hazzadous closed this Jun 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants