Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define useful celery beat task for development #3762

Merged
merged 5 commits into from Mar 23, 2018

Conversation

@humitos
Copy link
Member

@humitos humitos commented Mar 10, 2018

Define the task here and remove it from readthedocs-ops.

@humitos humitos requested review from agjohnson and ericholscher Mar 10, 2018
Copy link
Member

@ericholscher ericholscher left a comment

I like having these as part of the core RTD code, since they do make sense outside of the scale of RTD (unlike the cold storage stuff). We need to make sure all the existing prod configs work well with this though.

orphan_domain_path = os.path.join(domain_path, cname)
log.info('Unlinking orphan CNAME: %s', orphan_domain_path)
os.unlink(orphan_domain_path)

Copy link
Member

@ericholscher ericholscher Mar 12, 2018

I wonder if these should go in another file. projects/tasks is already huge, so it might make sense to move this into core/tasks.py or somewhere else specifically for scheduled tasks?

Copy link
Member Author

@humitos humitos Mar 12, 2018

Yes. I agree with this.

I created this issue (#3775) to refactor this and include some other tasks also --we can discuss in the issue.

Copy link
Contributor

@agjohnson agjohnson Mar 15, 2018

I actually dislike putting things in core. Why not split up into:

  • readthedocs.projects.tasks.build
  • readthedocs.projects.tasks.sync
  • etc

Some of the tasks might make more sense in builds/ as well. I find core/ is not a helpful concept, save for only the most generalized of code.

@@ -240,6 +243,19 @@ def USE_PROMOS(self): # noqa
CELERY_CREATE_MISSING_QUEUES = True

CELERY_DEFAULT_QUEUE = 'celery'
CELERYBEAT_SCHEDULE = {
Copy link
Member

@ericholscher ericholscher Mar 12, 2018

We need to make sure we aren't overridding this elsewhere. We need a good pattern here, but I'm guessing we aren't extending this everywhere else we use it, so this might get killed by a downstream config file overwriting it.

Copy link
Member Author

@humitos humitos Mar 12, 2018

I found we use this setting in 3 places:

@ericholscher please confirm this, and I will open a PR for corporate ops to modify this file https://github.com/readthedocs/readthedocs-corporate-ops/blob/master/salt/corporate/readthedocsinc/settings/base.py

Copy link
Contributor

@agjohnson agjohnson Mar 15, 2018

You need a property override there as well, correct.

@humitos humitos mentioned this pull request Mar 12, 2018
8 tasks
@humitos humitos self-assigned this Mar 22, 2018
Copy link
Contributor

@agjohnson agjohnson left a comment

Changes here look good, I've noted some pieces for our next deploy as well.

Feel free to merge when tests come back

@agjohnson
Copy link
Contributor

@agjohnson agjohnson commented Mar 23, 2018

Looks like just a linting error. We can address that separately. Merging!

@agjohnson agjohnson merged commit b242b41 into master Mar 23, 2018
1 check failed
@agjohnson agjohnson deleted the humitos/celerybeat/definetask branch Mar 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants