New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define useful celery beat task for development #3762

Merged
merged 5 commits into from Mar 23, 2018

Conversation

Projects
None yet
3 participants
@humitos
Member

humitos commented Mar 10, 2018

Define the task here and remove it from readthedocs-ops.

@humitos humitos requested review from agjohnson and ericholscher Mar 10, 2018

@ericholscher

I like having these as part of the core RTD code, since they do make sense outside of the scale of RTD (unlike the cold storage stuff). We need to make sure all the existing prod configs work well with this though.

orphan_domain_path = os.path.join(domain_path, cname)
log.info('Unlinking orphan CNAME: %s', orphan_domain_path)
os.unlink(orphan_domain_path)

This comment has been minimized.

@ericholscher

ericholscher Mar 12, 2018

Member

I wonder if these should go in another file. projects/tasks is already huge, so it might make sense to move this into core/tasks.py or somewhere else specifically for scheduled tasks?

This comment has been minimized.

@humitos

humitos Mar 12, 2018

Member

Yes. I agree with this.

I created this issue (#3775) to refactor this and include some other tasks also --we can discuss in the issue.

This comment has been minimized.

@agjohnson

agjohnson Mar 15, 2018

Contributor

I actually dislike putting things in core. Why not split up into:

  • readthedocs.projects.tasks.build
  • readthedocs.projects.tasks.sync
  • etc

Some of the tasks might make more sense in builds/ as well. I find core/ is not a helpful concept, save for only the most generalized of code.

@@ -240,6 +243,19 @@ def USE_PROMOS(self): # noqa
CELERY_CREATE_MISSING_QUEUES = True
CELERY_DEFAULT_QUEUE = 'celery'
CELERYBEAT_SCHEDULE = {

This comment has been minimized.

@ericholscher

ericholscher Mar 12, 2018

Member

We need to make sure we aren't overridding this elsewhere. We need a good pattern here, but I'm guessing we aren't extending this everywhere else we use it, so this might get killed by a downstream config file overwriting it.

This comment has been minimized.

@humitos

humitos Mar 12, 2018

Member

I found we use this setting in 3 places:

@ericholscher please confirm this, and I will open a PR for corporate ops to modify this file https://github.com/readthedocs/readthedocs-corporate-ops/blob/master/salt/corporate/readthedocsinc/settings/base.py

This comment has been minimized.

@agjohnson

agjohnson Mar 15, 2018

Contributor

You need a property override there as well, correct.

@humitos humitos referenced this pull request Mar 12, 2018

Open

Refactor tasks files #3775

0 of 8 tasks complete

@humitos humitos self-assigned this Mar 22, 2018

@agjohnson

Changes here look good, I've noted some pieces for our next deploy as well.

Feel free to merge when tests come back

@agjohnson

This comment has been minimized.

Contributor

agjohnson commented Mar 23, 2018

Looks like just a linting error. We can address that separately. Merging!

@agjohnson agjohnson merged commit b242b41 into master Mar 23, 2018

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details

@agjohnson agjohnson deleted the humitos/celerybeat/definetask branch Mar 23, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment