Add basic support for lower priority PR builds

[ericholscher]

To test this I added this to our tasks:

@app.task(ignore_result=True)
def priority_test(priority):
    import time
    time.sleep(1)
    print(priority)

Then ran this script in a manage.py shell_plus:

from readthedocs.projects.tasks import priority_test

for i in range(50):
    priority_test.apply_async((i%10,), queue="default", priority=i%10)

from redis import Redis
r = Redis.from_url(settings.BROKER_URL)
for k in r.keys('default*'):
    print(k.decode())
    print(r.llen(k))
    #print(r.delete(k))

[humitos]

The example shows it works locally. It puts 20 tasks on each queue (it shows 19 because I suppose 2 are consumed immediately by the workers and they are gone when it checks redis). After that, all the tasks with Priority 1 are consumed first and then all the tasks with priority 0.

[ericholscher]

@humitos I cleaned up the logic around it now that we have it working.

[humitos]

Happy to have this working! I like the idea of having 3 priorities.

I'm a little confused about how this works (I may need to read more the documentation :) ), but in the meanwhile: do you know if these priorities apply to all the queues by default? I mean, will we end up with N queues * 3? Will workers automatically listen on those queues or do we need to update them to listen in build:default{1,2,3} for example? (which I haven't in my local test, BTW)

[humitos]

nit: maybe these fit better in builds.contants?

[humitos]

Want we check for being a translation here before checking if external and assigning CELERY_MEDIUM?

[humitos]

We should modify/add a test that triggers a PR and check it's CELERY_LOW.

[ericholscher]

Addressed review feedback, going to ship this and QA this week 👍