Skip to content

Commit

Permalink
Merge pull request #13 from toudi/support_for_lambda_job_name
Browse files Browse the repository at this point in the history
Support for lambda GEARMAN_JOB_NAME.
  • Loading branch information
Fred Wenzel committed Feb 1, 2013
2 parents a8b05f0 + 9b917d2 commit 6081442
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 2 deletions.
34 changes: 34 additions & 0 deletions README.md
Expand Up @@ -46,6 +46,40 @@ the task name by specifying `name` parameter of the decorator. Here's how:
def my_task_function(foo):
pass
### ``GEARMAN_JOB_NAME`` method
GEARMAN_JOB_NAME is the lambda function which takes original task name as
an argument, and returns altered version of the task name. Very important thing
to note here is that this changes the internal naming only! So you submit your job
like you would normally do, only it has a different internal name.

The default behaviour of this method is as follows:

new_task_name = crc32(getcwd()) + '.' + task_name

This may seem like a very strange idea, but it is there for a reason. Namely, the
task name uniqueness.

Let's imagine you have two instances of the same project on the same machine.
Typical situation would be one instance being the test/development instance and
the other one being the production one. So naturally you submit a job like you normally
would do:

client.submit_job('send_mail', ...)

Only now, gearman wouldn't know which worker should process the task. Should it be
the production one? Or the testing? hash of getcwd() solves this issue, because the
production environment is located in different location than the testing, and your
call is transparently renamed to

client.submit_job('some-hash.send_mail', ...)

If you would like to change this behaviour, simply define GEARMAN_JOB_NAME function
in the settings:

GEARMAN_JOB_NAME = lambda name: name

which would leave the internal task name unchanged.

### Task parameters
The gearman docs specify that the job function can accept only one parameter
(usually refered to as the ``data`` parameter). Additionally, that parameter
Expand Down
19 changes: 17 additions & 2 deletions django_gearman/models.py
@@ -1,9 +1,16 @@
import pickle

from zlib import adler32
from os import getcwd
import gearman
from django.conf import settings


def default_taskname_decorator(task_name):
return "%s.%s" % (str(adler32(getcwd()) & 0xffffffff), task_name)

task_name_decorator = getattr(settings, 'GEARMAN_JOB_NAME', default_taskname_decorator)


class PickleDataEncoder(gearman.DataEncoder):
@classmethod
def encode(cls, encodable_object):
Expand Down Expand Up @@ -53,11 +60,14 @@ def parse_data(self, arg, args=None, kwargs=None, *arguments, **karguments):
def submit_job(
self, task, orig_data = None, unique=None, priority=None,
background=False, wait_until_complete=True, max_retries=0,
poll_timeout=None, args=None, kwargs=None, *arguments, **karguments):
poll_timeout=None, args=None, kwargs=None, *arguments, **karguments):
"""
Handle *args and **kwargs before passing it on to GearmanClient's
submit_job function.
"""
if callable(task_name_decorator):
task = task_name_decorator(task)

data = self.parse_data(orig_data, args, kwargs, *arguments, **karguments)

return super(DjangoGearmanClient, self).submit_job(
Expand Down Expand Up @@ -91,3 +101,8 @@ def __init__(self, **kwargs):
"""Instantiate Gearman worker with servers from settings file."""
return super(DjangoGearmanWorker, self).__init__(
settings.GEARMAN_SERVERS, **kwargs)

def register_task(self, task_name, task):
if callable(task_name_decorator):
task_name = task_name_decorator(task_name)
return super(DjangoGearmanWorker, self).register_task(task_name, task)

0 comments on commit 6081442

Please sign in to comment.