implement batch task sending #52
Conversation
|
|
||
| def test_batch(self): | ||
| """Batch tasks are run""" | ||
| li = [tasks.echo.subtask(args=("foo %i" % i, )) for i in range(2)] |
There was a problem hiding this comment.
While I see the point of having args and kwargs explicitly, it doesn't look as nice as just:
li = [tasks.echo.subtask("foo %i" % i) for i in range(2)]Celery went with this approach, but I am not sure how to fit the host argument in this case.
There was a problem hiding this comment.
If there wasn't a host parameter I would do the same with couldn't come up with better solution :(
There was a problem hiding this comment.
Actually, I have looked into Celery sources and realized that they have implemented it the same way. In fact, they have two implementations:
.signature(args, kwargs)(.subtaskis an alias to.signature).s(*args, **kwargs)(it, essentially, calls.signature(args=args, kwargs=kwargs)) +.si(*args, **kwrags)(they have a special case of "immutable" tasks)
kuyruk/task.py
Outdated
| @@ -194,6 +198,9 @@ def _module_name(self): | |||
| return name | |||
|
|
|||
|
|
|||
| SubTask = namedtuple("SubTask", "task, args, kwargs, host") | |||
There was a problem hiding this comment.
I would use a tuple instead of a string:
SubTask = namedtuple("SubTask", ("task", "args", "kwargs", "host"))Is there any benefit of having them in a comma-separated string instead of a tuple or a list?
There was a problem hiding this comment.
No, they are the same. Just a different notation. Tuple looks better, I will update it.
tasks.py
Outdated
| kuyruk = Kuyruk() | ||
|
|
||
|
|
||
| @kuyruk.task(retry=5, fail_delay=10000, reject_delay=10000) |
There was a problem hiding this comment.
You have dropped fail_delay support recently.
|
Should the |
|
tasks.py and send.py are committed accidentally. I am removing those now. |
| @@ -53,6 +54,9 @@ def __call__(self, *args, **kwargs): | |||
| logger.debug("Task.__call__ args=%r, kwargs=%r", args, kwargs) | |||
| self.send_to_queue(args, kwargs) | |||
|
|
|||
| def subtask(self, args=(), kwargs={}, host=None): | |||
There was a problem hiding this comment.
I usually avoid using mutable default parameters: http://python-guide-pt-br.readthedocs.io/en/latest/writing/gotchas/#mutable-default-arguments
There was a problem hiding this comment.
I am aware of this issue but I am not mutating them inside the function.
|
I am really happy to see this new feature! @cenkalti Thank you! |
|
Thank you @frol. I have uploaded the new version: https://pypi.python.org/pypi/Kuyruk/8.3.0 |
No description provided.