Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Custom key generation function #91

Open
adimux opened this issue Apr 15, 2019 · 2 comments
Open

Suggestion: Custom key generation function #91

adimux opened this issue Apr 15, 2019 · 2 comments

Comments

@adimux
Copy link

adimux commented Apr 15, 2019

Hi,
First of all, thanks for creating this library!

Related to the key collisions issue #19, I was wondering if passing a function that generates the key from the task's arguments as an alternative to specifying the arguments would be a good solution?

In my case, I need to add the tenant's name somewhere in the key because I want the tasks to only be locked for the same tenant.

The solution proposed seems to address this kind of issues in a generic way.

@adimux adimux changed the title Custom key generation Custom key generation function Apr 15, 2019
@cameronmaske
Copy link
Owner

cameronmaske commented Apr 15, 2019

Hi @adimux

I can see the value in something like this.
It seems like this could fit nicely here and use a user provided function instead of queue_once_key.

def generate_key(name, call_args, once_settings):
   return "bla"


@celery.task(base=QueueOnce, once={'key_func': generate_key})
def example_task():
    ....

I'm don't think key_func is the best name, but I'd need more time to think of a better name. I don't have the bandwidth to implement this currently, but if you'd like to submit a PR, I will get it merged + released.

Alternatively, if you want to work around this on the current version of celery once, you could just pass a tenant_id to the task's arguments (even if it isn't used in the function), and that, in turn, will be used to generate the key.

i.e.

@celery.task(base=QueueOnce)
def process_video(filename, tenant_id):
 pass 


process_video.delay('chase.mp4', 'ubuntu_1')
process_video.delay('chase.mp4', 'ubuntu_2')

@cameronmaske cameronmaske changed the title Custom key generation function Suggestion: Custom key generation function Apr 15, 2019
@adimux
Copy link
Author

adimux commented Apr 16, 2019

Hi @cameronmaske
That's what I was thinking.
If I am still going to use celery-once, I might submit this PR soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants