Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add functions to set a custom queue name #159

Closed
wants to merge 1 commit into from

Conversation

kobayashi
Copy link
Contributor

This introduces new functions to customize queue name from default "celery" to own queue name for each redis and AMQP.
Related Issues: #43 , #136

@grutz
Copy link

grutz commented Nov 5, 2020

I really love this change since most celery environments I've worked in have had 4 or more queues to spread out the demand/load. However having to put the queue name in the broker configuration would mean making n+1 brokers for the number of queues.

In python you can pass the queue name across the task request:

https://docs.celeryproject.org/en/stable/userguide/routing.html#manual-routing

from feeds.tasks import import_feed
import_feed.apply_async(args=['http://cnn.com/rss'],
                        queue='feed_tasks',
                        routing_key='feed.import')

This PR is certainly a necessary feature. Passing the queue in the Delay call seems a must as well.

@yoonsio
Copy link
Member

yoonsio commented Nov 11, 2020

@kobayashi @grutz thank you for your pr and feedback.
I prefer to avoid adding any more constructors. We should really look into passing flexible options.
The internal broker queue is exposed as public variable so this feature is already supported.
For example, you can do the following instead of using the constructor.

broker := AMQPCeleryBroker{
  Connection: conn,
  Channel:    channel,
  Exchange:   NewAMQPExchange("myexchange"),
  Queue:      NewAMQPQueue("myqueue"),
  Rate:       4,
}

Please refer to #144

@yoonsio yoonsio closed this Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants