Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add amqp queue driver #402

Merged
merged 11 commits into from
Oct 5, 2018
Merged

Add amqp queue driver #402

merged 11 commits into from
Oct 5, 2018

Conversation

josephmancuso
Copy link
Member

@josephmancuso josephmancuso commented Oct 4, 2018

This PR adds support for real message brokers for queues. Currently we just have an async driver which sends jobs into a seperate thread which works pretty well actually but this PR introduces support for any amqp standardized message broker like Rabbitmq.

Once this PR gets merged the way to use it will be:

  1. install RabbitMQ on your machine (and production machine)
  2. change the driver to amqp and add the required credentials:
DRIVER = 'amqp'
...
DRIVERS = {
    'amqp': {
        'username': 'guest',
        'password': 'guest',
        'host': 'localhost',
        'port': '5672',
        'channel': 'default',
    }
}
  1. Start RabbitMQ. Something like:
$ rabbitmq-server

  ##  ##
  ##  ##      RabbitMQ 3.7.8. Copyright (C) 2007-2018 Pivotal Software, Inc.
  ##########  Licensed under the MPL.  See http://www.rabbitmq.com/
  ######  ##
  ##########  Logs: /usr/local/var/log/rabbitmq/rabbit@localhost.log
                    /usr/local/var/log/rabbitmq/rabbit@localhost_upgrade.log

              Starting broker...
 completed with 6 plugins.

which starts the server.

  1. Then start the Masonite worker in a separate terminal:
$ craft queue:work
  1. Pass jobs into the queue like normal:
from app.jobs import SomeJob
...
def show(self, Queue):
    # do your normal logic
    Queue.push(SomeJob)

and you should see them in the worker terminal window:

screen shot 2018-10-04 at 12 03 31 am

That's it!

@josephmancuso josephmancuso added the next minor Issue scheduled for the next minor release label Oct 4, 2018
@josephmancuso josephmancuso self-assigned this Oct 4, 2018
@josephmancuso josephmancuso requested a review from a team October 4, 2018 04:02
@coveralls
Copy link

coveralls commented Oct 4, 2018

Coverage Status

Coverage decreased (-0.5%) to 91.492% when pulling 51cc2c4 on add-amqp-driver into ff1de57 on master.

@bjorntheart
Copy link
Contributor

I'm very excited about this one @josephmancuso. Great work

@josephmancuso
Copy link
Member Author

josephmancuso commented Oct 4, 2018

we will need to find a way to pass in variables to the job. I am thinking we do it like Laravel and just instantiate the object instead of passing the class:

from app.jobs import SomeJob
...
def show(self, Queue):
    # do your normal logic
    Queue.push(SomeJob(variable1, variable2))

But if this is going into the minor we also need to support this for async driver. This MAY need some workaround for 2.0 that won't be required for 2.1.

This would mean that the constructor will no longer support being resolved but we can still make the handle method resolved if anything.

I want to make sure this goes into the next minor so @aisola and I can use it in some current projects we have.

@josephmancuso
Copy link
Member Author

@MasoniteFramework/reviewers Without any objections, I'm going to merge this in tonight.

@josephmancuso
Copy link
Member Author

josephmancuso commented Oct 4, 2018

The latest commit also adds the ability to do 2 things:

  1. You can add the class like normal:
from app.jobs import SayHello

def show(self, Queue):
    Queue.push(SayHello)

This will resolve the constructor

  1. Or you can pass now pass in an object and instantiate it which is useful for passing in variables from the container:
from app.jobs import SayHello

def show(self, Queue):
    Queue.push(SayHello('variable1', 'variable2'))

Instantiating the class will NOT resolve the constructor. So it is currently either have the constructor resolved or pass in data from controller (which can include any dependencies you need anyway so not THAT big of a deal.)

@josephmancuso josephmancuso merged commit 652b79e into master Oct 5, 2018
@josephmancuso josephmancuso deleted the add-amqp-driver branch October 18, 2018 03:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
next minor Issue scheduled for the next minor release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants