Queue Interop Support #104

ASKozienko · 2017-08-01T10:25:27Z

No description provided.

makasim · 2017-08-01T13:13:30Z

Tests fail because of php5.5 requirement and dependency on enqueue/amqp-lib 0.7 which has not been released yet.

makasim · 2017-08-01T13:15:45Z

This approach is based purely on queue-interop interfaces and does require enqueue. We can propose another solution based on enqueue client. It'll introduce a dependency on enqueue but it allows to remove a lot of low-level MQ stuff from here.

makasim · 2017-08-01T12:38:58Z

src/drivers/queue_interop/Queue.php

+        while (true) {
+            if ($message = $consumer->receive()) {
+                list($ttr, $body) = explode(';', $message->getBody(), 2);
+                if ($this->handleMessage(null, $body, $ttr, 1)) {


should we reject it on false?

Queue reserves message for handling before method calling.
If method returns false, queue should keep message in reserve. If false, message must be deleted.

@ASKozienko could you please do message requeue on false and ack on true?

makasim · 2017-08-01T12:39:31Z

src/drivers/queue_interop/Queue.php

+            $this->getContext()->createQueue($this->queueName)
+        );
+
+        while (true) {


it would be good to add usleep. 10-100ms would be ok.

I think ability to configure delay would be good with 10ms default.

I was wrong. There is no need for sleep since receive is blocking (it waits on a socket, or does sleep internally). No need to do it here.

makasim · 2017-08-01T13:21:05Z

src/drivers/queue_interop/Queue.php

+
+        $this->producer->send(
+            $this->getContext()->createQueue($this->queueName),
+            $this->getContext()->createMessage("$ttr;$message")


@samdark what does ttr mean?

@makasim "time to read"

Time to reserve. See guide about this:
https://github.com/yiisoft/yii2-queue/blob/master/docs/guide/retryable.md
https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-ru/retryable.md

makasim · 2017-08-01T13:27:43Z

@samdark could you please have a look?

samdark · 2017-08-02T10:30:28Z

composer.json

    },
    "suggest": {
        "ext-pcntl": "Need for process signals.",
        "yiisoft/yii2-redis": "Need for Redis queue.",
        "pda/pheanstalk": "Need for Beanstalk queue.",
        "php-amqplib/php-amqplib": "Need for AMQP queue.",
-        "ext-gearman": "Need for Gearman queue."
+        "ext-gearman": "Need for Gearman queue.",
+        "php-enqueue/amqp-ext": "Needs for support of AMQP with queue interop",


Needs for support of AMQP with queue interop → Required for AMQP with queue interop

Same for below descriptions.

samdark · 2017-08-02T10:31:13Z

docs/guide/driver-queue-interop.md

+
+The driver works with many queue brokers.
+
+Full list of supported brokers you can find on the [Queue Interop](https://github.com/queue-interop/queue-interop) page


You can find full list of...

samdark · 2017-08-02T10:31:44Z

docs/guide/driver-queue-interop.md

+
+Full list of supported brokers you can find on the [Queue Interop](https://github.com/queue-interop/queue-interop) page
+
+To get it works you have to install and setup one of the supported implementation.


In order for it to work you need to install and configure one the implementations supported.

samdark · 2017-08-02T10:34:05Z

docs/guide/driver-queue-interop.md

+
+To get it works you have to install and setup one of the supported implementation.
+
+Configuration example for the RabbitMQ AMQP:


I'm not sure it's a good example since there's RabbitMQ driver that works w/o Queue Interop already. It all depends on the goal.

@zhuravljov Do we want to remove our own implementations and use Queue Interop ones instead? That would be less maintenance and more profit for PHP overall. Or do we want to have additional drivers through Queue Interop?

samdark · 2017-08-02T10:50:35Z

docs/guide/driver-queue-interop.md

+
+`listen` command has options:
+
+- `--verbose`, `-v`: print executing statuses into console.


print executing statuses into console → print execution status into console

samdark · 2017-08-02T10:50:50Z

docs/guide/driver-queue-interop.md

+`listen` command has options:
+
+- `--verbose`, `-v`: print executing statuses into console.
+- `--isolate`: verbose mode of a job execute. If enabled, execute result of each job will be printed.


verbose mode of a job execute → verbose mode of a job execution

execute result → execution result

samdark · 2017-08-02T10:52:11Z

src/drivers/queue_interop/Queue.php

+            $this->getContext()->createQueue($this->queueName)
+        );
+
+        while (true) {


I think ability to configure delay would be good with 10ms default.

samdark · 2017-08-02T10:55:45Z

src/drivers/queue_interop/Queue.php

+            }
+
+            $rc = new \ReflectionClass($this->factoryClass);
+            if (false == $rc->implementsInterface(PsrConnectionFactory::class)) {


http://php.net/is_a could be used instead of reflection.

samdark · 2017-08-02T11:46:43Z

src/drivers/queue_interop/Queue.php

@@ -108,8 +110,7 @@ private function getContext()
                throw new \LogicException(sprintf('The "factoryClass" option "%s" is not a class', $this->factoryClass));
            }

-            $rc = new \ReflectionClass($this->factoryClass);
-            if (false == $rc->implementsInterface(PsrConnectionFactory::class)) {
+            if (false == is_a($this->factoryClass, PsrConnectionFactory::class)) {


=== could be used.

samdark · 2017-08-02T11:47:46Z

Overall the code looks fine but I'd like to hear @zhuravljov opinion on future development and path we should take.

makasim · 2017-08-03T13:44:08Z

src/drivers/queue_interop/Queue.php

+    protected function pushMessage($message, $ttr, $delay, $priority)
+    {
+        if ($delay) {
+            throw new NotSupportedException('Delayed work is not supported in the driver.');


@ASKozienko Could you please implement this (since their support has been merged php-enqueue/enqueue-dev#149)

makasim · 2017-08-07T14:21:50Z

src/drivers/queue_interop/Queue.php

@@ -117,6 +123,21 @@ private function getContext()
            /** @var PsrConnectionFactory $factory */
            $factory = new $this->factoryClass(isset($this->factoryConfig['dsn']) ? $this->factoryConfig['dsn'] : $this->factoryConfig);

+            if ($factory instanceof DelayStrategyAware) {


@ASKozienko I am not sure factory implements delay strategy aware interface.

makasim · 2017-08-07T14:34:31Z

Consider marking classes final so it easier for us to change in future (without BC breaks)

makasim · 2017-08-08T14:38:31Z

@samdark @zhuravljov tests fail only on php5.5 because enqueue requires php5.6 as the minimum version. How should we proceed?

samdark · 2017-08-08T14:40:40Z

There are only two ways:

Lowering enqueue requirements. Is it possible? Is 5.6 actually used?
Raising yii2-queue requirements. Doable but we prefer not to do so unless needed.

makasim · 2017-08-08T14:59:34Z

@samdark I am thinking of dropping 5.x at all. Why are you so stick to it?

samdark · 2017-08-08T15:10:17Z

Because we have Yii 2.0 and it works with 5.4. Even when we'll release 2.1 that requires PHP 7.1, we'll have to support 2.0 for some time cause there are countless projects in production using it.

makasim · 2017-08-10T13:57:37Z

@samdark we decided to skip the tests for php5.5 for now. If someone wants to use yiiqueue with enqueue they have to make sure they are on php5.6 at least.

Could you please review it?

samdark · 2017-08-11T10:40:53Z

src/drivers/queue_interop/Queue.php

+
+class Queue extends CliQueue
+{
+    const RABBITMQ_DELAY_DLX = 'dlx';


Why there are rabbit-specific things in a general purpose class?

We tried to keep it as simple as possible, so everything in one class.

we can move amqp interop specific stuff to separate class. Is it ok?

rob006 · 2017-08-13T22:49:26Z

composer.json

@@ -45,7 +61,8 @@
            "yii\\queue\\file\\": "src/drivers/file",
            "yii\\queue\\gearman\\": "src/drivers/gearman",
            "yii\\queue\\redis\\": "src/drivers/redis",
-            "yii\\queue\\sync\\": "src/drivers/sync"
+            "yii\\queue\\sync\\": "src/drivers/sync",
+            "yii\\queue\\queue_interop\\": "src/drivers/queue_interop"


Can't we use just yii\queue\interop as a namespace?

zhuravljov · 2017-12-05T05:03:37Z

Merged from #158.

queue interop

4abd1c8

ASKozienko changed the title ~~Queue Interop Support~~ [WIP] Queue Interop Support Aug 1, 2017

queue interop

6b89628

ASKozienko force-pushed the queue-interop branch from bae3cd6 to 6b89628 Compare August 1, 2017 10:54

ASKozienko added 4 commits August 1, 2017 14:06

queue interop

481060a

queue interop

9cc793e

queue interop

0f6eb76

queue interop

248aaf2

ASKozienko changed the title ~~[WIP] Queue Interop Support~~ Queue Interop Support Aug 1, 2017

makasim reviewed Aug 1, 2017

View reviewed changes

queue interop

d53f763

zhuravljov added the type:enhancement Enhancement label Aug 2, 2017

makasim mentioned this pull request Aug 2, 2017

Proposals queue-interop/queue-interop#4

Open

samdark requested changes Aug 2, 2017

View reviewed changes

queue interop

497ab57

samdark requested changes Aug 2, 2017

View reviewed changes

makasim reviewed Aug 3, 2017

View reviewed changes

queue interop

cff0a63

makasim reviewed Aug 7, 2017

View reviewed changes

ASKozienko added 2 commits August 8, 2017 09:37

queue interop

4e6fa97

queue interop

79f27a0

ASKozienko added 9 commits August 10, 2017 14:46

exclude enqueue for php 5.5

f177722

exclude enqueue for php 5.5

15478f6

exclude enqueue for php 5.5

a774912

exclude enqueue for php 5.5

7103cdb

exclude enqueue for php 5.5

2a80a50

exclude enqueue for php 5.5

cb24354

require queue interop

aa7e7bb

require queue interop

e0d34fb

require queue interop

ebcfd96

samdark reviewed Aug 13, 2017

View reviewed changes

rob006 reviewed Aug 13, 2017

View reviewed changes

ASKozienko and others added 8 commits August 15, 2017 13:55

remove require queue interop

f62b2d1

require queue interop

454f66d

queue interop

9dbb593

Merge branch 'master' into queue-interop

1f7657f

queue interop

9bd6041

queue interop

8ad8868

queue interop

6cb788b

queue interop

0b60c94

zhuravljov closed this Dec 5, 2017


		The driver works with many queue brokers.

		Full list of supported brokers you can find on the [Queue Interop](https://github.com/queue-interop/queue-interop) page


		Full list of supported brokers you can find on the [Queue Interop](https://github.com/queue-interop/queue-interop) page

		To get it works you have to install and setup one of the supported implementation.


		To get it works you have to install and setup one of the supported implementation.

		Configuration example for the RabbitMQ AMQP:


		`listen` command has options:

		- `--verbose`, `-v`: print executing statuses into console.

Queue Interop Support #104

Queue Interop Support #104

Conversation

ASKozienko commented Aug 1, 2017

makasim commented Aug 1, 2017

makasim commented Aug 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

makasim Aug 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

makasim commented Aug 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

samdark commented Aug 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

makasim commented Aug 7, 2017

makasim commented Aug 8, 2017

samdark commented Aug 8, 2017

makasim commented Aug 8, 2017

samdark commented Aug 8, 2017

makasim commented Aug 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhuravljov commented Dec 5, 2017

makasim Aug 3, 2017 •

edited

Loading