Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Queue Service #13851

Open
niden opened this Issue Feb 22, 2019 · 21 comments

Comments

Projects
None yet
@niden
Copy link
Member

niden commented Feb 22, 2019

In Phalcon v4 we removed Beanstalkd (#13364, #13850) since the project is no longer supported.

This issue serves as a discussion and design board for potentially a new Queue adapter that can be used for future integrations with popular queue systems.

https://github.com/queue-interop/queue-interop is a good candidate

Alternatively, we can leave the queue system completely out of Phalcon and use implementations from the incubator or custom built ones from developers based on the needs of their applications.

@niden niden self-assigned this Feb 22, 2019

@CameronHall

This comment has been minimized.

Copy link
Member

CameronHall commented Feb 23, 2019

Queue Interop looks good. I'd be happy with creating an adapter so we can supply a queue-interop compatible client.

@JABirchall

This comment has been minimized.

Copy link

JABirchall commented Feb 23, 2019

I think a QueueAdapaterInterface would be good but I also think phalcon should implement an out the box queue adapter for some of the popular queue services.

ref: #13385
Some good comments on that NFR for possible implimentation.

@ruudboon

This comment has been minimized.

Copy link
Member

ruudboon commented Feb 24, 2019

+1 for an common interface with at least a redis adapter. The queue interop looks like a good example. They are trying to pitch it to fig, if that get’s approved it would be even better.

@benwills

This comment has been minimized.

Copy link

benwills commented Feb 24, 2019

I second the idea for default integration with Redis.

@stamster

This comment has been minimized.

Copy link
Contributor

stamster commented Feb 25, 2019

The good abstraction layer is a necessity here - so the underlying concrete adapter implementation does not matter that much like it was the case with Beanstalk.

@jellisii

This comment has been minimized.

Copy link

jellisii commented Feb 25, 2019

A implementation of the BeanstalkD adapter would be nice. I have a few projects that rely on BeanstalkD to get work done, so this isn't exactly a unbiased request.

I still advocate for an adapter-style interface that multiple implementations can be written to. That does appear what queue interop seems to be driving for. I do understand that because there's not a good base like ANSI SQL to start from that there's difficulty around it.

@Jurigag

This comment has been minimized.

Copy link
Member

Jurigag commented Feb 25, 2019

Definitely there should be option to implement our own adapter and just switch to any messaging system we want like beanstalkd, redis, rabbitmq, apache kafka etc. Interop looks promising.

@david-duncan

This comment has been minimized.

Copy link

david-duncan commented Feb 25, 2019

What would this implement that packages like interop don't already implement?

Are we proposing a wrapper for a wrapper?

@Jurigag

This comment has been minimized.

Copy link
Member

Jurigag commented Feb 25, 2019

Yea that's a good question if we need actually any adapter and built-in queues for phalcon.

@niden

This comment has been minimized.

Copy link
Member Author

niden commented Feb 25, 2019

The way I was (personally) looking at it, it would be similar to what happens in PSR. You create an Queue class that consumes the interop-queue interface and offers functionality to Phalcon applications.

Then you can inject an adapter that implements that interface, whatever the adapter is for (redis, nats etc.).

@scrnjakovic

This comment has been minimized.

Copy link
Contributor

scrnjakovic commented Feb 26, 2019

@jellisii did you just miss @niden mentioning beanstalkd is being dropped in v4? Which, btw, was announced way back. Beanstalkd was ditched for a reason - it's uncertain future and stability as every now and then, maintainer disappears like a fart in the wind.

I'm all in for creating an interface with at least redis implementation, unless someone else has already worked out the abstraction, in which case we can just port it to zephir.

@Zaszczyk

This comment has been minimized.

Copy link
Contributor

Zaszczyk commented Mar 3, 2019

I think Phalcon team should not focus on queue adapter. There are a lot of queues, which have own mature and stable libraries/plugins/adapters. Developers will prefer to use them, than new and poor Phalcon adapter.

@stamster

This comment has been minimized.

Copy link
Contributor

stamster commented Mar 4, 2019

@Zaszczyk
We're talking about interoperability here - having a layer of abstraction which will allow devs to switch the concrete queue service on one place (IoC service), similar like we have with DB wrapper/adapters.

"new and poor Phalcon adapter"?
So why we have a framework there at first place? Let's do everything with composer and bundle up a custom framework per each project of ours then.

@aat2703

This comment has been minimized.

Copy link

aat2703 commented Mar 4, 2019

I actually like the way Laravel handles queues with a class per queue action...

SendEmail::dispatch(); // Job is now dispatched to queue for example AWS SQS

https://laravel.com/docs/5.7/queues

@jellisii

This comment has been minimized.

Copy link

jellisii commented Mar 11, 2019

@scrnjakovic I did not miss that, but if there's a queue adapter, as has been suggested using queue-interop, having the ability to import BeanstalkD functionality from something like incubator would be nice.

I use BeanstalkD in several projects because Phalcon made it easy to do so. Deprecation of the existing feature for 4.x and dropping for 5.x would make the transition easier for existing users of that particular Phalcon feature, but it sounds like that option is no longer available.

@stamster

This comment has been minimized.

Copy link
Contributor

stamster commented Mar 15, 2019

@jellisii BeanstalkD is an abandoned project, w/o any bright future. Sadly, even the new maintainer gave up on the project soon after getting access to it. So that shall be removed from Phalcon for sure.

Once upon a time, I was also a happy user of BeanstalkD via Phalcon.

@erisrayanesh

This comment has been minimized.

Copy link

erisrayanesh commented Mar 15, 2019

However the Beanstalkd project is abandoned but if its latest stable version works good without any critical or security issues for most of currently using developers, it would be unnecessary to remove its adapter until Phalcon 5.x . So it is a good idea to encourage developers for using other developing and progressing queue service projects with new featured adapters in Phalcon 4.x .

@CameronHall

This comment has been minimized.

Copy link
Member

CameronHall commented Mar 16, 2019

I agree. Considering it is an abandoned project, it won't be getting any updates. This means there's minimal if any maintenance debt within the adapter. I believe that we should leave it in 4.0.x at least on the basis that there will be minimal if any maintenance overhead and the argument to remove it is to "fix something that isn't broken".

Alternatively, we could move it to the incubator and not supply any adapters in the framework.

@scrnjakovic

This comment has been minimized.

Copy link
Contributor

scrnjakovic commented Mar 16, 2019

The latest version was released in 2014 and numerous bugs were reported since then, none of which were fixed. If you want to use it, no one is stoping you, you still can. It's just not going to be the part of the core.

@erisrayanesh

This comment has been minimized.

Copy link

erisrayanesh commented Mar 17, 2019

@scrnjakovic I'm not a fan of the Beanstalkd project and current Phalcon queue service which is now supporting only the Beanstalkd. I'm just saying that like the idea of @niden and @jellisii about creating an adapter interface as what Phalcon does for caching, database or logger, it still could implement an adapter for the Beanstalkd. Because as I reviewed the beanstalkd's commits, probably it's going to be active again since some minor changes committed couple of month ago and discussions made about making release here.

@Rajeshr34

This comment has been minimized.

Copy link

Rajeshr34 commented Mar 17, 2019

BeanstalkD ++

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.