Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Beanstalkd from Phalcon #13364

Closed
sergeyklay opened this Issue May 2, 2018 · 30 comments

Comments

Projects
@sergeyklay
Copy link
Member

sergeyklay commented May 2, 2018

I propose to remove beanstalkd from Phalcon and move this functionality to the incubator or other repo. This may be done for 4.0.0 version.

Actually beanstalkd isn't a popular queue system. In addition, beanstalk doesn't provides any unique feature (except a bit magic terminology e.g. tube, peek-ready, etc.). This project has zero activity for many years, unanswered questions like Is this project still active, lack of any out of box HA tools, replication tools, cluster integration, etc.

In fact many of our users do not use beanstalkd and even did not use. I think that in the framework there should be truly top requested and most used tools.

@409H

This comment has been minimized.

Copy link

409H commented May 3, 2018

In fact many of our users do not use beanstalkd and even did not use.

I'm curious, how did you poll this?

I'm actually a user that uses the beanstalkd functionality of Phalcon, though I don't see how moving it into Incubator will affect me much, but is it a task worth doing? What would be the pros of moving it out of the core and into Incubator?

@niden

This comment has been minimized.

Copy link
Member

niden commented May 3, 2018

@409H I too use beanstalkd on an environment with Phalcon. There have been quite a number of issues to maintain that environment, so we switched to AWS SQS to remove the burden. The maintenance cost for beanstalk became too much for us.

As for Phalcon, we are trying to keep the framework as slim as possible but as feature rich as possible. As mentioned by Serghei, the project has low activity (well zero actually) so we really don't want to support connection to a system that will potentially have bugs or is abandoned.

Moving the functionality to the incubator will have minimal impact to existing applications, since all you will have to do now is include the related composer package.

HTH

@Jurigag

This comment has been minimized.

Copy link
Member

Jurigag commented May 3, 2018

If we will add rabbitmq as replacement then fine. Also i use it in my environment.

@davidtavarez

This comment has been minimized.

Copy link

davidtavarez commented May 3, 2018

rabbitmq is a good option

@409H

This comment has been minimized.

Copy link

409H commented May 3, 2018

@niden I understand. Count me in favour of the proposal then.

@ruudboon

This comment has been minimized.

Copy link
Member

ruudboon commented May 3, 2018

I’m using the beanstalkd a lot. The reason for me to go for beanstalk is the native integration with Phalcon.
+1 for me to keep it part of Phalcon

@HanXHX

This comment has been minimized.

Copy link

HanXHX commented May 3, 2018

Best way: a good refactoring with https://github.com/queue-interop/queue-interop

@benfavre

This comment has been minimized.

Copy link

benfavre commented May 3, 2018

I use beanstalkd, don't care for it beeing in core, make's no difference if the functionnality is ported to incubator and preserved as is.
Not sure the rationale for removing it has anything to do with the status of the project.
If we have something in core it should be something that supports all queue systems, not a specific one (even tho beanstalkd is cool imho).

@aat2703

This comment has been minimized.

Copy link

aat2703 commented May 3, 2018

Used Beanstalkd for a while... But we switched to AWS SQS... It scales easily and is highly available... I don’t see any reason for it being in core... Maybe in the incubator 👍

@rudiservo

This comment has been minimized.

Copy link
Contributor

rudiservo commented May 3, 2018

I think that the removal or not of beanstalkd it's a case of if it is still a viable software option or it's just not relevant for usage, kind of like mysql and mysqli.
Has for adding support to other software it's a simple case of normalizing a wrapper class and creating functionality for any queue software, it should not matter if we support beanstalkd or not, it should matter the range of options that Phalcon has available for use.
Maybe have a version 3.x with deprecated beanstalkd and other options available (incubator) so that people can have a window of time long enough to migrate avoiding projects getting issues when they do an upgrade?

@rpacker

This comment has been minimized.

Copy link

rpacker commented May 3, 2018

We use it, but either way, I'd vote to add some other queues such as RabbitMQ, and SQS.

@marcelwenting

This comment has been minimized.

Copy link

marcelwenting commented May 4, 2018

New guy here, made an account specifically for this item.
Beanstalkd has become my queue system of choice because of the integration with PhalconPHP, even though I support the many cons that have been mentioned.
Therefor I would like to see another queueing system like RabbitMQ supported in the core before beanstalkd is removed from the core.
I would also like to suggest at transition period, where you would have 2 queueing systems in the core to smoothen the transition for those who do use beanstalkd.

@benfavre

This comment has been minimized.

Copy link

benfavre commented May 4, 2018

@marcelwenting if it's moved to the incubator and can just add it to composer then you don't need to change, so we don't have to add another one to core either.

@marcelwenting

This comment has been minimized.

Copy link

marcelwenting commented May 4, 2018

@benfavre What you are saying is definitely right, but it:

  1. assumes one is using composer in the project.
  2. Can cause a lot of work when supporting many projects(codebase, ci/cd etc).
  3. It is not part of the precompiled framework anymore, therefor have a negative impact on performance.
    The beauty of Phalcon is to have a full framework with support for the different components(db, cache, log etc).
    Where other components like databases, cache and image always gave multiple options. Queue never did. I think by having the framework support for other queuing systems, before removing the only current option is just the more devfriendly way to go.
    Again; I support the the current idea of beanstalkd not being the a really viable option because of the lack of support/updates, but would just like a proper alternative introduced first.
@benfavre

This comment has been minimized.

Copy link

benfavre commented May 4, 2018

@marcelwenting So we shouldn't remove anything and instead, access to Beanstalkd should be replaced by a higher level implementation, and we can move Beanstalkd to be one of the options of the wrapper.

@metaliciouz

This comment has been minimized.

Copy link
Contributor

metaliciouz commented May 4, 2018

Alternative: https://github.com/pdezwart/php-amqp (PECL extension)

@niden

This comment has been minimized.

Copy link
Member

niden commented May 4, 2018

@marcelwenting Allow me to disagree and here is why:

Queue systems are designed to work asynchronously and therefore the performance impact is minimal if not negligible. I am thinking of a use case where a queue system is used to email people (notifications) for instance. In that use case your delay will be network latency vs. PHP execution and since this is going to be asynchronous then it will have no impact for the main applications.

There are plenty of other implementations but the core is that queues are used to move processing from the main app and offload it to another process.

Having the queue system in the incubator allows users more choice with the help of composer.

Finally to answer a concern I saw above, there will be ample time before this gets implemented so that developers can adapt their code.

@scrnjakovic

This comment has been minimized.

Copy link
Contributor

scrnjakovic commented May 28, 2018

I'd also like us have a queue system wrapper with adapters, and then roll out beanstalkd when we have decent alternatives.

@jellisii

This comment has been minimized.

Copy link

jellisii commented Jun 4, 2018

To extend @scrnjakovic's comment, having a base Queue adapter would fall in line with the Database/Model pattern that is well established within Phalcon.

As a current user of BeanstalkD in several projects, going the route of an adapter would probably still break things for me, but also would allow for BeanstalkD to be contained in a single file/folder that still lived within the main code, bootstrap the addition of multiple other queue adapters alongside it.

@stamster

This comment has been minimized.

Copy link
Contributor

stamster commented Jun 13, 2018

All in all - alternative must be made avail. So abstraction layer (interface) which will provide same functionality across different adapters. Example is session adapter in Phalcon.

@Jurigag

This comment has been minimized.

Copy link
Member

Jurigag commented Jun 13, 2018

Tbh im personally thinking if we need whole adapter over redis/rabbitmq queue. Why reinvent a wheel? Just add only queue adapter using proper extension and some kind of cli class to make it handling of jobs added to queues easier, like we do:

$queue->put('hello', 'world');

and

class HelloHandler extends Phalcon\Cli\Task\RabbitMQTask
{
    public function handle($message)
    {
         echo $message->body; // displays world
    }

    public function config()
    {
        // custom function where we can type channel name, by default it uses class name or run it with some kind of sleep or whatever
    }
}

And run it in cli. I think we should use already provided extensions, don't write whole rabbitmq/redis queue adapter.

Though php-resque is the whole thing and it's php code.

@jellisii

This comment has been minimized.

Copy link

jellisii commented Jun 29, 2018

I guess a question that needs to be answered is "what are the common things that you have to do to a queue?" Obviously Put/Push, Pop/Reserve, watch, some sort of job inspection method. I presume other queuing systems (Gearman, RabbitMQ, etc) have some sort of construct for multiple queues so being able switch and inspect queues. Getting the verbs for acting on queues to a common list would bootstrap the idea of a queue driver/adapter/extension, much like has been done for other components.

@sky3hao

This comment has been minimized.

Copy link

sky3hao commented Aug 14, 2018

I recommend NSQ

@jellisii

This comment has been minimized.

Copy link

jellisii commented Aug 15, 2018

It looks like there may be a second breath of life in BeanstalkD if today's discussion has any momentum behind it: https://github.com/cornstalkd/cornstalkd

@sergeyklay

This comment has been minimized.

Copy link
Member Author

sergeyklay commented Aug 17, 2018

FYI: beanstalkd is get well in beanstalkd/beanstalkd

@stamster

This comment has been minimized.

Copy link
Contributor

stamster commented Aug 17, 2018

Well, a little too late IMHO but let's where that will end.

I'm about to implement RabbitMQ native so it looks more promising on the long run than beanstalkd.

@mia0x75

This comment has been minimized.

Copy link

mia0x75 commented Oct 4, 2018

how about nsq? https://nsq.io/, a realtime distributed messaging platform written in golang.

@stamster

This comment has been minimized.

Copy link
Contributor

stamster commented Oct 22, 2018

It seems that beanstalkd just died (again).

beanstalkd/beanstalkd#399

For the folks giving thumbs down here - well, you don't have to take it personally - it's just facts, we all loved (past-tense) Beanstalkd.

@jellisii

This comment has been minimized.

Copy link

jellisii commented Oct 22, 2018

I still advocate for a driver instead of committing to a single queue.

@niden niden added this to To do in 4.0 Release via automation Feb 21, 2019

niden added a commit to niden/cphalcon that referenced this issue Feb 22, 2019

[phalcon#13364] - Bye bye beanstalk! We love you! Next version we wil…
…l add a Queue adapter for more adaptations

@niden niden moved this from To do to In progress in 4.0 Release Feb 22, 2019

@niden niden referenced this issue Feb 22, 2019

Merged

T13364 beanstalk removal #13850

2 of 3 tasks complete

niden added a commit to niden/cphalcon that referenced this issue Feb 23, 2019

niden added a commit to niden/cphalcon that referenced this issue Feb 23, 2019

[phalcon#13364] - Merge remote-tracking branch 'upstream/4.0.x' into …
…T13364-beanstalk-removal

* upstream/4.0.x:
  T13844 mcall undefined errors (phalcon#13849)

niden added a commit that referenced this issue Feb 23, 2019

T13364 beanstalk removal (#13850)
* [4.0.x] - Added unicode flag for email filter

* [4.0.x] - Updating the ext folder

* [4.0.x] - Added Arr helper class

* Revert "[4.0.x] - Added Arr helper class"

This reverts commit f8153d3.

* Revert "[4.0.x] - Updating the ext folder"

This reverts commit 655eafe.

* [#13364] - Bye bye beanstalk! We love you! Next version we will add a Queue adapter for more adaptations

* [#13364] - Updated the changelog
@niden

This comment has been minimized.

Copy link
Member

niden commented Feb 23, 2019

Resolved.

Discussion for new component: #13851

@niden niden closed this Feb 23, 2019

4.0 Release automation moved this from In progress to Done Feb 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.