Remove Beanstalkd from Phalcon

[sergeyklay]

I propose to remove beanstalkd from Phalcon and move this functionality to the incubator or other repo. This may be done for 4.0.0 version.

Actually beanstalkd isn't a popular queue system. In addition, beanstalk doesn't provides any unique feature (except a bit magic terminology e.g. tube, peek-ready, etc.). This project has zero activity for many years, unanswered questions like Is this project still active, lack of any out of box HA tools, replication tools, cluster integration, etc.

In fact many of our users do not use beanstalkd and even did not use. I think that in the framework there should be truly top requested and most used tools.

[409H]

In fact many of our users do not use beanstalkd and even did not use.

I'm curious, how did you poll this?

I'm actually a user that uses the beanstalkd functionality of Phalcon, though I don't see how moving it into Incubator will affect me much, but is it a task worth doing? What would be the pros of moving it out of the core and into Incubator?

[niden]

@409H I too use beanstalkd on an environment with Phalcon. There have been quite a number of issues to maintain that environment, so we switched to AWS SQS to remove the burden. The maintenance cost for beanstalk became too much for us.

As for Phalcon, we are trying to keep the framework as slim as possible but as feature rich as possible. As mentioned by Serghei, the project has low activity (well zero actually) so we really don't want to support connection to a system that will potentially have bugs or is abandoned.

Moving the functionality to the incubator will have minimal impact to existing applications, since all you will have to do now is include the related composer package.

HTH

[Jurigag]

If we will add rabbitmq as replacement then fine. Also i use it in my environment.

[davidtavarez]

rabbitmq is a good option

[409H]

@niden I understand. Count me in favour of the proposal then.

[ruudboon]

I’m using the beanstalkd a lot. The reason for me to go for beanstalk is the native integration with Phalcon.
+1 for me to keep it part of Phalcon

[HanXHX]

Best way: a good refactoring with https://github.com/queue-interop/queue-interop

[benfavre]

I use beanstalkd, don't care for it beeing in core, make's no difference if the functionnality is ported to incubator and preserved as is.
Not sure the rationale for removing it has anything to do with the status of the project.
If we have something in core it should be something that supports all queue systems, not a specific one (even tho beanstalkd is cool imho).

[aat2703]

Used Beanstalkd for a while... But we switched to AWS SQS... It scales easily and is highly available... I don’t see any reason for it being in core... Maybe in the incubator 👍

[rudiservo]

I think that the removal or not of beanstalkd it's a case of if it is still a viable software option or it's just not relevant for usage, kind of like mysql and mysqli.
Has for adding support to other software it's a simple case of normalizing a wrapper class and creating functionality for any queue software, it should not matter if we support beanstalkd or not, it should matter the range of options that Phalcon has available for use.
Maybe have a version 3.x with deprecated beanstalkd and other options available (incubator) so that people can have a window of time long enough to migrate avoiding projects getting issues when they do an upgrade?

[rpacker]

We use it, but either way, I'd vote to add some other queues such as RabbitMQ, and SQS.

[marcelwenting]

New guy here, made an account specifically for this item.
Beanstalkd has become my queue system of choice because of the integration with PhalconPHP, even though I support the many cons that have been mentioned.
Therefor I would like to see another queueing system like RabbitMQ supported in the core before beanstalkd is removed from the core.
I would also like to suggest at transition period, where you would have 2 queueing systems in the core to smoothen the transition for those who do use beanstalkd.

[benfavre]

@marcelwenting if it's moved to the incubator and can just add it to composer then you don't need to change, so we don't have to add another one to core either.

[marcelwenting]

@benfavre What you are saying is definitely right, but it:

1. assumes one is using composer in the project.
2. Can cause a lot of work when supporting many projects(codebase, ci/cd etc).
3. It is not part of the precompiled framework anymore, therefor have a negative impact on performance.
   The beauty of Phalcon is to have a full framework with support for the different components(db, cache, log etc).
   Where other components like databases, cache and image always gave multiple options. Queue never did. I think by having the framework support for other queuing systems, before removing the only current option is just the more devfriendly way to go.
   Again; I support the the current idea of beanstalkd not being the a really viable option because of the lack of support/updates, but would just like a proper alternative introduced first.

[benfavre]

@marcelwenting So we shouldn't remove anything and instead, access to Beanstalkd should be replaced by a higher level implementation, and we can move Beanstalkd to be one of the options of the wrapper.

[stramunin]

Alternative: https://github.com/pdezwart/php-amqp (PECL extension)

[niden]

@marcelwenting Allow me to disagree and here is why:

Queue systems are designed to work asynchronously and therefore the performance impact is minimal if not negligible. I am thinking of a use case where a queue system is used to email people (notifications) for instance. In that use case your delay will be network latency vs. PHP execution and since this is going to be asynchronous then it will have no impact for the main applications.

There are plenty of other implementations but the core is that queues are used to move processing from the main app and offload it to another process.

Having the queue system in the incubator allows users more choice with the help of composer.

Finally to answer a concern I saw above, there will be ample time before this gets implemented so that developers can adapt their code.

[scrnjakovic]

I'd also like us have a queue system wrapper with adapters, and then roll out beanstalkd when we have decent alternatives.

[jellisii]

To extend @scrnjakovic's comment, having a base Queue adapter would fall in line with the Database/Model pattern that is well established within Phalcon.

As a current user of BeanstalkD in several projects, going the route of an adapter would probably still break things for me, but also would allow for BeanstalkD to be contained in a single file/folder that still lived within the main code, bootstrap the addition of multiple other queue adapters alongside it.

[stamster]

All in all - alternative must be made avail. So abstraction layer (interface) which will provide same functionality across different adapters. Example is session adapter in Phalcon.

[Jurigag]

Tbh im personally thinking if we need whole adapter over redis/rabbitmq queue. Why reinvent a wheel? Just add only queue adapter using proper extension and some kind of cli class to make it handling of jobs added to queues easier, like we do:

$queue->put('hello', 'world');

and

class HelloHandler extends Phalcon\Cli\Task\RabbitMQTask
{
    public function handle($message)
    {
         echo $message->body; // displays world
    }

    public function config()
    {
        // custom function where we can type channel name, by default it uses class name or run it with some kind of sleep or whatever
    }
}

And run it in cli. I think we should use already provided extensions, don't write whole rabbitmq/redis queue adapter.

Though php-resque is the whole thing and it's php code.

[jellisii]

I guess a question that needs to be answered is "what are the common things that you have to do to a queue?" Obviously Put/Push, Pop/Reserve, watch, some sort of job inspection method. I presume other queuing systems (Gearman, RabbitMQ, etc) have some sort of construct for multiple queues so being able switch and inspect queues. Getting the verbs for acting on queues to a common list would bootstrap the idea of a queue driver/adapter/extension, much like has been done for other components.

[sky3hao]

I recommend NSQ

[jellisii]

It looks like there may be a second breath of life in BeanstalkD if today's discussion has any momentum behind it: https://github.com/cornstalkd/cornstalkd

[sergeyklay]

FYI: beanstalkd is get well in beanstalkd/beanstalkd

[stamster]

Well, a little too late IMHO but let's where that will end.

I'm about to implement RabbitMQ native so it looks more promising on the long run than beanstalkd.

[mia0x75]

how about nsq? https://nsq.io/, a realtime distributed messaging platform written in golang.

[stamster]

It seems that beanstalkd just died (again).

beanstalkd/beanstalkd#399

For the folks giving thumbs down here - well, you don't have to take it personally - it's just facts, we all loved (past-tense) Beanstalkd.

[jellisii]

I still advocate for a driver instead of committing to a single queue.

[niden]

Resolved.

Discussion for new component: https://github.com/phalcon/cphalcon/issues/13851

[zz]

Beanstalkd just released 1.11
https://github.com/beanstalkd/beanstalkd/releases/tag/v1.11

[maxttor]

buried early

[slothentic]

Really disappointed to see this was removed from Phalcon. I've used nearly every messaging system and beanstalkd has just the right amount of features and simplicity. RabbitMQ is likely overkill for most Phalcon projects. Removing this was a mistake.