Add Beanstalk Queue

[mrimann]

We're developing an application that generates tasks to a queue. There are multiple workers that resolve one task at a time.

To ensure the tasks are handled in a First-In-First-Out order - and to avoid handing the same task to two parallel running workers, we've settled for Beanstalk as the queue of choice.

I think it would be cool to have a simple possibility to include a beanstalk-service to ddev, similar to the additional services like Solr and Memcached as described on https://ddev.readthedocs.io/en/stable/users/extend/additional-services/

For now, a possible workaround would be to run the queue worker via CLI to test the execution of the single tasks. This is no problem via ddev ssh and issuing the command to the application. But writing new tasks to or fetching the next task from the queue would probably need changes to our application (to circumvent the situation that there just is no Beanstalk-service available within ddev).

[rfay]

So the additional services are just examples, please do give it a try based on the examples. What we like to do then is incubate the results on Stack Overflow with the "ddev" tag, https:// stackoverflow.com/tags/ddev - we're happy to help if you have questions along the way. The Drupal and TYPO3 slack #ddev channels are more interactive, but we're happy to help here as well as there and Stack Overflow.

[mrimann]

Thanks @rfay for the hint. I've added that now - and also provided documentation change + a config file template, see related commit.

If you think this goes into the right direction, I'd create a pull-request out of that change after some further testing on my side.

[rfay]

Excellent! We like to incubate new things in Stack Overflow because if every solution people ever came up with using ddev were in the docs they'd be infinitely long and complex, so we try to keep them more concise. So please start there, thanks!

[mrimann]

To be honest, I just don't get it... For me and from my perspective, it makes no sense to spread stuff across different places (e.g. the documentation (where I'd expect it) and on Stack Overflow).

That said, it would be time to update your CONTRIBUTING file - it doesn't mention Stack Overflow, and just tells to create issues - well as I did :-) SCNR.

If you fear the documentation get's too overloaded and unreadable, I agree - that could happen if it grows uncontrolled. But it might just need some changes to the structure (e.g. not listing all services (Solr / Memcache / Beanstalk) including all details on one single page - but to split that up to an overview page that explains the overview, and a list of ready-made integrations, linked to their respective detail page). I think that could avoid it getting too messy.

[rfay]

Great point about the CONTRIBUTING file.

We're always happy to have issues, there's no problem at all with that. Our support section does have it right. But I don't think the problem here is that we don't want a contribution or PR, just that we can't handle an infinite number of this type of doc, so try to stick with a few teaching ones. And it's really, really hard for you to keep up docs that are in the ddev repo, requires a PR every time, whereas little changes to a SO article are trivial and ongoing, and various people can contribute without friction.

[rfay]

@mrimann - Do you still use beanstalkd? This recipe actually ended up becoming an add-on, but I don't know if anybody ever used it. https://hub.docker.com/r/schickling/beanstalkd hasn't been touched for 6 years and is amd64-only.

Anyway, if you're interested in this topic...

• Consider demoting this add-on as of March 15, 2024 ddev-beanstalkd#9

[mrimann]

@rfay I'm not using it actively as far as I remember.

[rfay]

Thanks for that contribution so long ago. I imagine the add-on will be demoted :)