- How does it work, high-level?
-
Beanstalkd is a big to-do list for your distributed application. If there is a unit of work that you want to defer to later (say, sending an email, pushing some data to a slow external service, pulling data from a slow external service, generating high-quality image thumbnails) you just describe it as a beanstalk “job”. Some processes (such as web request handlers), called “producers”, put jobs into the queue. Other processes, called “workers”, take jobs out of the queue and run them.
Also see http://nubyonrails.com/articles/about-this-blog-beanstalk-messaging-queue for an introduction to work-queues in general and beanstalkd in particular.
- Are the jobs persistent? What happens if the power goes out?
-
Currently beanstalkd is an in-memory-only queue; there’s no persistence. We started out this way for simplicity and speed, and we wanted to get a useful tool out the door as soon as possible.
Persistence is next on the to-do list. It’ll be implemented right after beanstalkd 1.0 is ready.
- Does beanstalk inherently support distributed servers?
-
Yes, although this is handled by the clients, just as with memcached. The beanstalkd server doesn’t know anything about other beanstalkd instances that are running.
- Does the protocol help implement blocking, or is that the domain of
the client libraries? In other words, do the client libraries
themselves still have to poll? -
Yes, there is a blacking “reserve” command in the beanstalk protocol. A client can simply try to read the response — this will block until a job is available and reserved for the client.