Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Sequence numbers / Synchronous clients? #27

Closed
thijsterlouw opened this Issue Dec 7, 2009 · 5 comments

Comments

Projects
None yet
3 participants

I just read the protocol discription for Beanstalkd. The project looks great and the protocol is really simple. Should be very straightforward to implement in a language such as Erlang. The only problem I could see is the lack of sequence numbers. This basically would force synchronous usage for the clients of Beanstalkd ? Do you have any plans for a binary protocol with sequence numbers?

If the responses always arrive in the same order of the requests (fifo), then sending/receiving async using a queue should also be a viable option?

Nonetheless, a binary protocol like memcached would also be great :)

Owner

kr commented Dec 10, 2009

There was discussion of a binary protocol back when beanstalkd was still young, but nothing came of it. I think it would be easier to revisit now that the text protocol changes so rarely.

And yes, you can use the existing protocol asynchronously, but you have to keep track of the order of commands you sent and responses you expect. In particular, you can send many commands before reading the replies (aka pipelining).

There is an Erlang client library at http://github.com/tim/erlang-beanstalk ; perhaps that will be illustrative. At some point I'd like to make a proof-of-concept in node.js as well.

thanks for the reply. I looked at that erlang-beanstalk client, but it's implementation is too simple for our purposes: Tim uses a (blocking) gen_tcp:send/recv combination, which means that each tcp connection (server process) can only handle one request. When you have many hundreds of requests coming quickly, this design will easily lead to timeouts in the gen_server:call(). Thus it is unsuitable for high request rates. Using async with seq. numbers is the easiest approach, but we will probably use pipelining instead for our client.

Contributor

JensRantil commented Apr 3, 2016

@thijsterlouw @kr I suggest this issue can be closed, right?

Owner

kr commented Apr 9, 2016

Yeah, this would involve a major redesign of the protocol. The existing protocol appears to be good enough in practice.

@kr kr closed this Apr 9, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment