Every repository with this icon (
) is private.
Every repository with this icon (
) is public.
| Description: | Beanstalk is a simple, fast work queue. edit |
-
Can be used, e.g., for deduplication of work.
Comments
-
This falls outside the main scope of beanstalkd's purpose, but I think it opens up a lot of interesting possibilities.
Such tubes can be marked as special by prefixing the name with a special char. I think "~" would work fine as that char.
Comments
-
2 votes
Client disconnect shouldn't auto-release reserved job onto front of queue.
0 comments Created 5 months ago by pdaCurrently when a client disconnects while it has a reserved job, that job is instantly released onto the front of the queue, without respect of its remaining TTR.
This means that:
- A worker executing long jobs cannot disconnect/reconnect to beanstalkd during execution. Perhaps that would make it too hard to track which client owns which job reservation though...
- A job that causes a worker to crash constantly hogs the front of the queue, blocking subsequent jobs from coming through.
Perhaps auto-release due to disconnect should at least release onto the end of the queue, at a lower priority, or with a delay..?
Comments
-
It would be nice to have a client that just forwards jobs from one beanstalkd instance to another. See http://groups.google.com/group/beanstalk-talk/t/984bfc8037847e53 for some motivation for this setup.
Comments
-
Probably just means adding a CRC to stored records.
Comments
-
When submitting jobs around 2MB, beanstalkd gets really slow. See here for the original report:
http://groups.google.com/group/beanstalk-talk/t/f27bd53d62d23f0c
Comments
-
Given a job id, I would like to be able to reprioritize it. Currently I can only achieve this by calling peek, delete, then recreating the job.
One possibility would be to allow release to be called without first having reserved the job. Currently the interface for delete and release is inconsistent:
put 0 0 60 3 foo INSERTED 1 delete 1 DELETED put 0 0 60 3 bar INSERTED 2 release 2 100 0 NOT_FOUND delete 2 DELETEDComments
-
0 votes
Broken pipe error and heavy CPU use with misbehaving clients
4 comments Created about 1 month ago by pkieltyka| See below... This happens when clients send reserve | commands constantly on an empty queue.
I'm running the master beanstalkd on Snow Leopard and it's been working well, except I've come across the error:
./beanstalkd: prot.c:672 in check_err: writev(): Broken pipe ./beanstalkd: prot.c:672 in check_err: writev(): Broken pipe ..
Without the clients doing anything else to the queue, this causes beanstalkd to run at 97% CPU usage. I'm not sure what caused this specifically but it occurs when I have one client adding things onto the queue and another removing them (about 10 items per second).
Cheers
Peter
Comments
Another thing is I noticed that beanstalkd's memory usage increases slowly and never releases it's allocated memory. Perhaps I'm doing something wrong or its my environment, but I'm interested to know the expected behaviour. By the way, I am using a persistent queue via the binlog.
Something is wrong if it takes 97% cpu with so little activity. I will try to reproduce it.
I looked into this some more and it results from my client code doing crazy things, it was sending "reserve" commands in an infinite loop without any jobs on the queue. I've fixed my code since then and it's been working great.
Perhaps though, you could ignore subsequent reserve commands for a particular connection .. or something.
Thx.
-
0 votes
0 comments Created about 1 month ago by kraccept LF in addition to CR LF to terminate commandskrxComments
-
0 votes
1 comment Created 28 days ago by kradd underscore "_" to list of valid tube name characterskrx -
Our jankity shell scripts don't work reliably with all versions of netcat.
I'm leaning toward a really simple, custom python framework. But if there are any really good protocol testing tools out there, I'd love to know.
Comments
Yes please! ;-)
+1 I saw that in the roadmap, and know exactly what I'd use it for.
What would be the difference between this and the existing job ids? If we're suggesting features I'd love deleting or re-prioritising jobs by id - related?
You have no control of existing job ids. The idea is to be able to assign job ids on the client. This way, multiple (distributed) clients could interact (reschedule, cancel, pop, etc) the same job.
You can already delete any job by id. The only exception: you cannot delete a job some other client currently has reserved.
Re-prioritising jobs is a nice idea. Feel free to open a ticket for it.
kr, you can delete by ID, but it assumes I know the id. I want the following scenario: client A queues new job (name: job1), client B, which knows nothing about client A, reaches in and modifies the payload / time to run / etc for job1. To accomplish this, the name has to depend on some external knowledge (not beanstalkd assigning job ids). For example.. let the job id be an email of the user, or a qualified url, etc.
Got it. That's exactly what this ticket is for.
I should try to be more clear about who I'm replying to. Sorry about that.
Ok now I understand the motivation for this feature, and it's another +1 from me. Definitely opens up useful possibilities. I'll open another ticket for reprioritizing jobs.