Browse files

incorporate matt sanford's feedback.

  • Loading branch information...
1 parent 465baeb commit e45676a7a99d51bfbf7639c30f15ea2a4caf0fef Robey Pointer committed Aug 13, 2009
Showing with 25 additions and 5 deletions.
  1. +1 −1
  2. +23 −3 docs/
  3. +1 −1 src/main/scala/net/lag/kestrel/PersistentQueue.scala
@@ -38,7 +38,7 @@ Kestrel is:
- small
- Currently about 1.5K lines of scala (including comments), because it relies
+ Currently about 2K lines of scala (including comments), because it relies
on Apache Mina (a rough equivalent of Danger's ziggurat or Ruby's
EventMachine) and actors -- and frankly because Scala is extremely
@@ -15,7 +15,11 @@ Generally queue names should be limited to alphanumerics `[A-Za-z0-9]`, dash
(`-`) and underline (`_`). In practice, kestrel doesn't enforce any
restrictions other than the name can't contain slash (`/`) because that can't
be used in filenames, squiggle (`~`) because it's used for temporary files,
-and dot (`.`) because it's reserved for future use.
+plus (`+`) because it's used for fanout queues, and dot (`.`) because it's
+reserved for future use. Queue names are case-sensitive, but if you're running
+kestrel on OS X or Windows, you will want to refrain from taking advantage of
+this, since the journal filenames on those two platforms are *not*
A cluster of kestrel servers is like a memcache cluster: the servers don't
know about each other, and don't do any cross-communication, so you can add as
@@ -190,7 +194,7 @@ Memcache commands
Add an item to a queue. It may fail if the queue has a size or item limit
and it's full.
-- `GET <queue-name>`
+- `GET <queue-name>[options]`
Remove an item from a queue. It will return an empty response immediately if
the queue is empty. The queue name may be followed by options separated
@@ -222,6 +226,14 @@ Memcache commands
Return the first available item from the queue, if there is one, but don't
remove it. You can't combine this with any of the reliable read options.
+ For example, to open a new read, waiting up to 250msec for an item:
+ GET work/t=500/open
+ Or to close an existing read and open a new one:
+ GET work/close/open
- `DELETE <queue-name>`
Drop a queue, discarding any items in it, and deleting any associated
@@ -293,7 +305,8 @@ happens in two stages, using the `/open` and `/close` options to `GET`.
When `/open` is used, and an item is available, kestrel will remove it from
the queue and send it to the client as usual. But it will also set the item
aside. If a client disconnects while it has an open read, the item is put back
-into the queue, at the head, so it will be the next item fetched.
+into the queue, at the head, so it will be the next item fetched. Only one
+item can be "open" per client connection.
A previous open request is closed with `/close`. The server will reject any
attempt to open another read when one is already open, but it will ignore
@@ -312,6 +325,13 @@ To use this tactic successfully, work items should be idempotent, meaning the
work could be done 2 or 3 times and have the same effect as if it had been
done only once (except wasting some resources).
+ GET dirty_jobs/close/open
+ (receives job 1)
+ GET dirty_jobs/close/open
+ (closes job 1, receives job 2)
+ ...etc...
Server stats
@@ -108,7 +108,7 @@ class PersistentQueue(persistencePath: String, val name: String,
// whether to sync the journal after each transaction
val syncJournal = overlay(PersistentQueue.syncJournal)
- // move
+ // (optional) move expired items over to this queue
val expiredQueue = overlay(PersistentQueue.expiredQueue)
// clients waiting on an item in this queue

0 comments on commit e45676a

Please sign in to comment.