Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 598 lines (434 sloc) 24.689 kB
0c1b88b lots more docs.
Robey Pointer authored
1
62482e2 cleanup.
Robey Pointer authored
2 A working guide to kestrel
3 ==========================
0c1b88b lots more docs.
Robey Pointer authored
4
1ae7657 @zuercher kestrel: server status & zookeeper server sets
zuercher authored
5 Kestrel is a very simple message queue that runs on the JVM. It supports
6 multiple protocols:
e224a6f @zuercher minor docs/guide.md update
zuercher authored
7
8 - memcache: the memcache protocol, with some extensions
9 - thrift: Apache Thrift-based RPC
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
10 - text: a simple text-based protocol
0c1b88b lots more docs.
Robey Pointer authored
11
12 A single kestrel server has a set of queues identified by a name, which is
13 also the filename of that queue's journal file (usually in
14 `/var/spool/kestrel`). Each queue is a strictly-ordered FIFO of "items" of
15 binary data. Usually this data is in some serialized format like JSON or
16 ruby's marshal format.
17
d6199a9 explain queue names.
Robey Pointer authored
18 Generally queue names should be limited to alphanumerics `[A-Za-z0-9]`, dash
19 (`-`) and underline (`_`). In practice, kestrel doesn't enforce any
20 restrictions other than the name can't contain slash (`/`) because that can't
21 be used in filenames, squiggle (`~`) because it's used for temporary files,
e45676a incorporate matt sanford's feedback.
Robey Pointer authored
22 plus (`+`) because it's used for fanout queues, and dot (`.`) because it's
23 reserved for future use. Queue names are case-sensitive, but if you're running
24 kestrel on OS X or Windows, you will want to refrain from taking advantage of
25 this, since the journal filenames on those two platforms are *not*
26 case-sensitive.
d6199a9 explain queue names.
Robey Pointer authored
27
0c1b88b lots more docs.
Robey Pointer authored
28 A cluster of kestrel servers is like a memcache cluster: the servers don't
29 know about each other, and don't do any cross-communication, so you can add as
1ae7657 @zuercher kestrel: server status & zookeeper server sets
zuercher authored
30 many as you like. The simplest clients have a list of all servers in the
31 cluster, and pick one at random for each operation. In this way, each queue
32 appears to be spread out across every server, with items in a loose ordering.
33 More advanced clients can find kestrel servers via ZooKeeper.
0c1b88b lots more docs.
Robey Pointer authored
34
58746d2 more docs!
Robey Pointer authored
35 When kestrel starts up, it scans the journal folder and creates queues based
36 on any journal files it finds there, to restore state to the way it was when
37 it last shutdown (or was killed or died). New queues are created by referring
38 to them (for example, adding or trying to remove an item). A queue can be
39 deleted with the "delete" command.
40
0c1b88b lots more docs.
Robey Pointer authored
41
42 Configuration
43 -------------
44
70ef6d0 add some more docs, and make the guide refer to them.
Robey Pointer authored
45 The config files for kestrel are scala expressions loaded at runtime, usually
46 from `production.scala`, although you can use `development.scala` by passing
47 `-Dstage=development` to the java command line.
48
49 The config file evaluates to a `KestrelConfig` object that's used to configure
50 the server as a whole, a default queue, and any overrides for specific named
51 queues. The fields on `KestrelConfig` are documented here with their default
52 values:
e224a6f @zuercher minor docs/guide.md update
zuercher authored
53 [KestrelConfig.html](http://robey.github.com/kestrel/api/main/api/net/lag/kestrel/config/KestrelConfig.html)
0c1b88b lots more docs.
Robey Pointer authored
54
55 To confirm the current configuration of each queue, send "dump_config" to
56 a server (which can be done over telnet).
57
58 To reload the config file on a running server, send "reload" the same way.
70ef6d0 add some more docs, and make the guide refer to them.
Robey Pointer authored
59 You should immediately see the changes in "dump_config", to confirm. Reloading
1ae7657 @zuercher kestrel: server status & zookeeper server sets
zuercher authored
60 will only affect queue and alias configuration, not global server configuration.
61 To change the server configuration, restart the server.
0c1b88b lots more docs.
Robey Pointer authored
62
70ef6d0 add some more docs, and make the guide refer to them.
Robey Pointer authored
63 Logging is configured according to `util-logging`. The logging configuration
64 syntax is described here:
fd3f9d4 add more docs and clean them up a bit
Robey Pointer authored
65 [util-logging](https://github.com/twitter/util/blob/master/util-logging/README.markdown)
0c1b88b lots more docs.
Robey Pointer authored
66
70ef6d0 add some more docs, and make the guide refer to them.
Robey Pointer authored
67 Per-queue configuration is documented here:
e224a6f @zuercher minor docs/guide.md update
zuercher authored
68 [QueueBuilder.html](http://robey.github.com/kestrel/api/main/api/net/lag/kestrel/config/QueueBuilder.html)
fd3f9d4 add more docs and clean them up a bit
Robey Pointer authored
69
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
70 Queue alias configuration is documented here:
71 [AliasBuilder.html](http://robey.github.com/kestrel/api/main/api/net/lag/kestrel/config/AliasBuilder.html)
72
fd3f9d4 add more docs and clean them up a bit
Robey Pointer authored
73
74 Full queues
75 -----------
76
77 A queue can have the following limits set on it:
78
79 - `maxItems` - total items in the queue
80 - `maxSize` - total bytes of data in the items in the queue
81
82 If either of these limits is reached, no new items can be added to the queue.
83 (Clients will receive an error when trying to add.) If you set
84 `discardOldWhenFull` to true, then all adds will succeed, and the oldest
85 item(s) will be silently discarded until the queue is back within the item
86 and size limits.
87
88 `maxItemSize` limits the size of any individual item. If an add is attempted
89 with an item larger than this limit, it always fails.
74f21d8 document expiration_timer_frequency_seconds, max_item_size, and move_…
Robey Pointer authored
90
0c1b88b lots more docs.
Robey Pointer authored
91
92 The journal file
93 ----------------
94
95 The journal file is the only on-disk storage of a queue's contents, and it's
96 just a sequential record of each add or remove operation that's happened on
97 that queue. When kestrel starts up, it replays each queue's journal to build
98 up the in-memory queue that it uses for client queries.
99
58746d2 more docs!
Robey Pointer authored
100 The journal file is rotated in one of two conditions:
0c1b88b lots more docs.
Robey Pointer authored
101
1531d0d finish cleaning up the guide.
Robey Pointer authored
102 1. the queue is empty and the journal is larger than `defaultJournalSize`
0c1b88b lots more docs.
Robey Pointer authored
103
1531d0d finish cleaning up the guide.
Robey Pointer authored
104 2. the journal is larger than `maxJournalSize`
0c1b88b lots more docs.
Robey Pointer authored
105
1531d0d finish cleaning up the guide.
Robey Pointer authored
106 For example, if `defaultJournalSize` is 16MB (the default), then if the queue
107 is empty and the journal is larger than 16MB, it will be truncated into a new
108 (empty) file. If the journal is larger than `maxJournalSize` (1GB by default),
109 the journal will be rewritten periodically to contain just the live items.
0c1b88b lots more docs.
Robey Pointer authored
110
1531d0d finish cleaning up the guide.
Robey Pointer authored
111 You can turn the journal off for a queue (`keepJournal` = false) and the queue
0c1b88b lots more docs.
Robey Pointer authored
112 will exist only in memory. If the server restarts, all enqueued items are
1531d0d finish cleaning up the guide.
Robey Pointer authored
113 lost. You can also force a queue's journal to be sync'd to disk periodically,
114 or even after every write operation, at a performance cost, using
115 `syncJournal`.
0c1b88b lots more docs.
Robey Pointer authored
116
1531d0d finish cleaning up the guide.
Robey Pointer authored
117 If a queue grows past `maxMemorySize` bytes (128MB by default), only the
0c1b88b lots more docs.
Robey Pointer authored
118 first 128MB is kept in memory. The journal is used to track later items, and
119 as items are removed, the journal is played forward to keep 128MB in memory.
120 This is usually known as "read-behind" mode, but Twitter engineers sometimes
121 refer to it as the "square snake" because of the diagram used to brainstorm
122 the implementation. When a queue is in read-behind mode, removing an item will
123 often cause 2 disk operations instead of one: one to record the remove, and
124 one to read an item in from disk to keep 128MB in memory. This is the
125 trade-off to avoid filling memory and crashing the JVM.
126
127
58746d2 more docs!
Robey Pointer authored
128 Item expiration
129 ---------------
0c1b88b lots more docs.
Robey Pointer authored
130
58746d2 more docs!
Robey Pointer authored
131 When they come from a client, expiration times are handled in the same way as
132 memcache: if the number is small (less than one million), it's interpreted as
133 a relative number of seconds from now. Otherwise it's interpreted as an
134 absolute unix epoch time, in seconds since the beginning of 1 January 1970
135 GMT.
0c1b88b lots more docs.
Robey Pointer authored
136
58746d2 more docs!
Robey Pointer authored
137 Expiration times are immediately translated into an absolute time, in
1531d0d finish cleaning up the guide.
Robey Pointer authored
138 *milliseconds*, and if it's further in the future than the queue's `maxAge`,
a3c112f fix typos
Robey Pointer authored
139 the `maxAge` is used instead. An expiration of 0, which is usually the
58746d2 more docs!
Robey Pointer authored
140 default, means an item never expires.
0c1b88b lots more docs.
Robey Pointer authored
141
58746d2 more docs!
Robey Pointer authored
142 Expired items are flushed from a queue whenever a new item is added or
fd3f9d4 add more docs and clean them up a bit
Robey Pointer authored
143 removed. Additionally, if the global config option `expirationTimerFrequency`
144 is set, a background thread will periodically remove expired items from the
145 head of each queue. The provided `production.conf` sets this to one second.
146 If this is turned off, an idle queue won't have any items expired, but you
147 can still trigger a check by doing a "peek" on it.
148
149 Normally, expired items are discarded. If `expireToQueue` is set, then
150 expired items are moved to the specified queue just as if a client had put
151 it there. The item is added with no expiration time, but that can be
152 overridden if the new queue has a default expiration policy.
153
154 To prevent stalling the server when it encounters a swarm of items that all
155 expired at the same time, `maxExpireSweep` limits the number of items that
156 will be removed by the background thread in a single round. This is primarily
157 useful as a throttling mechanism when using a queue as a way to delay work.
74f21d8 document expiration_timer_frequency_seconds, max_item_size, and move_…
Robey Pointer authored
158
7841145 @gphat Expiring queues.
gphat authored
159 Queue expiration
160 ----------------
161
e224a6f @zuercher minor docs/guide.md update
zuercher authored
162 Whole queues can be configured to expire as well. If `maxQueueAge` is set
52c5fac @gphat Modify various bit after review by robey.
gphat authored
163 `expirationTimerFrequency` is used to check the queue age. If the queue is
164 empty, and it has been longer than `maxQueueAge` since it was created then
165 the queue will be deleted.
0c1b88b lots more docs.
Robey Pointer authored
166
e224a6f @zuercher minor docs/guide.md update
zuercher authored
167
58746d2 more docs!
Robey Pointer authored
168 Fanout Queues
169 -------------
170
171 If a queue name has a `+` in it (like "`orders+audit`"), it's treated as a
172 fanout queue, using the format `<parent>+<child>`. These queues belong to a
173 parent queue -- in this example, the "orders" queue. Every item written into
174 a parent queue will also be written into each of its children.
175
176 Fanout queues each have their own journal file (if the parent queue has a
177 journal file) and otherwise behave exactly like any other queue. You can get
178 and peek and even add items directly to a child queue if you want. It uses the
179 parent queue's configuration instead of having independent child queue
180 configuration blocks.
181
182 When a fanout queue is first referenced by a client, the journal file (if any)
183 is created, and it will start receiving new items written to the parent queue.
184 Existing items are not copied over. A fanout queue can be deleted to stop it
185 from receiving new items.
186
6993502 Document fanoutOnly.
Cory Watson authored
187 `fanoutOnly` may be set to true if the queue in question will only serve write
188 point for fanout queues. No journal file will be kept for the parent, only
189 for the child queues. This saves the overhead of writing to the parent and
190 removes the need to empty it. Note that setting `fanoutOnly` to true and
191 having no fanouts for the queue effectively makes it a black hole.
e224a6f @zuercher minor docs/guide.md update
zuercher authored
192
ff976b0 @zuercher introduce queue aliases (including stats)
zuercher authored
193 Queue Aliases
194 -------------
195
196 Queue aliases are somewhat similar to fanout queues, but without a required
197 naming convention or implicit creation of child queues. A queue alias can
198 only be used in set operations. Kestrel responds to attempts to retrieve
199 items from the alias as if it were an empty queue. Delete and flush requests
200 are also ignored.
58746d2 more docs!
Robey Pointer authored
201
e224a6f @zuercher minor docs/guide.md update
zuercher authored
202
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
203 Protocols
204 ---------
205
206 Kestrel supports three protocols: memcache, thrift and text. The
207 [Finagle project](http://twitter.github.com/finagle/) can be used to connect clients
208 to a Kestrel server via the memcache or thrift protocols.
bb7eb5e rename xid to id in the thrift interface, and link it from the docs
Robey Pointer authored
209
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
210 ### Thrift
211 ----------
212
213 The thrift protocol is documented in the thrift IDL:
bb7eb5e rename xid to id in the thrift interface, and link it from the docs
Robey Pointer authored
214 [kestrel.thrift](https://github.com/robey/kestrel/blob/master/src/main/thrift/kestrel.thrift)
215
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
216 Reliable reads via the thrift protocol are specified by indicating how long the server
217 should wait before aborting the unacknowledged read.
bb7eb5e rename xid to id in the thrift interface, and link it from the docs
Robey Pointer authored
218
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
219
220 ### Memcache
221 ------------
222
223 The official memcache protocol is described here:
224 [protocol.txt](https://github.com/memcached/memcached/blob/master/doc/protocol.txt)
225
226 The kestrel implementation of the memcache protocol commands is described below.
58746d2 more docs!
Robey Pointer authored
227
228 - `SET <queue-name> <flags (ignored)> <expiration> <# bytes>`
229
230 Add an item to a queue. It may fail if the queue has a size or item limit
231 and it's full.
232
e45676a incorporate matt sanford's feedback.
Robey Pointer authored
233 - `GET <queue-name>[options]`
66c498b docs!
Robey Pointer authored
234
58746d2 more docs!
Robey Pointer authored
235 Remove an item from a queue. It will return an empty response immediately if
236 the queue is empty. The queue name may be followed by options separated
237 by `/`:
238
66c498b docs!
Robey Pointer authored
239 - `/t=<milliseconds>`
240
241 Wait up to a given time limit for a new item to arrive. If an item arrives
242 on the queue within this timeout, it's returned as normal. Otherwise,
243 after that timeout, an empty response is returned.
244
245 - `/open`
246
247 Tentatively remove an item from the queue. The item is returned as usual
248 but is also set aside in case the client disappears before sending a
249 "close" request. (See "Reliable Reads" below.)
58746d2 more docs!
Robey Pointer authored
250
66c498b docs!
Robey Pointer authored
251 - `/close`
58746d2 more docs!
Robey Pointer authored
252
66c498b docs!
Robey Pointer authored
253 Close any existing open read. (See "Reliable Reads" below.)
58746d2 more docs!
Robey Pointer authored
254
66c498b docs!
Robey Pointer authored
255 - `/abort`
58746d2 more docs!
Robey Pointer authored
256
66c498b docs!
Robey Pointer authored
257 Cancel any existing open read, returing that item to the head of the
258 queue. It will be the next item fetched. (See "Reliable Reads" below.)
58746d2 more docs!
Robey Pointer authored
259
66c498b docs!
Robey Pointer authored
260 - `/peek`
58746d2 more docs!
Robey Pointer authored
261
66c498b docs!
Robey Pointer authored
262 Return the first available item from the queue, if there is one, but don't
263 remove it. You can't combine this with any of the reliable read options.
58746d2 more docs!
Robey Pointer authored
264
d1b4ee7 @rtyler Update guide.md
rtyler authored
265 For example, to open a new read, waiting up to 500msec for an item:
e45676a incorporate matt sanford's feedback.
Robey Pointer authored
266
267 GET work/t=500/open
268
269 Or to close an existing read and open a new one:
270
271 GET work/close/open
272
58746d2 more docs!
Robey Pointer authored
273 - `DELETE <queue-name>`
274
275 Drop a queue, discarding any items in it, and deleting any associated
276 journal files.
277
278 - `FLUSH <queue-name>`
279
280 Discard all items remaining in this queue. The queue remains live and new
281 items can be added. The time it takes to flush will be linear to the current
282 queue size, and any other activity on this queue will block while it's being
283 flushed.
284
285 - `FLUSH_ALL`
286
287 Discard all items remaining in all queues. The queues are flushed one at a
288 time, as if kestrel received a `FLUSH` command for each queue.
0c1b88b lots more docs.
Robey Pointer authored
289
58746d2 more docs!
Robey Pointer authored
290 - `VERSION`
291
292 Display the kestrel version in a way compatible with memcache.
293
294 - `SHUTDOWN`
295
296 Cleanly shutdown the server and exit.
297
298 - `RELOAD`
299
300 Reload the config file and reconfigure all queues. This should have no
301 noticable effect on the server's responsiveness.
302
303 - `STATS`
304
305 Display server stats in memcache style. They're described below.
306
307 - `DUMP_STATS`
308
309 Display server stats in a more readable style, grouped by queue. They're
310 described below.
311
55c7342 allow memcache protocol to have an optional max_items on "monitor".
Robey Pointer authored
312 - `MONITOR <queue-name> <seconds> [max-items]`
1531d0d finish cleaning up the guide.
Robey Pointer authored
313
e224a6f @zuercher minor docs/guide.md update
zuercher authored
314 Monitor a queue for a time, fetching any new items that arrive, up to an
315 optional maximum number of items. Clients are queued in a fair fashion,
316 per-item, so many clients may monitor a queue at once. After the given
317 timeout, a separate `END` response will signal the end of the monitor
318 period. Any fetched items are open transactions (see "Reliable Reads"
319 below), and should be closed with `CONFIRM`.
1531d0d finish cleaning up the guide.
Robey Pointer authored
320
321 - `CONFIRM <queue-name> <count>`
322
323 Confirm receipt of `count` items from a queue. Usually this is the response
324 to a `MONITOR` command, to confirm the items that arrived during the monitor
325 period.
326
1ae7657 @zuercher kestrel: server status & zookeeper server sets
zuercher authored
327 - `STATUS`
328
329 Displays the kestrel server's current status (see section on Server Status,
330 below).
331
332 - `STATUS <new-status>`
333
334 Switches the kestrel server's current status to the given status (see section
335 on Server Status, below).
336
74f21d8 document expiration_timer_frequency_seconds, max_item_size, and move_…
Robey Pointer authored
337
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
338 #### Reliable reads
339 -------------------
58746d2 more docs!
Robey Pointer authored
340
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
341 Note: this section is specific to the memcache protocol.
e224a6f @zuercher minor docs/guide.md update
zuercher authored
342
58746d2 more docs!
Robey Pointer authored
343 Normally when a client removes an item from the queue, kestrel immediately
344 discards the item and assumes the client has taken ownership. This isn't
345 always safe, because a client could crash or lose the network connection
346 before it gets the item. So kestrel also supports a "reliable read" that
347 happens in two stages, using the `/open` and `/close` options to `GET`.
348
349 When `/open` is used, and an item is available, kestrel will remove it from
350 the queue and send it to the client as usual. But it will also set the item
351 aside. If a client disconnects while it has an open read, the item is put back
e45676a incorporate matt sanford's feedback.
Robey Pointer authored
352 into the queue, at the head, so it will be the next item fetched. Only one
353 item can be "open" per client connection.
58746d2 more docs!
Robey Pointer authored
354
355 A previous open request is closed with `/close`. The server will reject any
356 attempt to open another read when one is already open, but it will ignore
357 `/close` if there's no open request, so that you can add `/close` to every
358 `GET` request for convenience.
359
360 If for some reason you want to abort a read without disconnecting, you can use
361 `/abort`. But because aborted items are placed back at the head of the queue,
362 this isn't a good way to deal with client errors. Since the error-causing item
363 will always be the next one available, you'll end up bouncing the same item
364 around between clients instead of making progress.
365
366 There's always a trade-off: either potentially lose items or potentially
367 receive the same item multiple times. Reliable reads choose the latter option.
368 To use this tactic successfully, work items should be idempotent, meaning the
369 work could be done 2 or 3 times and have the same effect as if it had been
370 done only once (except wasting some resources).
371
e45676a incorporate matt sanford's feedback.
Robey Pointer authored
372 Example:
373
374 GET dirty_jobs/close/open
375 (receives job 1)
376 GET dirty_jobs/close/open
377 (closes job 1, receives job 2)
378 ...etc...
58746d2 more docs!
Robey Pointer authored
379
74f21d8 document expiration_timer_frequency_seconds, max_item_size, and move_…
Robey Pointer authored
380
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
381 ### Text protocol
382 -----------------
383
384 Kestrel supports a limited, text-only protocol. You are encouraged to use the
385 memcache protocol instead.
386
387 The text protocol does not support reliable reads.
388
389
1ae7657 @zuercher kestrel: server status & zookeeper server sets
zuercher authored
390 Server Status
391 -------------
392
393 Each kestrel server maintains its current status. Normal statuses are
394
395 - `Up`: the server is available for all operations
396 - `ReadOnly`: the server is available for non-modifying operations only;
397 commands that modify queues (set, delete, flush) are rejected as
398 errors.
399 - `Quiescent`: the server rejects as an error operations on any queue. One
400 notable exception is transactions begun before the server entered
401 the quiesecent state may still be confirmed.
402
403 One additional status is `Down`, which is only used transiently when kestrel is
404 in the process of shutting down.
405
406 The server's current status is persisted (specified in
407 [KestrelConfig](http://robey.github.com/kestrel/api/main/api/net/lag/kestrel/config/KestrelConfig.html)).
408 When kestrel is restarted it automatically returns to it's previous status,
409 based on the value in the status file. If the status file does not exist or
410 cannot be read, kestrel uses a default status, also configured in KestrelConfig.
411
412 When changing from a less restrictive status to a more restrictive status
413 (e.g., from `Up` to `ReadOnly` or from `ReadOnly` to `Quiescent`), the
414 config option `statusChangeGracePeriod` determines how long kestrel will
415 continue to allow restricted operations to continue before it begins rejecting
416 them. This allows clients that are aware of the kestrel server's status a
417 grace period to learn the new status and cease the forbidden operations before
418 beginning to encounter errors.
419
420 ### ZooKeeper Server Sets
421 -------------------------
422
423 Kestrel uses Twitter's ServerSet library to support discovery of kestrel
424 servers allowing a given operation. The ServerSet class is documented here:
425 [ServerSet](http://twitter.github.com/commons/apidocs/index.html#com.twitter.common.zookeeper.ServerSet)
426
427 If the optional `zookeeper` field of `KestrelConfig` is specified, kestrel will
428 attempt to use the given configuration to join a logical set of kestrel servers.
429 The ZooKeeper host, port and other connection options are documented here:
430 [ZooKeeperBuilder](http://robey.github.com/kestrel/api/main/api/net/lag/kestrel/config/ZooKeeperBuilder.html)
431
432 Kestrel servers will join 0, 1, or 2 server sets depending on their current
433 status. When `Up`, the server joins two server sets: one for writes and one for
434 reads. When `ReadOnly`, the server joins only the read set. When `Quiescent`,
435 the server joins no sets. ZooKeeper-aware kestrel clients can watch the
436 server set for changes and adjust their connections accordingly. The
437 `statusChangeGracePeriod` configuration option may be used to allow clients
438 time to detect and react to the status change before they begin receiving
439 errors from kestrel.
440
441 The ZooKeeper path used to register the server set
442 is based on the `pathPrefix` option. Kestrel automatically appends `/write` and
443 `/read` to distinguish the write and read sets.
444
445 Kestrel advertises all of its endpoints in each server set that it joins.
446 The default endpoint is memcache, if configured. The default endpoint falls
447 back to the thrift endpoint and then the text protocol endpoint. All three
448 endpoints are advertised as additional endpoints under the names `memcache`,
449 `thrift` and `text`.
450
451 Consider setting the `defaultStatus` option to `Quiescent` to prevent kestrel
452 from prematurely advertising its status via ZooKeeper.
453
454 Installations that require additional customization of ZooKeeper credentials,
455 or other site-specific ZooKeeper initialization can override the
456 `clientInitializer` and `serverSetInitializer` options to invoke the
457 necessary site-specific code. The recommended implementation is to place
458 the site-specific code in its own JAR file, take the necessary steps to
459 include the JAR in kestrel's class path, and place as little logic as possible
460 in the kestrel configuration file.
461
462
58746d2 more docs!
Robey Pointer authored
463 Server stats
464 ------------
465
66c498b docs!
Robey Pointer authored
466 Global stats reported by kestrel are:
467
468 - `uptime` - seconds the server has been online
469 - `time` - current time in unix epoch
470 - `version` - version string, like "1.2"
471 - `curr_items` - total of items waiting in all queues
472 - `total_itmes` - total of items that have ever been added in this server's
473 lifetime
474 - `bytes` - total byte size of items waiting in all queues
475 - `curr_connections` - current open connections from clients
476 - `total_connections` - total connections that have been opened in this
477 server's lifetime
478 - `cmd_get` - total `GET` requests
479 - `cmd_set` - total `SET` requests
480 - `cmd_peek` - total `GET/peek` requests
481 - `get_hits` - total `GET` requests that received an item
482 - `get_misses` - total `GET` requests on an empty queue
483 - `bytes_read` - total bytes read from clients
484 - `bytes_written` - total bytes written to clients
0d8a507 @gphat Add more statistics along with tests and docs.
gphat authored
485 - `queue_creates` - total number of queues created
7841145 @gphat Expiring queues.
gphat authored
486 - `queue_deletes` - total number of queues deleted (includes expires)
487 - `queue_expires` - total number of queues expires
66c498b docs!
Robey Pointer authored
488
489 For each queue, the following stats are also reported:
490
491 - `items` - items waiting in this queue
492 - `bytes` - total byte size of items waiting in this queue
493 - `total_items` - total items that have been added to this queue in this
494 server's lifetime
495 - `logsize` - byte size of the queue's journal file
496 - `expired_items` - total items that have been expired from this queue in this
497 server's lifetime
498 - `mem_items` - items in this queue that are currently in memory
499 - `mem_bytes` - total byte size of items in this queue that are currently in
500 memory (will always be less than or equal to `max_memory_size` config for
501 the queue)
502 - `age` - time, in milliseconds, that the last item to be fetched from this
503 queue had been waiting; that is, the time between `SET` and `GET`; if the
504 queue is empty, this will always be zero
505 - `discarded` - number of items discarded because the queue was too full
506 - `waiters` - number of clients waiting for an item from this queue (using
507 `GET/t`)
721bf6b document the new stat.
Robey Pointer authored
508 - `open_transactions` - items read with `/open` but not yet confirmed
9afe5c4 @zuercher track number of transactional get requests and cancellations per queue
zuercher authored
509 - `transactions` - number of transactional get requests (irrespective of whether an
510 item was read or not)
511 - `canceled_transactions` - number of transactional get requests canceled (for any
512 reason)
5e4a681 @gphat Add create_time as a queue stat and document age_msec.
gphat authored
513 - `total_flushes` - total number of times this queue has been flushed
514 - `age_msec` - age of the last item read from the queue
515 - `create_time` - the time that the queue was created (in milliseconds since epoch)
66c498b docs!
Robey Pointer authored
516
c66bbb3 @zuercher more docs/guide.md cleanup
zuercher authored
517 Statistics may be retrieved by accessing the
518 [Ostrich admin HTTP service](https://github.com/twitter/ostrich) on the admin HTTP port.
519 For example: `http://kestrel.host:2223/stats.json?period=60`.
520
521 Statistics are also available via the memcache protocol using the `STATS` command.
522
523
66c498b docs!
Robey Pointer authored
524 Kestrel as a library
525 --------------------
526
527 You can use kestrel as a library by just sticking the jar on your classpath.
528 It's a cheap way to get a durable work queue for inter-process or inter-thread
529 communication. Each queue is represented by a `PersistentQueue` object:
530
1531d0d finish cleaning up the guide.
Robey Pointer authored
531 class PersistentQueue(val name: String, persistencePath: String,
532 @volatile var config: QueueConfig, timer: Timer,
533 queueLookup: Option[(String => Option[PersistentQueue])]) {
66c498b docs!
Robey Pointer authored
534
535 and must be initialized before using:
536
537 def setup(): Unit
58746d2 more docs!
Robey Pointer authored
538
66c498b docs!
Robey Pointer authored
539 specifying the path for the journal files (if the queue will be journaled),
1531d0d finish cleaning up the guide.
Robey Pointer authored
540 the name of the queue, a `QueueConfig` object (derived from `QueueBuilder`),
541 a timer for handling timeout reads, and optionally a way to find other named
542 queues (for `expireToQueue` support).
58746d2 more docs!
Robey Pointer authored
543
66c498b docs!
Robey Pointer authored
544 To add an item to a queue:
58746d2 more docs!
Robey Pointer authored
545
1531d0d finish cleaning up the guide.
Robey Pointer authored
546 def add(value: Array[Byte], expiry: Option[Time]): Boolean
58746d2 more docs!
Robey Pointer authored
547
66c498b docs!
Robey Pointer authored
548 It will return `false` if the item was rejected because the queue was full.
0c1b88b lots more docs.
Robey Pointer authored
549
66c498b docs!
Robey Pointer authored
550 Queue items are represented by a case class:
551
1531d0d finish cleaning up the guide.
Robey Pointer authored
552 case class QItem(addTime: Time, expiry: Option[Time], data: Array[Byte], var xid: Int)
66c498b docs!
Robey Pointer authored
553
554 and several operations exist to remove or peek at the head item:
555
556 def peek(): Option[QItem]
557 def remove(): Option[QItem]
558
559 To open a reliable read, set `transaction` true, and later confirm or unremove
560 the item by its `xid`:
561
562 def remove(transaction: Boolean): Option[QItem]
563 def unremove(xid: Int)
564 def confirmRemove(xid: Int)
565
1531d0d finish cleaning up the guide.
Robey Pointer authored
566 You can also asynchronously remove or peek at items using futures.
66c498b docs!
Robey Pointer authored
567
1531d0d finish cleaning up the guide.
Robey Pointer authored
568 def waitRemove(deadline: Option[Time], transaction: Boolean): Future[Option[QItem]]
569 def waitPeek(deadline: Option[Time]): Future[Option[QItem]]
66c498b docs!
Robey Pointer authored
570
571 When done, you should close the queue:
572
573 def close(): Unit
574 def isClosed: Boolean
575
576 Here's a short example:
577
1531d0d finish cleaning up the guide.
Robey Pointer authored
578 var queue = new PersistentQueue("work", "/var/spool/kestrel", config, timer, None)
66c498b docs!
Robey Pointer authored
579 queue.setup()
580
581 // add an item with no expiration:
582 queue.add("hello".getBytes, 0)
583
584 // start to remove it, then back out:
585 val item = queue.remove(true)
586 queue.unremove(item.xid)
587
588 // remove an item with a 500msec timeout, and confirm it:
1531d0d finish cleaning up the guide.
Robey Pointer authored
589 queue.waitRemove(500.milliseconds.fromNow, true)() match {
590 case None =>
591 println("nothing. :(")
592 case Some(item) =>
593 println("got: " + new String(item.data))
594 queue.confirmRemove(item.xid)
66c498b docs!
Robey Pointer authored
595 }
0c1b88b lots more docs.
Robey Pointer authored
596
66c498b docs!
Robey Pointer authored
597 queue.close()
Something went wrong with that request. Please try again.