Queries

There is an experimental, work-in-progress feature which allows you to describe a series of channel operations on a probe using a specialized “query language” when subscribing to a probe-channel endpoint. For example, the query lamina:test.output.where(failures > 0).rate() subscribes to the :lamina:test probe, isolates the :output key in that map, filters only those outputs whose @:failure sub-key is positive, and finally produces a channel which aggregates such probes and periodically emits a message saying how many there have been since the last probe.

That’s a bit much to swallow all at once, though, so let’s look in more detail at what features are available and how it all fits together.

First, each selector starts with the name of a probe (which may include * wildcards to match multiple probes). In our example, that was lamina:test. After that, there can be any number of transformations, each of which looks like a . member access on the previous channel. Our first transformation was of the simplest sort, a lookup. The transformation x.k simply takes all messages from the channel x, looks up :k in them as a map, and forwards them to its output. Note that there is a special lookup, x._, which simply returns x without looking anything up in it. This is obviously not useful on its own, but can be handy in scenarios where a lookup is required but you want to instead use the whole object.

The other transformations are more complicated, and more varied. Called “operators”, they take the general form channel.operator(args) and produce some new channel. The set of operators is open-ended, and can be extended by your program via a defoperator form, and thus they cannot all be listed here. However, we will look at examples of a few of the most common operators as we look at the different sorts of arguments that can be passed to an operator.

select

The select operator is a bit like clojure’s let form, and a bit like its select-keys function. For example, foo.select(a: x, b: y.z) accepts data looking like {:x 1, :y {:z 10}} and transforms it to {:a 1, :b 10}. Any keys not named in the select are dropped from the output object. These a: x groupings are called “pairs”, and are passed without modification to the operator; in this case select is responsible for taking the pairs {:a x} and {:b y.z} and producing a function which does the necessary looking-up and renaming. Note that, aince the value part of the pair is a lookup, you can use the special _ lookup to bind the entire incoming map. For example, x.select(important-stuff: a, whole-thing: _) will read the :a key and store it under :important-stuff, and put the entire map under :whole-thing in case it is needed after all.

where

where is like SQL’s WHERE operator, or clojure’s filter function. It takes a lookup, a comparison function (any of = < > ~=), and a value to compare to. It returns only those objects for which the comparison of the given lookup with the value is true. For example, xs.where(age < 10) selects those messages whose age is less than ten. Comparison operators must be specified infix. where supports multiple comparison clauses, separated by whitespace or commas.

periodic operators

There are a number of periodic operators, which consume all incoming messages without rebroadcasting anything, but at periodic intervals emit some kind of summary of all messages in the last period. All these operators take a period: <milliseconds> pair as an optional argument, which dictates their period. Some particularly useful pre-defined periodic operators are: sample, which retains only the last message from each period and re-transmits it; sum, which requires its inputs be integers and emits the sum over each period; and rate, which emits a count of received messages for each period.

group-by

Another periodic operator, group-by is special enough to deserve its own section, because it supports a parameter format which we have so far not discussed: a “tuple” is denoted by square brackets, such as [a b c]. But before discussing what a tuple means to group-by, let’s look at a simpler example, of ch.group-by(a). It emits, every period, a map similar to those produced by clojure.core/group-by. For example, if it receives three messages [{:a 1 :other 10}, {:a 1 :other 20}, {:a 2 :other 30}] in a period, then it will glom them all together into the single message {1 [{:a 1 :other 10} {:a 1 :other 20}], 2 [{:a 2 :other 30}]}. When given a tuple such as [a b], group-by will use (map the-message [:a :b]) as the key to group by, so the resulting maps will have keys like [5 "dog"] for inputs like {:a 5, :b "dog", :c "woof"}.