Skip to content

Commit

Permalink
changed filtering logic to only use one filtering expression
Browse files Browse the repository at this point in the history
  • Loading branch information
@jandelgado
jandelgado committed Jul 14, 2018
1 parent 0dcdf9f commit 67c54eb
Show file tree
Hide file tree
Showing 7 changed files with 188 additions and 222 deletions.
173 changes: 167 additions & 6 deletions README.md
View file
Expand Up @@ -31,6 +31,14 @@ and exchanges, inspect broker.
* [Poor mans shovel](#poor-mans-shovel)
* [Close connection](#close-connection)
* [JSON message format](#json-message-format)
* [Filtering output of info command](#filtering-output-of-info-command)
* [Filtering expressions](#filtering-expressions)
* [Evaluation context](#evaluation-context)
* [Examples](#examples-1)
* [Type reference](#type-reference)
* [Exchange type](#exchange-type)
* [Queue type](#queue-type)
* [Binding type](#binding-type)
* [Build from source](#build-from-source)
* [Test data generator](#test-data-generator)
* [Author](#author)
Expand Down Expand Up @@ -77,13 +85,15 @@ See [below](#build-from-source) if you prefer to compile from source.
## Usage

```
rabtap - RabbitMQ message tap.
rabtap - RabbitMQ wire tap.
Usage:
rabtap -h|--help
rabtap tap EXCHANGES [--uri URI] [--saveto=DIR] [-jknv]
rabtap (tap --uri URI EXCHANGES)... [--saveto=DIR] [-jknv]
rabtap info [--api APIURI] [--consumers] [--stats] [--show-default] [-knv]
rabtap info [--api APIURI] [--consumers] [--stats]
[--filter EXPR]
[--omit-empty] [--show-default] [-knv]
rabtap pub [--uri URI] EXCHANGE [FILE] [--routingkey=KEY] [-jkv]
rabtap sub QUEUE [--uri URI] [--saveto=DIR] [-jkvn]
rabtap exchange create EXCHANGE [--uri URI] [--type TYPE] [-adkv]
Expand All @@ -109,7 +119,8 @@ Examples:
# use RABTAP_APIURI environment variable to specify mgmt api uri instead of --api
export RABTAP_APIURI=http://guest:guest@localhost:15672/api
rabtap info
rabtap conn close "172.17.0.1:40874 -> 172.17.0.2:5672"
rabtap info --filter "binding.Exchange == 'amq.topic'"
rabtap conn close "172.17.0.1:40874 -> 172.17.0.2:5672"
Options:
EXCHANGES comma-separated list of exchanges and binding keys,
Expand All @@ -125,13 +136,15 @@ Options:
-b, --bindingkey KEY binding key to use in bind queue command.
--consumers include consumers and connections in output of info command.
-d, --durable create durable exchange/queue.
--filter EXPR Filter for info command to filter queues (see README.md)
-h, --help print this help.
-j, --json print/save/publish message metadata and body to a
single JSON file. JSON body is base64 encoded. Otherwise
metadata and body (as-is) are saved separately.
-k, --insecure allow insecure TLS connections (no certificate check).
-n, --no-color don't colorize output (also environment variable NO_COLOR)
--reason=REASON reason why the connection was closed
-o, --omit-empty don't show echanges without bindings in info command.
--reason=REASON reason why the connection was closed
[default: closed by rabtap].
-r, --routingkey KEY routing key to use in publish mode.
--saveto DIR also save messages and metadata to DIR.
Expand Down Expand Up @@ -159,8 +172,8 @@ Rabtap understand the following commands:
`AD` (auto delete) and `I` (internal). The features of a queue are displayed
in square brackets with `D` (durable), `AD` (auto delete) and `EX`
(exclusive). If `--statistics` option is enabled, basic statistics are
included in the output.

included in the output. The `--filter` option allows to filter output. See
[filtering](#filtering-output-of-info-command) section for details.
* `queue` - create/bind/remove queue
* `exchange` - create/remove exhange
* `connection` - close connections
Expand Down Expand Up @@ -377,6 +390,154 @@ messages in the following format:

Note that in JSON mode, the `Body` is base64 encoded.

## Filtering output of info command

When your brokers topology is complex, the output of the `info` command can
become very bloated. The `--filter` helps you to narrow output to
the desired information.

### Filtering expressions

A filtering expression is a function that evaluates to `true` or `false` (i.e.
a *predicate*). Rabtap allows the specification of predicates to be applied
when printing queues using the `info` command. The output will only proceed
if the predicate evaluates to `true`.

Rabtap uses the [govalute](https://github.com/Knetic/govaluate) to evaluate the
predicate. This allows or complex expressions.

See [official govaluate
documentation](https://github.com/Knetic/govaluate/blob/master/MANUAL.md) for
further information.

#### Evaluation context

During evaluation the context (i.e. the current exchange, queue and binding) is
available in the expression as variables:

* the current exchange is bound to the variable [exchange](#exchange-type)
* the current queue is bound to the variable [queue](#queue-type)
* the curren binding is bound to the variable [binding](#binding-type)

#### Examples

The examples assume that `RABTAP_APIURI` environment variable points to the
broker to be used, e.g. `http://guest:guest@localhost:15672/api`).

* `rabtap info --filter "exchange.Name == 'amq.direct'" --omit-empty`: Print
only queues bound to exchange `amq.direct` and skip all empty exchanges.
* `rabtap info --filter "queue.Name =~ '.*test.*'" --omit-empty`: Print all
queues with `test` in their name.
* `rabtap info --filter "queue.Name =~ '.*test.*' && exchange.Type == 'topic'" --omit-empty`: Like
before, but consider only exchanges of type `topic`.

### Type reference

The types reflect more or less the JSON API objects of the [REST API of
RabbitMQ](https://rawcdn.githack.com/rabbitmq/rabbitmq-management/v3.7.7/priv/www/api/index.html)
transformed to golang types.

#### Exchange type

```
type Exchange struct {
Name string
Vhost string
Type string
Durable bool
AutoDelete bool
Internal bool
MessageStats struct {
PublishOut
PublishOutDetails struct {
Rate float64
}
PublishIn int
PublishInDetails struct {
Rate float64
}
}
}
```

#### Queue type

```
type Queue struct {
MessagesDetails struct {
Rate float64
}
Messages
MessagesUnacknowledgedDetails struct {
Rate float64
}
MessagesUnacknowledged int
MessagesReadyDetails struct {
Rate float64
}
MessagesReady int
ReductionsDetails struct {
Rate float64
}
Reductions int
Node string
Exclusive bool
AutoDelete bool
Durable bool
Vhost string
Name string
MessageBytesPagedOut int
MessagesPagedOut int
BackingQueueStatus struct {
Mode string
Q1 int
Q2 int
Q3 int
Q4 int
Len int
NextSeqID int
AvgIngressRate float64
AvgEgressRate float64
AvgAckIngressRate float64
AvgAckEgressRate float64
}
MessageBytesPersistent int
MessageBytesRAM int
MessageBytesUnacknowledged int
MessageBytesReady int
MessageBytes int
MessagesPersistent int
MessagesUnacknowledgedRAM int
MessagesReadyRAM int
MessagesRAM int
GarbageCollection struct {
MinorGcs int
FullsweepAfter int
MinHeapSize int
MinBinVheapSize int
MaxHeapSize int
}
State string
Consumers int
IdleSince string
Memory int
}
```

#### Binding type

```
type Binding struct {
Source string
Vhost string
Destination string
DestinationType string
RoutingKey string
PropertiesKey string
}
```

## Build from source

To build rabtap from source, you need [go](https://golang.org/) and the
Expand Down
42 changes: 12 additions & 30 deletions cmd/main/broker_info_printer.go
View file
Expand Up @@ -31,8 +31,8 @@ const (
{{- ""}} host='{{ .Connection.Host }}:{{ .Connection.Port }}',
{{- ""}} peer='{{ .Connection.PeerHost }}:{{ .Connection.PeerPort }}')`
tplExchange = `
{{- $printname := .Exchange.Name }}{{ if eq $printname "" }}{{ $printname := "(default)" }}{{end}}
{{- ExchangeColor $printname }} (exchange, type '{{ .Exchange.Type }}'
{{- if eq .Exchange.Name "" }}{{ ExchangeColor "(default)" }}{{ else }}{{ ExchangeColor .Exchange.Name }}{{ end }}
{{- "" }} (exchange, type '{{ .Exchange.Type }}'
{{- if and .Config.ShowStats .Exchange.MessageStats }}, in=(
{{- .Exchange.MessageStats.PublishIn }}, {{printf "%.1f" .Exchange.MessageStats.PublishInDetails.Rate}}/s) msg, out=(
{{- .Exchange.MessageStats.PublishOut }}, {{printf "%.1f" .Exchange.MessageStats.PublishOutDetails.Rate}}/s) msg
Expand All @@ -56,7 +56,6 @@ type BrokerInfoPrinterConfig struct {
ShowConsumers bool
ShowStats bool
QueueFilter Predicate
ExchangeFilter Predicate
OmitEmptyExchanges bool
NoColor bool
}
Expand Down Expand Up @@ -269,10 +268,11 @@ func (s BrokerInfoPrinter) createConnectionNodes(

func (s BrokerInfoPrinter) shouldDisplayQueue(
queue *rabtap.RabbitQueue,
exchange *rabtap.RabbitExchange,
binding *rabtap.RabbitBinding) bool {

// apply queue filter
params := map[string]interface{}{"queue": queue, "binding": binding}
// apply filter
params := map[string]interface{}{"queue": queue, "binding": binding, "exchange": exchange}
if res, err := s.config.QueueFilter.Eval(params); err != nil || !res {
if err != nil {
log.Warnf("error evaluating queue filter: %s", err)
Expand All @@ -285,6 +285,7 @@ func (s BrokerInfoPrinter) shouldDisplayQueue(

func (s BrokerInfoPrinter) createQueueNodeFromBinding(
binding *rabtap.RabbitBinding,
exchange *rabtap.RabbitExchange,
brokerInfo *rabtap.BrokerInfo) []*TreeNode {

// standard binding of queue to exchange
Expand All @@ -297,7 +298,7 @@ func (s BrokerInfoPrinter) createQueueNodeFromBinding(
queue = &rabtap.RabbitQueue{Name: binding.Destination}
}

if !s.shouldDisplayQueue(queue, binding) {
if !s.shouldDisplayQueue(queue, exchange, binding) {
return []*TreeNode{}
}

Expand Down Expand Up @@ -333,7 +334,7 @@ func (s BrokerInfoPrinter) createExchangeNode(
brokerInfo))
} else {
// queue to exchange binding
exchangeNode.AddList(s.createQueueNodeFromBinding(&binding, brokerInfo))
exchangeNode.AddList(s.createQueueNodeFromBinding(&binding, exchange, brokerInfo))
}
}
return exchangeNode
Expand All @@ -349,31 +350,9 @@ func (s BrokerInfoPrinter) shouldDisplayExchange(
return false
}

// apply exchange filter
params := map[string]interface{}{"exchange": exchange}
if res, err := s.config.ExchangeFilter.Eval(params); err != nil || !res {
if err != nil {
log.Warnf("error evaluating exchange filter: %s", err)
} else {
return false
}
}
return true
}

func (s BrokerInfoPrinter) getExchangesToDisplay(
exchanges []rabtap.RabbitExchange,
vhost string) []rabtap.RabbitExchange {

var result []rabtap.RabbitExchange
for _, exchange := range exchanges {
if s.shouldDisplayExchange(&exchange, vhost) {
result = append(result, exchange)
}
}
return result
}

// buildTree renders given brokerInfo into a tree:
// RabbitMQ-Host
// +--VHost
Expand All @@ -396,7 +375,10 @@ func (s BrokerInfoPrinter) buildTree(brokerInfo *rabtap.BrokerInfo,
for vhost := range uniqueVhosts(brokerInfo.Exchanges) {
vhNode := NewTreeNode(s.renderVhostAsString(vhost))
root.Add(vhNode)
for _, exchange := range s.getExchangesToDisplay(brokerInfo.Exchanges, vhost) {
for _, exchange := range brokerInfo.Exchanges {
if !s.shouldDisplayExchange(&exchange, vhost) {
continue
}
exNode := s.createExchangeNode(&exchange, brokerInfo)
if s.config.OmitEmptyExchanges && !exNode.HasChildren() {
continue
Expand Down
2 changes: 0 additions & 2 deletions cmd/main/broker_info_printer_test.go
View file
Expand Up @@ -125,7 +125,6 @@ func ExampleBrokerInfoPrinter_Print() {
ShowConsumers: true,
ShowDefaultExchange: false,
QueueFilter: TruePredicate,
ExchangeFilter: TruePredicate,
OmitEmptyExchanges: false,
NoColor: true},
)
Expand Down Expand Up @@ -183,7 +182,6 @@ func ExampleBrokerInfoPrinter_printWithQueueFilter() {
ShowConsumers: true,
ShowDefaultExchange: false,
QueueFilter: queueFilter,
ExchangeFilter: TruePredicate,
OmitEmptyExchanges: true,
NoColor: true},
)
Expand Down

0 comments on commit 67c54eb

Please sign in to comment.