diff --git a/README.md b/README.md index d10ecfc..e8a3337 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,14 @@ and exchanges, inspect broker. * [Poor mans shovel](#poor-mans-shovel) * [Close connection](#close-connection) * [JSON message format](#json-message-format) +* [Filtering output of info command](#filtering-output-of-info-command) + * [Filtering expressions](#filtering-expressions) + * [Evaluation context](#evaluation-context) + * [Examples](#examples-1) + * [Type reference](#type-reference) + * [Exchange type](#exchange-type) + * [Queue type](#queue-type) + * [Binding type](#binding-type) * [Build from source](#build-from-source) * [Test data generator](#test-data-generator) * [Author](#author) @@ -77,13 +85,15 @@ See [below](#build-from-source) if you prefer to compile from source. ## Usage ``` -rabtap - RabbitMQ message tap. +rabtap - RabbitMQ wire tap. Usage: rabtap -h|--help rabtap tap EXCHANGES [--uri URI] [--saveto=DIR] [-jknv] rabtap (tap --uri URI EXCHANGES)... [--saveto=DIR] [-jknv] - rabtap info [--api APIURI] [--consumers] [--stats] [--show-default] [-knv] + rabtap info [--api APIURI] [--consumers] [--stats] + [--filter EXPR] + [--omit-empty] [--show-default] [-knv] rabtap pub [--uri URI] EXCHANGE [FILE] [--routingkey=KEY] [-jkv] rabtap sub QUEUE [--uri URI] [--saveto=DIR] [-jkvn] rabtap exchange create EXCHANGE [--uri URI] [--type TYPE] [-adkv] @@ -109,7 +119,8 @@ Examples: # use RABTAP_APIURI environment variable to specify mgmt api uri instead of --api export RABTAP_APIURI=http://guest:guest@localhost:15672/api rabtap info - rabtap conn close "172.17.0.1:40874 -> 172.17.0.2:5672" + rabtap info --filter "binding.Exchange == 'amq.topic'" + rabtap conn close "172.17.0.1:40874 -> 172.17.0.2:5672" Options: EXCHANGES comma-separated list of exchanges and binding keys, @@ -125,13 +136,15 @@ Options: -b, --bindingkey KEY binding key to use in bind queue command. --consumers include consumers and connections in output of info command. -d, --durable create durable exchange/queue. + --filter EXPR Filter for info command to filter queues (see README.md) -h, --help print this help. -j, --json print/save/publish message metadata and body to a single JSON file. JSON body is base64 encoded. Otherwise metadata and body (as-is) are saved separately. -k, --insecure allow insecure TLS connections (no certificate check). -n, --no-color don't colorize output (also environment variable NO_COLOR) - --reason=REASON reason why the connection was closed + -o, --omit-empty don't show echanges without bindings in info command. + --reason=REASON reason why the connection was closed [default: closed by rabtap]. -r, --routingkey KEY routing key to use in publish mode. --saveto DIR also save messages and metadata to DIR. @@ -159,8 +172,8 @@ Rabtap understand the following commands: `AD` (auto delete) and `I` (internal). The features of a queue are displayed in square brackets with `D` (durable), `AD` (auto delete) and `EX` (exclusive). If `--statistics` option is enabled, basic statistics are - included in the output. - + included in the output. The `--filter` option allows to filter output. See + [filtering](#filtering-output-of-info-command) section for details. * `queue` - create/bind/remove queue * `exchange` - create/remove exhange * `connection` - close connections @@ -377,6 +390,154 @@ messages in the following format: Note that in JSON mode, the `Body` is base64 encoded. +## Filtering output of info command + +When your brokers topology is complex, the output of the `info` command can +become very bloated. The `--filter` helps you to narrow output to +the desired information. + +### Filtering expressions + +A filtering expression is a function that evaluates to `true` or `false` (i.e. +a *predicate*). Rabtap allows the specification of predicates to be applied +when printing queues using the `info` command. The output will only proceed +if the predicate evaluates to `true`. + +Rabtap uses the [govalute](https://github.com/Knetic/govaluate) to evaluate the +predicate. This allows or complex expressions. + +See [official govaluate +documentation](https://github.com/Knetic/govaluate/blob/master/MANUAL.md) for +further information. + +#### Evaluation context + +During evaluation the context (i.e. the current exchange, queue and binding) is +available in the expression as variables: + +* the current exchange is bound to the variable [exchange](#exchange-type) +* the current queue is bound to the variable [queue](#queue-type) +* the curren binding is bound to the variable [binding](#binding-type) + +#### Examples + +The examples assume that `RABTAP_APIURI` environment variable points to the +broker to be used, e.g. `http://guest:guest@localhost:15672/api`). + +* `rabtap info --filter "exchange.Name == 'amq.direct'" --omit-empty`: Print + only queues bound to exchange `amq.direct` and skip all empty exchanges. +* `rabtap info --filter "queue.Name =~ '.*test.*'" --omit-empty`: Print all + queues with `test` in their name. +* `rabtap info --filter "queue.Name =~ '.*test.*' && exchange.Type == 'topic'" --omit-empty`: Like + before, but consider only exchanges of type `topic`. + +### Type reference + +The types reflect more or less the JSON API objects of the [REST API of +RabbitMQ](https://rawcdn.githack.com/rabbitmq/rabbitmq-management/v3.7.7/priv/www/api/index.html) +transformed to golang types. + +#### Exchange type + +``` +type Exchange struct { + Name string + Vhost string + Type string + Durable bool + AutoDelete bool + Internal bool + MessageStats struct { + PublishOut + PublishOutDetails struct { + Rate float64 + } + PublishIn int + PublishInDetails struct { + Rate float64 + } + } +} +``` + +#### Queue type + +``` +type Queue struct { + MessagesDetails struct { + Rate float64 + } + Messages + MessagesUnacknowledgedDetails struct { + Rate float64 + } + MessagesUnacknowledged int + MessagesReadyDetails struct { + Rate float64 + } + MessagesReady int + ReductionsDetails struct { + Rate float64 + } + Reductions int + Node string + Exclusive bool + AutoDelete bool + Durable bool + Vhost string + Name string + MessageBytesPagedOut int + MessagesPagedOut int + BackingQueueStatus struct { + Mode string + Q1 int + Q2 int + Q3 int + Q4 int + Len int + NextSeqID int + AvgIngressRate float64 + AvgEgressRate float64 + AvgAckIngressRate float64 + AvgAckEgressRate float64 + } + MessageBytesPersistent int + MessageBytesRAM int + MessageBytesUnacknowledged int + MessageBytesReady int + MessageBytes int + MessagesPersistent int + MessagesUnacknowledgedRAM int + MessagesReadyRAM int + MessagesRAM int + GarbageCollection struct { + MinorGcs int + FullsweepAfter int + MinHeapSize int + MinBinVheapSize int + MaxHeapSize int + } + State string + Consumers int + IdleSince string + Memory int +} + +``` + +#### Binding type + +``` +type Binding struct { + Source string + Vhost string + Destination string + DestinationType string + RoutingKey string + PropertiesKey string +} +``` + ## Build from source To build rabtap from source, you need [go](https://golang.org/) and the diff --git a/cmd/main/broker_info_printer.go b/cmd/main/broker_info_printer.go index b3f4ebf..a9d1f61 100644 --- a/cmd/main/broker_info_printer.go +++ b/cmd/main/broker_info_printer.go @@ -31,8 +31,8 @@ const ( {{- ""}} host='{{ .Connection.Host }}:{{ .Connection.Port }}', {{- ""}} peer='{{ .Connection.PeerHost }}:{{ .Connection.PeerPort }}')` tplExchange = ` - {{- $printname := .Exchange.Name }}{{ if eq $printname "" }}{{ $printname := "(default)" }}{{end}} - {{- ExchangeColor $printname }} (exchange, type '{{ .Exchange.Type }}' + {{- if eq .Exchange.Name "" }}{{ ExchangeColor "(default)" }}{{ else }}{{ ExchangeColor .Exchange.Name }}{{ end }} + {{- "" }} (exchange, type '{{ .Exchange.Type }}' {{- if and .Config.ShowStats .Exchange.MessageStats }}, in=( {{- .Exchange.MessageStats.PublishIn }}, {{printf "%.1f" .Exchange.MessageStats.PublishInDetails.Rate}}/s) msg, out=( {{- .Exchange.MessageStats.PublishOut }}, {{printf "%.1f" .Exchange.MessageStats.PublishOutDetails.Rate}}/s) msg @@ -56,7 +56,6 @@ type BrokerInfoPrinterConfig struct { ShowConsumers bool ShowStats bool QueueFilter Predicate - ExchangeFilter Predicate OmitEmptyExchanges bool NoColor bool } @@ -269,10 +268,11 @@ func (s BrokerInfoPrinter) createConnectionNodes( func (s BrokerInfoPrinter) shouldDisplayQueue( queue *rabtap.RabbitQueue, + exchange *rabtap.RabbitExchange, binding *rabtap.RabbitBinding) bool { - // apply queue filter - params := map[string]interface{}{"queue": queue, "binding": binding} + // apply filter + params := map[string]interface{}{"queue": queue, "binding": binding, "exchange": exchange} if res, err := s.config.QueueFilter.Eval(params); err != nil || !res { if err != nil { log.Warnf("error evaluating queue filter: %s", err) @@ -285,6 +285,7 @@ func (s BrokerInfoPrinter) shouldDisplayQueue( func (s BrokerInfoPrinter) createQueueNodeFromBinding( binding *rabtap.RabbitBinding, + exchange *rabtap.RabbitExchange, brokerInfo *rabtap.BrokerInfo) []*TreeNode { // standard binding of queue to exchange @@ -297,7 +298,7 @@ func (s BrokerInfoPrinter) createQueueNodeFromBinding( queue = &rabtap.RabbitQueue{Name: binding.Destination} } - if !s.shouldDisplayQueue(queue, binding) { + if !s.shouldDisplayQueue(queue, exchange, binding) { return []*TreeNode{} } @@ -333,7 +334,7 @@ func (s BrokerInfoPrinter) createExchangeNode( brokerInfo)) } else { // queue to exchange binding - exchangeNode.AddList(s.createQueueNodeFromBinding(&binding, brokerInfo)) + exchangeNode.AddList(s.createQueueNodeFromBinding(&binding, exchange, brokerInfo)) } } return exchangeNode @@ -349,31 +350,9 @@ func (s BrokerInfoPrinter) shouldDisplayExchange( return false } - // apply exchange filter - params := map[string]interface{}{"exchange": exchange} - if res, err := s.config.ExchangeFilter.Eval(params); err != nil || !res { - if err != nil { - log.Warnf("error evaluating exchange filter: %s", err) - } else { - return false - } - } return true } -func (s BrokerInfoPrinter) getExchangesToDisplay( - exchanges []rabtap.RabbitExchange, - vhost string) []rabtap.RabbitExchange { - - var result []rabtap.RabbitExchange - for _, exchange := range exchanges { - if s.shouldDisplayExchange(&exchange, vhost) { - result = append(result, exchange) - } - } - return result -} - // buildTree renders given brokerInfo into a tree: // RabbitMQ-Host // +--VHost @@ -396,7 +375,10 @@ func (s BrokerInfoPrinter) buildTree(brokerInfo *rabtap.BrokerInfo, for vhost := range uniqueVhosts(brokerInfo.Exchanges) { vhNode := NewTreeNode(s.renderVhostAsString(vhost)) root.Add(vhNode) - for _, exchange := range s.getExchangesToDisplay(brokerInfo.Exchanges, vhost) { + for _, exchange := range brokerInfo.Exchanges { + if !s.shouldDisplayExchange(&exchange, vhost) { + continue + } exNode := s.createExchangeNode(&exchange, brokerInfo) if s.config.OmitEmptyExchanges && !exNode.HasChildren() { continue diff --git a/cmd/main/broker_info_printer_test.go b/cmd/main/broker_info_printer_test.go index 4198500..062e55c 100644 --- a/cmd/main/broker_info_printer_test.go +++ b/cmd/main/broker_info_printer_test.go @@ -125,7 +125,6 @@ func ExampleBrokerInfoPrinter_Print() { ShowConsumers: true, ShowDefaultExchange: false, QueueFilter: TruePredicate, - ExchangeFilter: TruePredicate, OmitEmptyExchanges: false, NoColor: true}, ) @@ -183,7 +182,6 @@ func ExampleBrokerInfoPrinter_printWithQueueFilter() { ShowConsumers: true, ShowDefaultExchange: false, QueueFilter: queueFilter, - ExchangeFilter: TruePredicate, OmitEmptyExchanges: true, NoColor: true}, ) diff --git a/cmd/main/command_line.go b/cmd/main/command_line.go index c26b500..de28986 100644 --- a/cmd/main/command_line.go +++ b/cmd/main/command_line.go @@ -16,14 +16,14 @@ var RabtapAppVersion = "(version not specified)" const ( // note: usage is interpreted by docopt - this is code. - usage = `rabtap - RabbitMQ message tap. + usage = `rabtap - RabbitMQ wire tap. Usage: rabtap -h|--help rabtap tap EXCHANGES [--uri URI] [--saveto=DIR] [-jknv] rabtap (tap --uri URI EXCHANGES)... [--saveto=DIR] [-jknv] rabtap info [--api APIURI] [--consumers] [--stats] - [--qfilter FILTER] [--efilter FILTER] + [--filter EXPR] [--omit-empty] [--show-default] [-knv] rabtap pub [--uri URI] EXCHANGE [FILE] [--routingkey=KEY] [-jkv] rabtap sub QUEUE [--uri URI] [--saveto=DIR] [-jkvn] @@ -50,7 +50,7 @@ Examples: # use RABTAP_APIURI environment variable to specify mgmt api uri instead of --api export RABTAP_APIURI=http://guest:guest@localhost:15672/api rabtap info - rabtap info --qfilter "binding.exchange == 'amq.topic'" + rabtap info --filter "binding.Exchange == 'amq.topic'" rabtap conn close "172.17.0.1:40874 -> 172.17.0.2:5672" Options: @@ -67,7 +67,7 @@ Options: -b, --bindingkey KEY binding key to use in bind queue command. --consumers include consumers and connections in output of info command. -d, --durable create durable exchange/queue. - --efilter FILTER Filter for exchanges of info command (see README.md) + --filter EXPR Filter for info command to filter queues (see README.md) -h, --help print this help. -j, --json print/save/publish message metadata and body to a single JSON file. JSON body is base64 encoded. Otherwise @@ -75,8 +75,6 @@ Options: -k, --insecure allow insecure TLS connections (no certificate check). -n, --no-color don't colorize output (also environment variable NO_COLOR) -o, --omit-empty don't show echanges without bindings in info command. - --qfilter FILTER Filter for queues of info command (see README.md). Implies - --omit-empty. --reason=REASON reason why the connection was closed [default: closed by rabtap]. -r, --routingkey KEY routing key to use in publish mode. @@ -144,7 +142,6 @@ type CommandLineArgs struct { ShowConsumers bool // info mode: also show consumer ShowStats bool // info mode: also show statistics QueueFilter *string // info mode: optional filter for queues - ExchangeFilter *string // info mode: optional filter for exchanges OmitEmptyExchanges bool // info mode: do not show exchanges wo/ bindings Durable bool // queue create, exchange create Autodelete bool // queue create, exchange create @@ -208,12 +205,8 @@ func parseInfoCmdArgs(args map[string]interface{}) (CommandLineArgs, error) { ShowStats: args["--stats"].(bool), ShowDefaultExchange: args["--show-default"].(bool)} - if args["--efilter"] != nil { - filter := args["--efilter"].(string) - result.ExchangeFilter = &filter - } - if args["--qfilter"] != nil { - filter := args["--qfilter"].(string) + if args["--filter"] != nil { + filter := args["--filter"].(string) result.QueueFilter = &filter } var err error diff --git a/cmd/main/command_line_test.go b/cmd/main/command_line_test.go index f3c812d..47904f3 100644 --- a/cmd/main/command_line_test.go +++ b/cmd/main/command_line_test.go @@ -151,7 +151,6 @@ func TestCliInfoCmd(t *testing.T) { assert.False(t, args.InsecureTLS) assert.False(t, args.NoColor) assert.Nil(t, args.QueueFilter) - assert.Nil(t, args.ExchangeFilter) assert.False(t, args.OmitEmptyExchanges) } @@ -184,7 +183,7 @@ func TestCliInfoCmdApiFromEnv(t *testing.T) { func TestCliInfoCmdAllOptionsAreSet(t *testing.T) { args, err := ParseCommandLineArgs( []string{"info", "--api=APIURI", "--stats", "--consumers", - "--efilte=EFILTER", "--qfilter=QFILTER", "--omit-empty", + "--filter=EXPR", "--omit-empty", "--no-color", "-k", "--show-default"}) assert.Nil(t, err) @@ -193,8 +192,7 @@ func TestCliInfoCmdAllOptionsAreSet(t *testing.T) { assert.Equal(t, "APIURI", args.APIURI) assert.Nil(t, args.SaveDir) assert.False(t, args.Verbose) - assert.Equal(t, "QFILTER", *args.QueueFilter) - assert.Equal(t, "EFILTER", *args.ExchangeFilter) + assert.Equal(t, "EXPR", *args.QueueFilter) assert.True(t, args.ShowStats) assert.True(t, args.ShowConsumers) assert.True(t, args.ShowDefaultExchange) diff --git a/cmd/main/main.go b/cmd/main/main.go index 6352890..415dadf 100644 --- a/cmd/main/main.go +++ b/cmd/main/main.go @@ -49,8 +49,6 @@ func createFilterPredicate(expr *string) (Predicate, error) { func startCmdInfo(args CommandLineArgs, title string) { queueFilter, err := createFilterPredicate(args.QueueFilter) failOnError(err, "invalid queue filter predicate", os.Exit) - exchangeFilter, err := createFilterPredicate(args.ExchangeFilter) - failOnError(err, "invalid exchange filter predicate", os.Exit) cmdInfo(CmdInfoArg{ rootNode: title, client: rabtap.NewRabbitHTTPClient(args.APIURI, getTLSConfig(args.InsecureTLS)), @@ -59,8 +57,7 @@ func startCmdInfo(args CommandLineArgs, title string) { ShowConsumers: args.ShowConsumers, ShowDefaultExchange: args.ShowDefaultExchange, QueueFilter: queueFilter, - ExchangeFilter: exchangeFilter, - OmitEmptyExchanges: args.QueueFilter != nil, + OmitEmptyExchanges: args.OmitEmptyExchanges, NoColor: args.NoColor}, out: NewColorableWriter(os.Stdout)}) } diff --git a/doc/filter.md b/doc/filter.md deleted file mode 100644 index f2c6ec8..0000000 --- a/doc/filter.md +++ /dev/null @@ -1,163 +0,0 @@ -# rabtap filtering options - - - -* [Filtering expressions](#filtering-expressions) - * [Filtering exchanges](#filtering-exchanges) - * [Examples](#examples) - * [Filtering queues](#filtering-queues) - * [Examples](#examples-1) -* [Type reference](#type-reference) - * [Exchange type](#exchange-type) - * [Queue type](#queue-type) - * [Bindig type](#bindig-type) - - - -When your brokers topology contains many exchanges and queues, the output of -the `info` command can become very bloated. The `--efilter` and `--qfilter` for -flexible filtering of exchanges and queues to be displayed in the `info` -command to narrow output to the desired information. - -## Filtering expressions - -A filtering expression is a function that evaluates to true or false (i.e. -a predicate). The currently evaluated expression or queue will only be printed -if the predicate evaluates to true. - -Rabtap uses the [govalute](https://github.com/Knetic/govaluate) to evaluate the -predicate. During evaluation the context (i.e. the current exchange or queue) -is available in the expression context as a variable. - -See [official govaluate -documentation](https://github.com/Knetic/govaluate/blob/master/MANUAL.md) for -further information. - -### Filtering exchanges - -The `--efilter EXPR` option is used to filter exchanges. The current exchange -is bound to the variable `exchange` during evaluation. The variable is of -type [Exchange](#exchange-type). - -#### Examples - -Only show exchange `amq.direct`: -`$ ./rabtap-linux-amd64 info --efilter "exchange.Name == 'amq.direct'"` - -### Filtering queues - -The `--qfilter EXPR` option is used to filter queues. The current queue is -bound to the variable `queue` during evaluation. The `queue` variable is of -type [Queue](#queue-type). The associated binding is bound to the variable -`binding`. The `binding` variable is of type [Binding](#binding-type). - -### Examples -TODO - -## Type reference - -Documentation of the types of `exchange`, `queue` and `binding` variables used -in the filtering expressions. The types reflect more or less the JSON api -objects of the [REST API of -RabbitMQ](https://rawcdn.githack.com/rabbitmq/rabbitmq-management/v3.7.7/priv/www/api/index.html). - -### Exchange type - -``` -type Exchange struct { - Name string - Vhost string - Type string - Durable bool - AutoDelete bool - Internal bool - MessageStats struct { - PublishOut - PublishOutDetails struct { - Rate float64 - } - PublishIn int - PublishInDetails struct { - Rate float64 - } - } -} -``` - -### Queue type - -``` -type Queue struct { - MessagesDetails struct { - Rate float64 - } - Messages - MessagesUnacknowledgedDetails struct { - Rate float64 - } - MessagesUnacknowledged int - MessagesReadyDetails struct { - Rate float64 - } - MessagesReady int - ReductionsDetails struct { - Rate float64 - } - Reductions int - Node string - Exclusive bool - AutoDelete bool - Durable bool - Vhost string - Name string - MessageBytesPagedOut int - MessagesPagedOut int - BackingQueueStatus struct { - Mode string - Q1 int - Q2 int - Q3 int - Q4 int - Len int - NextSeqID int - AvgIngressRate float64 - AvgEgressRate float64 - AvgAckIngressRate float64 - AvgAckEgressRate float64 - } - MessageBytesPersistent int - MessageBytesRAM int - MessageBytesUnacknowledged int - MessageBytesReady int - MessageBytes int - MessagesPersistent int - MessagesUnacknowledgedRAM int - MessagesReadyRAM int - MessagesRAM int - GarbageCollection struct { - MinorGcs int - FullsweepAfter int - MinHeapSize int - MinBinVheapSize int - MaxHeapSize int - } - State string - Consumers int - IdleSince string - Memory int -} - -``` - -### Bindig type - -``` -type Binding struct { - Source string - Vhost string - Destination string - DestinationType string - RoutingKey string - PropertiesKey string -} -```