mill - a simple CLI tool for Watermill
mill is a CLI tool for the Watermill library.
It has two basic functionalities, namely producing and consuming messages on the following Pub/Subs:
- Google Cloud Pub/Sub
See Watermill's Pub/Sub implementations for more details on how this works.
To install this tool, just execute:
go get -u github.com/ThreeDotsLabs/watermill/tools/mill
This will install a
mill binary in your system.
In consume mode, the tool subscribes to a topic/queue/subscription (nomenclature depending on the particular Pub/Sub provider) and prints the messages' payload to the standard output.
Other outputs, for example ones that add a timestamp or preserve UUIDs or metadata, are easily attainable by modification
of the marshaling function of the
io.Publisher of the
In produce mode, subsequent lines of data from the standard input are transformed into messages outgoing to the requested provider's topic/exchange.
The message's payload is set to the line from stdin, the UUID is auto-generated and the metadata is empty.
Similarly, the contents of the message could be parsed differently from stdin, by modification
of the unmarshaling function of the
io.Subscriber of the
The basic syntax of the tool is:
mill <provider> <command>
with the appropriate flags regulating the specific behaviour of each command.
command is usually one of
consume, but some providers may handle additional commands
that are specific for them.
The flags are context-specific, so the best way to find out about them is to use the
-h flag and study
which flags are available/required for the specific context and act accordingly.
A neat feature of producers and consumers is that you can use the power of stdin/stdout piping for stuff like:
myservice | tee myservice.log | mill kafka produce -t myservice-logs --brokers kafka-host:8082
And on another host:
mill kafka consume -t myservice-logs --brokers kafka-host:8082 >> myservice.log
In the above example, the host on which
myservice runs has its own copy of
myservice.log and any host that consumes
from the kafka topic will replicate the log entries in their local copy.
Google Cloud Pub/Sub
You can use
mill to create/remove subscriptions for Google Cloud Pub/Sub:
mill googlecloud subscription add -t <topic> <subscription_id> mill googlecloud subscription rm <subscription_id>
Additional flags are available for
subscription add to regulate the newly created subscription's settings.