piper contains three commands for piping json to Azure EventGrid, AWS EventBridge and Splunk
Piper, Piper, the red, red worm
woke last night to the sound of the storm
her words were words I sailed upon
-- Phish (from the song Piper)
go get github.com/willabides/piper/cmd/eventbridge-pipe
Add a bindown dependency:
$ bindown template-source add piper https://github.com/WillAbides/piper/releases/latest/download/bindown-templates.yml
$ bindown dependency add eventbridge-pipe piper#eventbridge-pipe
Usage: eventbridge-pipe --type=STRING --source=STRING
eventbridge-pipe posts events to AWS EventBridge.
example:
$ AWS_ACCESS_KEY='AKIA****************'
$ AWS_SECRET_KEY='shhh_this_is_a_secret'
$ data="$(cat <<"EOF"
{"action": "obj.add", "@timestamp": 1604953432032, "el_name": "foo", "doc_id": "asdf"}
{"action": "obj.rem", "@timestamp": 1604953732032, "el_name": "bar", "doc_id": "fdsa"}
EOF
)"
$ echo "$data" | \
eventbridge-pipe -s 'test-source' -t 'jp:action' -b 'my-bus' -T 'jp:"@timestamp"' \
-r 'jp:"el_name"'
Learn about JMESPath syntax at https://jmespath.org
Flags:
-h, --help Show context-sensitive help.
--region="us-east-1" The aws region to publish events to.
-t, --type=STRING Value for the DetailType field. JMESPath
expressions allowed with "jp:" prefix.
-b, --event-bus=STRING Value for the "EventBusName" field.
-r, --resource=RESOURCE,... An element for the list in the "Resources"
array. JMESPath expressions allowed with "jp:"
prefix.
-s, --source=STRING Value for the "Source" field. JMESPath
expressions allowed with "jp:" prefix.
-T, --timestamp=STRING Value for the "Time" field converted from epoch
milliseconds. JMESPath expressions allowed with
"jp:" prefix.
--batch-size=10 Number of events to send in a batch.
--flush-interval=2000 Time in milliseconds to wait before sending a
partial batch. Set to 0 to never send a partial
batch.
go get github.com/willabides/piper/cmd/eventgrid-pipe
Add a bindown dependency:
$ bindown template-source add piper https://github.com/WillAbides/piper/releases/latest/download/bindown-templates.yml
$ bindown dependency add eventgrid-pipe piper#eventgrid-pipe
Usage: eventgrid-pipe --subject=STRING --type=STRING <topic-endpoint>
eventgrid-pipe posts events to Azure Event Grid.
example:
$ topic_endpoint='https://mytopicendpoint.westus2-1.eventgrid.azure.net'
$ topic_key='shhh_secret_topic_key'
$ data="$(cat <<"EOF"
{"action": "obj.add", "@timestamp": 1604953432032, "el_name": "foo", "doc_id": "asdf"}
{"action": "obj.rem", "@timestamp": 1604953732032, "el_name": "bar", "doc_id": "fdsa"}
EOF
)"
$ echo "$data" | \
eventgrid-pipe "$topic_endpoint" \
-H "aeg-sas-key: $topic_key" \
-T 'jp:"@timestamp"' \
-t 'audit-log' \
-s 'jp:action' \
-i 'jp:doc_id'
Learn about JMESPath syntax at https://jmespath.org
Arguments:
<topic-endpoint> Endpoint for posting events
Flags:
-h, --help Show context-sensitive help.
-i, --id=STRING Value for the "id" field. If unset, a uuid will
be generated for each event. JMESPath expressions
allowed with "jp:" prefix.
-s, --subject=STRING Value for the "subject" field. JMESPath
expressions allowed with "jp:" prefix.
-t, --type=STRING Value for the "eventType" field. JMESPath
expressions allowed with "jp:" prefix.
-T, --timestamp="now" Value for the "eventTime" field converted from
epoch milliseconds. If unset, the current system
time will be used.JMESPath expressions allowed
with "jp:" prefix.
-H, --header=HEADER,... Header to sent with the request in the same
format as curl. e.g. '-H "aeg-sas-key: $EVKEY"'
--data-version="1.0" Value for the "dataVersion" field. JMESPath
expressions allowed with "jp:" prefix.
--batch-size=10 Number of events to send in a batch.
--flush-interval=2000 Time in milliseconds to wait before sending a
partial batch. Set to 0 to never send a partial
batch.
go get github.com/willabides/piper/cmd/splunk-pipe
Add a bindown dependency:
$ bindown template-source add piper https://github.com/WillAbides/piper/releases/latest/download/bindown-templates.yml
$ bindown dependency add splunk-pipe piper#splunk-pipe
Usage: splunk-pipe <endpoint>
splunk-pipe posts events to splunk.
example:
$ splunk_endpoint="http://localhost:8080"
$ splunk_hec_token="shhh_secret_token"
$ data="$(cat <<"EOF"
{"action": "obj.add", "@timestamp": 1604953432032, "el_name": "foo", "doc_id": "asdf"}
{"action": "obj.rem", "@timestamp": 1604953732032, "el_name": "bar", "doc_id": "fdsa"}
EOF
)"
$ echo "$data" | \
splunk-pipe "$splunk_endpoint" \
-H "Authorization: Splunk $splunk_hec_token" \
-T 'jp:"@timestamp"'
Learn about JMESPath syntax at https://jmespath.org
Arguments:
<endpoint> Endpoint for posting events
Flags:
-h, --help Show context-sensitive help.
-t, --sourcetype=STRING Value for the "sourcetype" field. JMESPath
expressions allowed with "jp:" prefix.
-s, --source=STRING Value for the "source" field. JMESPath
expressions allowed with "jp:" prefix.
-T, --timestamp=STRING Value for the "eventTime" field converted from
epoch milliseconds. JMESPath expressions allowed
with "jp:" prefix.
-H, --header=HEADER,... Header to sent with the request in the same
format as curl. e.g. '-H "Authorization: Splunk
$HEC_KEY"'
-h, --host=STRING Value for the "host" field. JMESPath expressions
allowed with "jp:" prefix.
--index=STRING Value for the "index" field. JMESPath expressions
allowed with "jp:" prefix.
--batch-size=10 Number of events to send in a batch.
--flush-interval=2000 Time in milliseconds to wait before sending a
partial batch. Set to 0 to never send a partial
batch.