Like netcat
, but for Amazon Kinesis.
$>./c2k --help
Usage of ./c2k:
-d string
Delimiter to split on (defaults to newline) (short) (default "\n")
-delimiter string
Delimiter to split on (defaults to newline) (default "\n")
-f Firehose mode
-i string
Type of Shard Iterator to use. Valid choices: AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, TRIM_HORIZON (short) (default "TRIM_HORIZON")
-iter string
Type of Shard Iterator to use. Valid choices: AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER, TRIM_HORIZON (default "TRIM_HORIZON")
-l Listen to stream instead of sending data (short)
-listen
Listen to stream instead of sending data
-p string
AWS Profile name to use for authentication (short) (default "default")
-partitionKey string
Partition key (default "1")
-pk string
Partition key (short) (default "1")
-profile string
AWS Profile name to use for authentication (default "default")
-r string
AWS region, defaults to us-east-1 (short) (default "us-east-1")
-region string
AWS region, defaults to us-east-1 (default "us-east-1")
-s string
Stream name to put data (short)
-sId string
Shard ID for listen purposes (short) (default "ALL")
-shardId string
Shard ID for listen purposes (default "ALL")
-sn string
Sequence number to use for iterators that use a sequence number (short)
-startingSeqNum string
Sequence number to use for iterators that use a sequence number
-streamName string
Stream name to put data
c2k operates in two modes. The first mode sends data to Kinesis. The second mode listens or reads data from a Kinesis stream.
c2k also supports kinesis firehose. You use the -f
flag to activate firehose mode.
c2k -s stream-name -f access.log
A common use case for c2k is sending each line of a file into Kinesis. Here is how you can do this with c2k:
c2k -s your-stream access.log error.log
This will put each line of access.log
and error.log
as a record into the Kinesis stream named your-stream
. By default c2k will use the default crendentials in ~/.aws/credentials
, but you can choose the profile with the -p
option.
You can also listen for data in a Kinesis stream. By default c2k will listen to all shards in the stream, but you can specify a single shard id as well.
c2k -l -s your-stream -shardId 1
This will stream from shard id 1 of the stream named your-stream
. c2k will write the data from the stream to standard out. By default, c2k will used the TRIM_HORIZON
type of shard iterator.