Skip to content

A Node.js readable stream for Kinesis, with automatic shard-cycling abilities

Notifications You must be signed in to change notification settings

tcql/kinesis-cycling-readable

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

kinesis-cycling-readable

Usage

var kinesisRead = require('kinesis-cycling-readable');

var stream = kinesisRead(
  new AWS.Kinesis({region: 'us-east-1'}),
  'stream-name'
});

stream.on('data', function (records) {
});

Upon encountering an ProvisionedThroughputExceededException, the stream will automatically cycle to the next shard in the stream and keep reading.

Options

{
  readpause: 1000, // Milliseconds to wait between `getRecords` calls
  cyclepause: 1000, // Milliseconds to wait after cycling shards
  allowLooping: false, // If true, when the last shard is cycled, start back at the first shard
                       // If false, the stream emits an error when the last shard is exhausted
  iteratorType: 'LATEST', // Type of ShardIterator to request. Must be one of 
                          // TRIM_HORIZON | LATEST | AT_TIMESTAMP
  iteratorTimestamp: null // Timestamp to use for `AT_TIMESTAMP` ShardIterators
}

About

A Node.js readable stream for Kinesis, with automatic shard-cycling abilities

Resources

Stars

Watchers

Forks

Packages

No packages published