2.6

It should work now

3.0.0-rc4

Release Notes

• Fixed a bug with the KCL Reader where the reader was nil
• Stripped out Pipe of Death from both the Producer and Consumer

3.0.0-rc3

• Removed unnecessary CloseWithContext function in producer

Kinetic v3.0.0-rc2

RELEASE NOTES

• Refactored Consumer to remove rate limiting logic from the consume function
• Moved rate limiters (count and size) to KinesisReader
• Removed throttle function because it was redundant
• Added thorough documentation of how the kinetic library works in doc.go

Kinetic Refactor

RELEASE NOTES

• Producer
  • refactored to abstract out the StreamWriter interface to support writing to Kinesis and Firehose
  • refactored to remove the retry queue entirely
    • we determined that the original implementation could lead to an overflow of goroutines (and ultimately unbounded memory issues)
    • new implementation utilizes a dispatcher / worker model throttled by a rate limiter
  • implemented shard monitoring to periodically check the number of active shards and adjust the producer's rate limiter automatically
• Consumer (previously Listener)
  • refactored to abstract out the StreamReader interface to support reading from Kinesis using the aws-sdk-go library and the Kinesis Client Library (KCL) using the Multilang daemon protocol
  • Added rate limiting on the GetRecords calls
  • Added the KclReader which implements the StreamReader interface and uses KCL to retrieve messages from Kinesis
    • Added the ability to checkpoint periodically (on demand and automatically, through configurable parameters)
• Updated stats collection interfaces and added a default stats collector which utilizes rcrowley metrics
• Major refactor to consolidate multiple packages down to a single (main) package
• API clean up by utilizing function option methods pattern to adjust configurable parameters
• Added a testing harness (testexec) which builds an executable binary with a CLI that can be used to test the Producer / Consumer in different modes of operation (read, write, readwrite)

2.5.0

Added

Dynamic request limit for GetRecords with backoff

2.4.0

Added

• less greedy Send()s with runtime.Gosched()
• synchronous CloseSync() on Listener and Producer

2.3.0

• Add TryToSend that drops when a channel is to prevent infinitely growing memory utilization.

2.2.3

Fixed

• Auth with metadata on refresh

2.2.2

Added

• Rate limiting for shard iterator requests

Fixed

• Race conditions in tests

Changes

• Utilize atomic package for counters