v0.3.2 - Adaptive Batching

Performance and Reliability Improvements

Adaptive Batching Strategy

Replaced single large fetch with intelligent batching approach:

• Batch size: 5,000 messages per batch
• Adaptive timeout: Scales with batch size (batch_size/20 ms, minimum 100ms)
• Smart termination: Stops when server returns ErrNoMessages

Performance Results

Small streams (15k messages):

• v0.3.0/v0.3.1: 1-2s
• v0.3.2: ~750ms → 2-3x faster ⚡

Medium streams (65k messages):

• v0.3.0/v0.3.1: 1-2s (inconsistent)
• v0.3.2: ~3.3s → Consistent and reliable ✅

Large streams (200k messages):

• v0.3.0/v0.3.1: Timeouts or incomplete results
• v0.3.2: ~10s → Handles correctly ✅

Fixes

• ❌ v0.3.1 bug fixed: invalid argument: batch size must be at least 1 error when NumPending was 0
• ✅ Race condition eliminated: No longer uses NumPending approach that caused inconsistent results
• ✅ Consistency guaranteed: Timeout per batch ensures server completes gathering
• ✅ Scales correctly: Time proportional to actual data size

Dependencies

Updated all dependencies to latest versions:

• AWS SDK v2: v1.32.5 → v1.40.1
• imroc/req: v3.48.0 → v3.56.0
• Ginkgo: v2.20.2 → v2.27.2
• And more...

Migration from v0.3.1

If you're on v0.3.1: Upgrade immediately - it has a critical bug causing batch size must be at least 1 errors

Recommended version: v0.3.2 (this release) for both performance and reliability