Skip to content

AWS for Fluent Bit 2.31.4
Compare
Choose a tag to compare
@matthewfala matthewfala released this 01 Mar 19:38
v2.31.4
9bb03be

2.31.4

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.1
  • Amazon Kinesis Streams for Fluent Bit 1.10.1
  • Amazon Kinesis Firehose for Fluent Bit 1.7.1

Compared to 2.31.3, this release removes the following fixes that may introduce instabilities to the S3 plugin:

  • Bug - Support Retry_Limit option in S3 plugin to set retries fluent-bit:6475
  • Bug - Format S3 filename with timestamp from the first log in uploaded file, rather than the time the first log was buffered by the s3 output aws-for-fluent-bit:459

Same as 2.31.3, this release includes the following fixes and features that we are working on getting accepted upstream:

  • Bug - Resolve cloudwatch_logs duplicate tag match SIGSEGV issue introduced in 2.29.0 aws-for-fluent-bit:542
  • Feature - Add kinesis_firehose and kinesis_streams support for time_key_format milliseconds with %3N option, and nanoseconds 9N and %L options fluent-bit:2831
  • Feature - Support OpenSearch Serverless data ingestion via OpenSearch plugin fluent-bit:6448
  • Enhancement - Transition S3 to fully synchronous file uploads to improve plugin stability fluent-bit:6573
  • Bug - Mitigate Datadog output plugin issue by reverting recent PR aws-for-fluent-bit:491
  • Bug - Resolve S3 logic to display log_key missing warning message if the configured log_key field is missing from log payload fluent-bit:6557
  • Bug - ECS Metadata filter gracefuly handle task metadata query errors and cache metadata processing state to improve performance aws-for-fluent-bit:505
  • Bug - Resolve a rare Datadog segfault that occurs when remapping tags aws-for-fluent-bit:491
  • Bug - Resolve net.keepalive SIGSEGV issue and priority event loop corruption bug aws-for-fluent-bit:542

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
kinesis_firehose tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
kinesis_streams stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(18360) 0%(32342)
kinesis_streams tcp Log Loss ✅ ✅ 0%(24)
Log Duplication 0%(500) ✅ 0%(26068)
s3 stdstream Log Loss ✅ 7%(1085784) 27%(4931846)
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ 14%(2636934)
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
Assets 2