v4.39.0

For installation instructions check out the getting started guide.

Added

• New timeplus input. (@ye11ow)
• New snowflake_streaming output. (@rockwotj)
• Redpanda Connect will now use an optional /etc/redpanda/connector_list.yaml config to determine which connectors are available to run. (@Jeffail)
• (Benthos) Field follow_redirects added to the http processor. (@ooesili)
• New CLI flag --secrets added. (@Jeffail)
• New CLI flag --disable-telemetry added. (@Jeffail)
• New experimental spicedb watch input. (@simon0191)
• New redpanda_common input and output. (@Jeffail)
• New redpanda input and output. (@Jeffail)
• New snowflake_streaming output. (@rockwotj)

Fixed

• The kafka, kafka_franz and redpanda_migrator outputs no longer waste CPU for large batches. (@rockwotj)

Changed

• The aws_sqs output field url now supports interpolation functions. (@rockwotj)
• (Benthos) CLI --set flags can now mutate array values indexed from the end via negative integers. E.g. --set 'foo.-1=meow' would set the last index of the array foo to the value of meow. (@Jeffail)

The full change log can be found here.

v4.39.0-rc1

For installation instructions check out the getting started guide.

NOTE: This is a release candidate, you can download a binary from this page.

Added

• New timeplus input. (@ye11ow)
• New snowflake_streaming output. (@rockwotj)
• Redpanda Connect will now use an optional /etc/redpanda/connector_list.yaml config to determine which connectors are available to run. (@Jeffail)
• (Benthos) Field follow_redirects added to the http processor. (@ooesili)
• New CLI flag --secrets added. (@Jeffail)
• New CLI flag --disable-telemetry added. (@Jeffail)

Changed

• The aws_sqs output field url now supports interpolation functions. (@rockwotj)
• (Benthos) CLI --set flags can now mutate array values indexed from the end via negative integers. E.g. --set 'foo.-1=meow' would set the last index of the array foo to the value of meow. (@Jeffail)

The full change log can be found here.

v4.38.0

For installation instructions check out the getting started guide.

Added

• Anonymous telemetry data is now sent by Connect instances after running for >5 mins. Details about which data is sent, when it is sent, and how to disable it can be found in the telemetry README. (@Jeffail)
• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)
• New azure_data_lake_gen2 output. (@ooesili)
• New timeplus output. (@ye11ow)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

Changed

• All components with a default path field value (such as the aws_s3 output) containing the deprecated function count have now been changed to use the new function counter. This could potentially change behaviour in cases where multiple components are executing a mapping with a count function sharing the same of the old default count, and these counters need to cascade. This is an extremely unlikely scenario, but for all users of these components it is recommended that your path is defined explicitly, and in a future major version we will be removing the defaults.

The full change log can be found here.

v4.38.0-rc5

For installation instructions check out the getting started guide.

Added

• Anonymous telemetry data is now sent by Connect instances after running for >5 mins. Details about which data is sent, when it is sent, and how to disable it can be found in the telemetry README. (@Jeffail)
• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)
• New azure_data_lake_gen2 output. (@ooesili)
• New timeplus output. (@ye11ow)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

Changed

• All components with a default path field value (such as the aws_s3 output) containing the deprecated function count have now been changed to use the new function counter. This could potentially change behaviour in cases where multiple components are executing a mapping with a count function sharing the same of the old default count, and these counters need to cascade. This is an extremely unlikely scenario, but for all users of these components it is recommended that your path is defined explicitly, and in a future major version we will be removing the defaults.

The full change log can be found here.

v4.38.0-rc4

For installation instructions check out the getting started guide.

Added

• Anonymous telemetry data is now sent by Connect instances after running for >5 mins. Details about which data is sent, when it is sent, and how to disable it can be found in the telemetry README. (@Jeffail)
• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)
• New azure_data_lake_gen2 output. (@ooesili)
• New timeplus output. (@ye11ow)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

Changed

• All components with a default path field value (such as the aws_s3 output) containing the deprecated function count have now been changed to use the new function counter. This could potentially change behaviour in cases where multiple components are executing a mapping with a count function sharing the same of the old default count, and these counters need to cascade. This is an extremely unlikely scenario, but for all users of these components it is recommended that your path is defined explicitly, and in a future major version we will be removing the defaults.

The full change log can be found here.

v4.38.0-rc3

For installation instructions check out the getting started guide.

NOTE: This is a release candidate, you can download a binary from this page.

Added

• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

The full change log can be found here.

v4.37.0

For installation instructions check out the getting started guide.

Added

• New experimental gcp_vertex_ai_embeddings processor. (@rockwotj)
• New experimental aws_bedrock_embeddings processor. (@rockwotj)
• New experimental cohere_chat and cohere_embeddings processors. (@rockwotj)
• New experimental questdb output. (@sklarsa)
• Field metadata_max_age added to the kafka_franz input. (@Scarjit)
• Field metadata_max_age added to the kafka_migrator input. (@mihaitodor)
• New experimental cypher output. (@rockwotj)
• New experimental couchbase output. (@rockwotj)
• Field fetch_in_order added to the schema_registry input. (@mihaitodor)

Fixed

• Fixed a bug with the input_resource field for the kafka_migrator output where new topics weren't created as expected. (@mihaitodor)
• Fixed a bug in the kafka_migrator input which could lead to extra duplicate messages during a consumer group rebalance. (@mihaitodor)
• kafka_migrator, kafka_migrator_offsets and kafka_migrator_bundle components renamed to redpanda_migrator, redpanda_migrator_offsets and redpanda_migrator_bundle (@mihaitodor)

Fixed

• Fixes a panic in the parquet_encode processor (@mihaitodor)

The full change log can be found here.

v4.37.0-rc2

For installation instructions check out the getting started guide.

Added

• New experimental gcp_vertex_ai_embeddings processor. (@rockwotj)
• New experimental aws_bedrock_embeddings processor. (@rockwotj)
• New experimental cohere_chat and cohere_embeddings processors. (@rockwotj)
• New experimental questdb output. (@sklarsa)
• Field metadata_max_age added to the kafka_franz input. (@Scarjit)
• Field metadata_max_age added to the kafka_migrator input. (@mihaitodor)
• New experimental cypher output. (@rockwotj)
• New experimental couchbase output. (@rockwotj)
• Field fetch_in_order added to the schema_registry input. (@mihaitodor)

Fixed

• Fixed a bug with the input_resource field for the kafka_migrator output where new topics weren't created as expected. (@mihaitodor)
• Fixed a bug in the kafka_migrator input which could lead to extra duplicate messages during a consumer group rebalance. (@mihaitodor)
• kafka_migrator, kafka_migrator_offsets and kafka_migrator_bundle components renamed to redpanda_migrator, redpanda_migrator_offsets and redpanda_migrator_bundle (@mihaitodor)

Fixed

• Fixes a panic in the parquet_encode processor (@mihaitodor)

The full change log can be found here.

v4.36.1-rc1

For installation instructions check out the getting started guide.

NOTE: This is a release candidate, you can download a binary from this page.

Added

• New experimental gcp_vertex_ai_embeddings processor. (@rockwotj)
• New experimental aws_bedrock_embeddings processor. (@rockwotj)
• New experimental cohere_chat and cohere_embeddings processors. (@rockwotj)
• New experimental questdb output. (@sklarsa)

The full change log can be found here.

v4.36.0

For installation instructions check out the getting started guide.

Added

• Fields replication_factor and replication_factor_override added to the kafka_migrator input and output. (@mihaitodor)

Fixed

• The schema_registry_encode and schema_registry_decode processors no longer unescape path separators in the schema name. (@Mizaro)
• (Benthos) The switch output metrics now emit the case id as part of their labels. This is a regression introduced in v4.25.0. (@mihaitodor)
• (Benthos) Fixed a bug where certain logs used the %w verb to print errors resulting in incorrect output. (@mihaitodor)
• (Benthos) The logger no longer tries to replace Go fmt verbs in log messages. (@mihaitodor)

The full change log can be found here.