Skip to content

@github-actions github-actions released this Sep 8, 2021

For installation instructions check out the getting started guide.

Added

  • New experimental gcp_bigquery output.
  • Go API: It's now possible to parse a config spec directly with ParseYAML.
  • Bloblang methods and functions now support named parameters.
  • Field args_mapping added to the cassandra output.
  • For NATS, NATS Streaming and Jetstream components the config now supports specifying either nkey_file or user_credentials_file to configure authentication.

The full change log can be found here.

Assets 17

@github-actions github-actions released this Sep 3, 2021

For installation instructions check out the getting started guide.

Fixed

  • Upgraded dependencies, preventing a crash on start up caused by MacOS builds made with Go 1.17.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Sep 1, 2021

For installation instructions check out the getting started guide.

Added

  • The mqtt input and output now support sending a last will, configuring a keep alive timeout, and setting retained out output messages.
  • Go API: New stream builder AddBatchProducerFunc and AddBatchConsumerFunc methods.
  • Field gzip_compression added to the elasticsearch output.
  • The redis_streams input now supports creating the stream with the MKSTREAM command (enabled by default).
  • The kafka output now supports manual partition allocation using interpolation functions in the field partition.

Fixed

  • The bloblang method contains now correctly compares numerical values in arrays and objects.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Aug 19, 2021

For installation instructions check out the getting started guide.

Added

  • Go API: Added ability to create and register BatchBuffer plugins.
  • New system_window buffer for processing message windows (sliding or tumbling) following the system clock.
  • Field root_cas added to all TLS configuration blocks.
  • The sftp input and output now support key based authentication.
  • New Bloblang function nanoid.
  • The gcp_cloud_storage output now supports custom collision behaviour with the field collision_mode.
  • Field priority added to the amqp_0_9 output.
  • Operator keys added to the redis processor.
  • The http_client input when configured in stream mode now allows message body interpolation functions within the URL and header parameters.

Fixed

  • Fixed a panic that would occur when executing a pipeline where processor or input resources reference rate limits.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Aug 2, 2021

For installation instructions check out the getting started guide.

Added

  • The elasticsearch output now supports delete, update and index operations.
  • Go API: Added ability to create and register BatchInput plugins.

Fixed

  • Prevented the http_server input from blocking graceful pipeline termination indefinitely.
  • Removed annoying nil error log from HTTP client components when parsing responses.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Jul 26, 2021

For installation instructions check out the getting started guide.

Added

  • The redis_streams, redis_pubsub and redis_list outputs now all support batching for higher throughput.
  • The amqp_1 input and output now support passing and receiving metadata as annotations.
  • Config unit test definitions can now use files for both the input and expected output.
  • Field track_properties added to the azure_queue_storage input for enriching messages with properties such as the message backlog.
  • Go API: The new plugin APIs, available at ./public/service, are considered stable.
  • The streams mode API now uses the setting http.read_timeout for timing out stream CRUD endpoints.

Fixed

  • The Bloblang function random_int now only resolves dynamic arguments once during the lifetime of the mapping. Documentation has been updated in order to clarify the behaviour with dynamic arguments.
  • Fixed an issue where plugins registered would return failed to obtain docs for X type Y linting errors.
  • HTTP client components are now more permissive regarding invalid Content-Type headers.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Jul 19, 2021

For installation instructions check out the getting started guide.

Added

  • New CLI flag --set (-s) for overriding arbitrary fields in a config. E.g. -s input.type=http_server would override the config setting the input type to http_server.
  • Unit test definitions now support mocking components.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Jul 12, 2021

For installation instructions check out the getting started guide.

Added

  • The nats input now supports acks.
  • The memory and file cache types now expose metrics akin to other caches.

Fixed

  • The switch output when retry_until_success is set to false will now provide granular nacks to pre-batched messages.
  • The URL printed in error messages when HTTP client components fail should now show interpolated values as they were interpreted.
  • Go Plugins API V2: Batched processors should now show in tracing, and no longer complain about spans being closed more than once.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Jun 25, 2021

For installation instructions check out the getting started guide.

Added

  • Algorithm lz4 added to the compress and decompress processors.
  • New experimental aws_dynamodb_partiql processor.
  • Go Plugins API: new run opt OptUseContext for an extra shutdown mechanism.

Fixed

  • Fixed an issue here the http_client would prematurely drop connections when configured with stream.enabled set to true.
  • Prevented closed output brokers from leaving child outputs running when they've failed to establish a connection.
  • Fixed metrics prefixes in streams mode for nested components.

The full change log can be found here.

Assets 16

@github-actions github-actions released this Jun 16, 2021

For installation instructions check out the getting started guide.

Added

  • CLI flag max-token-length added to the blobl subcommand.
  • Go Plugins API: Plugin components can now be configured seamlessly like native components, meaning the namespace plugin is no longer required and configuration fields can be placed within the namespace of the plugin itself. Note that the old style (within plugin) is still supported.
  • The http_client input fields url and headers now support interpolation functions that access metadata and contents of the last received message.
  • Rate limit resources now emit checked, limited and error metrics.
  • A new experimental plugins API is available for early adopters, and can be found at ./public/x/service.
  • A new experimental template system is available for early adopters, examples can be found in ./template.
  • New beta Bloblang method bloblang for executing dynamic mappings.
  • All http components now support a beta jwt authentication mechanism.
  • New experimental schema_registry_decode processor.
  • New Bloblang method parse_duration for parsing duration strings into an integer.
  • New experimental twitter_search input.
  • New field args_mapping added to the sql processor and output for mapping explicitly typed arguments.
  • Added format csv to the unarchive processor.
  • The redis processor now supports incrby operations.
  • New experimental discord input and output.
  • The http_server input now adds a metadata field http_server_verb.
  • New Bloblang methods parse_yaml and format_yaml.
  • CLI flag env-file added to Benthos for parsing dotenv files.
  • New mssql SQL driver for the sql processor and output.
  • New POST endpoint /resources/{type}/{id} added to Benthos streams mode for dynamically mutating resource configs.

Changed

  • Go Plugins API: The Bloblang ArgSpec now returns a public error type ArgError.
  • Components that support glob paths (file, csv, etc) now also support super globs (double asterisk).
  • The aws_kinesis input is now stable.
  • The gcp_cloud_storage input and output are now beta.
  • The kinesis input is now deprecated.
  • Go Plugins API: the minimum version of Go required is now 1.16.

Fixed

  • Fixed a rare panic caused when executing a workflow resource processor that references branch resources across parallel threads.
  • The mqtt input with multiple topics now works with brokers that would previously error on multiple subscriptions.
  • Fixed initialisation of components configured as resources that reference other resources, where under certain circumstances the components would fail to obtain a true reference to the target resource. This fix makes it so that resources are accessed only when used, which will also make it possible to introduce dynamic resources in future.
  • The streams mode endpoint /streams/{id}/stats should now work again provided the default manager is used.

The full change log can be found here.

Assets 16