Skip to content

v5.9.0

Latest

Choose a tag to compare

@github-actions github-actions released this 15 Nov 23:30
· 56 commits to main since this release
13f7226

What's New

  • Add UseJsonSerializerContext to the JSON serializer and deserializer builders to use a System.Text.Json source-generated context
  • Configure in the System.Text.Json and Newtonsoft.Json serializer and deserializer builders now applies to the previously configured options instead of replacing them
  • Filter the produced messages with a custom, asynchronous IOutboundMessageFilter, optionally resolved via DI
  • Filter the consumed messages per endpoint with a predicate or an IInboundMessageFilter, optionally resolved via DI: the filtered messages are skipped and committed
  • Support batch processing with MQTT 5
  • Support chunking with MQTT 5
  • Add ITestingHelper.Outbox to inspect, clear, wait for and process the outbox in the tests
  • Store a batch of messages published to the outbox without blocking a thread
  • Annotate IInboundEnvelope<TMessage>.IsTombstone for the nullable reference types analysis: the Message is known to be not null after checking that the envelope isn't a tombstone

Fixes

  • Take the serializer options into account when comparing JsonMessageSerializer and JsonMessageDeserializer instances
  • Take the serializer settings into account when comparing NewtonsoftJsonMessageSerializer and NewtonsoftJsonMessageDeserializer instances
  • Stabilize the consumption with AllowStreaming, especially the error handling and the shutdown
  • Prevent the consumer from stalling when one of several subscribers stops enumerating a stream
  • Prevent the consumer from stalling when one of several batch or stream subscribers is canceled
  • Stabilize the batch processing, which could lose messages, process them twice or commit them unprocessed when a batch failed or was retried, or the consumer stopped
  • Prevent the consumer from stalling when a batch subscriber stops enumerating while the batch is completed by its max wait time
  • Prevent a batch completed by its max wait time from committing a message that no subscriber received (e.g. because they canceled the enumeration)
  • Prevent a message from being processed twice when a batch with multiple subscribers is completed by its max wait time
  • Prevent a batch or stream subscriber from receiving a null message, or from seeing a normal end, when the consumer stops or the batch or stream fails
  • Prevent the thread pool starvation, which could stall the consumer (e.g. exceeding the Kafka max.poll.interval.ms), when many batches or streams are consumed at once by subscribers enumerating them synchronously (IEnumerable<T>, IMessageStreamEnumerable<T>) or subscribing to an IObservable<T>
  • Complete the batches when their max wait time elapses and discard the incomplete chunked messages when their timeout elapses, instead of up to a second later
  • Allow multiple endpoints in a Kafka consumer processing all partitions together
  • Fix the Kafka consumer not leaving the group when disconnecting with the cooperative sticky partition assignment strategy and auto-commit, which delayed the reassignment of its partitions until the session timeout
  • Prevent a static Kafka consumer (group.instance.id) from leaving the group, and triggering a rebalance, when disconnecting
  • Redeliver the unacknowledged QoS 1 and 2 messages when a mocked MQTT persistent session is resumed
  • Keep the MQTT consumer reconnecting when a QoS 2 acknowledgment fails because of a lost connection
  • Prevent the broker from dropping a message of a shared subscription routed to an MQTT client while it disconnects (observed with EMQX)
  • Fix publishing a collection of messages hanging when the producer endpoint's filter skips one of them
  • Fix publishing a collection of chunked messages, which could complete before all chunks were produced, silently skip some of the messages or miss some errors
  • Invoke the callbacks passed to IProducer.Produce once per message, also when the message is filtered out or split into chunks
  • Prevent the code following the publishing of a collection of messages from running on the broker client's thread, which could stall or deadlock the producer
  • Throw instead of hanging when publishing a batch of messages to the outbox fails or is canceled
  • Prevent the outbox worker from producing messages twice when a stored message targets an unknown endpoint
  • Apply the outbox worker retry delay to every failure to produce the stored messages, instead of retrying some of them in a tight loop
  • Fix the outbox worker stopping for good after many consecutive failures, when an exponential retry delay with a maximum delay is configured
  • Delay the first retry of the retry error policy and of the outbox worker by the initial delay, as documented

Breaking Changes

  • The batch max wait time is measured from the first message of the batch, as documented, instead of restarting with each message: this is a fix, but review the max wait time if you tuned it around the previous behavior
  • IInboundEnvelope redeclares IsTombstone for the nullable analysis: the actual envelopes are not affected, but the custom implementations, usually test doubles, must implement it and the substitutes (e.g. NSubstitute or Moq) must configure the property through IInboundEnvelope<TMessage>, since configuring it through IInboundEnvelope or IBrokerEnvelope doesn't apply to it
  • IOutboundMessageFilter is now asynchronous (ShouldProduceAsync replaces ShouldProduce): this doesn't affect the producer endpoint Filter predicates, since the endpoint builder didn't accept a custom implementation until now
  • HandleMessageAsync of Consumer returns a ValueTask<ConsumerPipelineContext> instead of a ValueTask (relevant only for custom broker integrations)

See docs for details.