v4.0.3

2025-02-27

Known issues

There's a potential regression where consuming messages could fail due to #23931 changes. This is being addressed in #24055. No workarounds currently exist.

Library updates

• [feat][misc] upgrade oxia java client to 0.5.0 (#23871)
• [fix] Bump org.apache.solr:solr-core from 8.11.3 to 9.8.0 in /pulsar-io/solr (#23899)
• [fix] Use Alpine 3.21 in base image (#23964)
• [fix][build] Upgrade json-smart to 2.5.2 (#23966)
• [improve] Upgrade to Netty 4.1.117.Final (#23863)
• [fix][sec] Upgrade to Netty 4.1.118 to address CVE-2025-24970 (#23965)

Broker

• [fix][broker] Fix bug in RangeCache where different instance of the key wouldn't ever match (#23903)
• [fix][broker] Apply dispatcherMaxReadSizeBytes also for replay reads for Shared and Key_Shared subscriptions (#23894)
• [fix][broker] Closed topics won't be removed from the cache (#23884)
• [fix][broker] Fix BucketDelayedDeliveryTracker thread safety (#24014)
• [fix][broker] Fix NPE during active consumer is disconnected (#24008)
• [fix][broker] Fix incorrect blockedConsumerOnUnackedMsgs value when maxUnackedMessagesPerConsumer is 1 (#23796)
• [fix][broker] Fix rate limiter token bucket and clock consistency issues causing excessive throttling and connection timeouts (#23930)
• [fix][broker] Fix repeatedly acquired pending reads quota (#23869)
• [fix][broker] Fix seeking by timestamp can be reset the cursor position to earliest (#23919)
• [fix][broker] Geo Replication lost messages or frequently fails due to Deduplication is not appropriate for Geo-Replication (#23697)
• [fix][broker] Make InflightReadsLimiter asynchronous and apply it for replay queue reads (#23901)
• [fix][broker] PIP-322 Fix issue with rate limiters where rates can exceed limits initially and consumption pauses until token balance is positive (#24012)
• [fix][broker] PIP-379 Key_Shared implementation race condition causing out-of-order message delivery (#23874)
• [fix][broker] Skip to persist cursor info if it failed by cursor closed (#23615)
• [fix][broker] fix broker identifying incorrect stuck topic (#24006)
• [fix][broker] fix broker may lost rack information (#23331)
• [fix][meta] Fix ephemeral Zookeeper put which creates a persistent znode (#23984)
• [fix][meta] Fix ephemeral handling of ZK nodes and fix MockZooKeeper ephemeral and ZK stat handling (#23988)
• [fix][ml] Fix cursor metadata compatability issue when switching the config unackedRangesOpenCacheSetEnabled (#23759)
• [fix][ml] Fix deadlock in PendingReadsManager (#23958)
• [fix][ml] Fix memory leak due to duplicated RangeCache value retain operations (#23955)
• [fix][ml] Fix memory leaks in ManagedCursorInfo and ManagedLedgerInfo decompression and compression (#23960)
• [fix][ml] incorrect non-durable cursor's backlog due to concurrently trimming ledger and non-durable cursor creation (#23951)
• [improve][broker] Avoid PersistentReplicator.expireMessages logic compute backlog twice (#23957)
• [improve][broker] Avoid logging errors when there is a connection issue during subscription. (#23939)
• [improve][broker] Avoid printing log for IncompatibleSchemaException in ServerCnx (#23938)
• [improve][broker] Do not print error logs for NotFound or Conflict errors when using the Admin API (#23928)
• [improve][broker] Don't print error logs for ProducerBusyException (#23929)
• [improve][broker] Fix non-persistent system topic schema compatibility (#23286)
• [improve][broker] Improve Consumer.equals performance (#23864)
• [improve][broker] Make the estimated entry size more accurate (#23931)
• [improve][broker] Refactor a private method to eliminate an unnecessary parameter (#23915)
• [improve][broker] Remove spamming logs for customized managed ledger (#23862)
• [improve][broker] Support values up to 2^32 in ConcurrentBitmapSortedLongPairSet (#23878)
• [improve][meta] Simplify getting parent path in ZKMetadataStore without using java.io.File (#23996)
• [improve][ml] Use lock-free queue in InflightReadsLimiter since there's no concurrent access (#23962)

Client

• [feat][client] Support forward proxy for the ZTS server in pulsar-client-auth-athenz (#23947)
• [fix] Avoid NPE when closing an uninitialized SameAuthParamsLookupAutoClusterFailover (#23911)
• [fix] Initialize UrlServiceProvider before trying to use transaction coordinator (#23914)
• [fix][client] Fix LoadManagerReport not found (#23886)
• [fix][client] Fix memory leak in ClientCnx.newLookup when there's TooManyRequestsException (#23971)
• [fix][client] Fix memory leak when message size exceeds max message size and batching is enabled (#23967)
• [fix][client] call redeliver 1 msg but did 2 msgs (#23943)
• [fix][client] fix retry topic with exclusive mode. (#23859)
• [improve][client] Avoid logging errors for retriable errors when creating producer (#23935)
• [improve][client] Update TypedMessageBuilder deliverAfter and deliverAt api comment (#23969)
• [improve][cli] Support additional msg metadata for V1 topic on peek message cmd (#23978)

Pulsar IO and Pulsar Functions

• [fix][io] Fix pulsar-io:pom not found (#23979)
• [improve][fn] Set default tenant and namespace for ListFunctions cmd (#23881)
• [improve][io] Allow skipping connector deployment (#23932)

Others

• [fix] fix for code scanning alert no. 48: Uncontrolled data used in path expression (#23985)
• [fix][build] Use amazoncorretto:21-alpine3.20 JDK build for Alpine 3.20 (#23898)
• [fix][build] Add develops for buildtools (#23992)
• [improve][proxy] Make keep-alive interval configurable in Pulsar Proxy (#23981)

Tests & CI

• [fix][test] Fix flaky DelayedDeliveryTest.testEnableTopicDelayedDelivery (#23893)
• [fix][test] Fix flaky test MetadataStoreTest.emptyStoreTest (#23998)
• [fix][test] Fix flaky test OneWayReplicatorUsingGlobalZKTest.testConfigReplicationStartAt (#24011)
• [fix][test] Fix quiet time implementation in BrokerTestUtil.receiveMessages (#23876)
• [fix][test] fix flaky testNegativeAcksWithBackoff when batch enabled. (#23986)
• [fix][test]Fix flaky test V1_ProducerConsumerTest.testConcurrentConsumerReceiveWhileReconnect (#24019)
• [improve][test] Add solution to PulsarMockBookKeeper for intercepting reads (#23875)
• [improve][test] Support decorating topic, subscription, dispatcher, ManagedLedger and ManagedCursors instances in tests (#23892)
• [improve][test] Upgrade Testcontainers to 1.20.4 and docker-java to 3.4.0 (#24003)
• [fix][ci] Configure Docker data-root to /mnt/docker to avoid running out of disk space ([#23909]...

v3.3.5

2025-02-27

Known issues

There's a potential regression where consuming messages could fail due to #23931 changes. This is being addressed in #24055. No workarounds currently exist.

Known issues

There's a potential regression where consuming messages could fail. This is being addressed in #24055. No workarounds currently exist.

Library updates

• [feat][misc] upgrade oxia java client to 0.5.0 (#23871)
• [fix] Use Alpine 3.21 in base image (#23964)
• [fix][build] Upgrade json-smart to 2.5.2 (#23966)
• [improve] Upgrade to Netty 4.1.117.Final (#23863)
• [fix][sec] Upgrade to Netty 4.1.118 to address CVE-2025-24970 (#23965)

Broker

• [fix][broker] Fix bug in RangeCache where different instance of the key wouldn't ever match (#23903)
• [fix][broker] Closed topics won't be removed from the cache (#23884)
• [fix][broker] Continue using the next provider for http authentication if one fails (#23842)
• [fix][broker] Fix BucketDelayedDeliveryTracker thread safety (#24014)
• [fix][broker] Fix NPE during active consumer is disconnected (#24008)
• [fix][broker] Fix acknowledgeCumulativeAsync block when ackReceipt is enabled (#23841)
• [fix][broker] Fix incorrect blockedConsumerOnUnackedMsgs value when maxUnackedMessagesPerConsumer is 1 (#23796)
• [fix][broker] Fix possible mark delete NPE when batch index ack is enabled (#23833)
• [fix][broker] Fix rate limiter token bucket and clock consistency issues causing excessive throttling and connection timeouts (#23930)
• [fix][broker] Fix repeatedly acquired pending reads quota (#23869)
• [fix][broker] Fix the retry mechanism in MetadataCache#readModifyUpdateOrCreate (#23686)
• [fix][broker] Geo Replication lost messages or frequently fails due to Deduplication is not appropriate for Geo-Replication (#23697)
• [fix][broker] Make InflightReadsLimiter asynchronous and apply it for replay queue reads (#23901)
• [fix][broker] PIP-322 Fix issue with rate limiters where rates can exceed limits initially and consumption pauses until token balance is positive (#24012)
• [fix][broker] PIP-399: Fix Metric Name for Delayed Queue (#23712)
• [fix][broker] Remove blocking calls from internalGetPartitionedStats (#23832)
• [fix][broker] Revert "[fix][broker] Cancel possible pending replay read in cancelPendingRead (#23384)" (#23855)
• [fix][broker] Support large number of unack message store for cursor recovery (#9292)
• [fix][broker] fix broker identifying incorrect stuck topic (#24006)
• [fix][broker] fix broker may lost rack information (#23331)
• [fix][meta] Fix ephemeral Zookeeper put which creates a persistent znode (#23984)
• [fix][meta] Fix ephemeral handling of ZK nodes and fix MockZooKeeper ephemeral and ZK stat handling (#23988)
• [fix][ml] Fix cursor metadata compatability issue when switching the config unackedRangesOpenCacheSetEnabled (#23759)
• [fix][ml] Fix deadlock in PendingReadsManager (#23958)
• [fix][ml] Fix memory leak due to duplicated RangeCache value retain operations (#23955)
• [fix][ml] Fix memory leaks in ManagedCursorInfo and ManagedLedgerInfo decompression and compression (#23960)
• [fix][ml] incorrect non-durable cursor's backlog due to concurrently trimming ledger and non-durable cursor creation (#23951)
• [improve][broker] Avoid PersistentReplicator.expireMessages logic compute backlog twice (#23957)
• [improve][broker] Avoid logging errors when there is a connection issue during subscription. (#23939)
• [improve][broker] Avoid printing log for IncompatibleSchemaException in ServerCnx (#23938)
• [improve][broker] Do not print error logs for NotFound or Conflict errors when using the Admin API (#23928)
• [improve][broker] Don't print error logs for ProducerBusyException (#23929)
• [improve][broker] Fix non-persistent system topic schema compatibility (#23286)
• [improve][broker] Improve Consumer.equals performance (#23864)
• [improve][broker] Make the estimated entry size more accurate (#23931)
• [improve][broker] Reduce unnecessary REPLICATED_SUBSCRIPTION_SNAPSHOT_REQUEST (#23839)
• [improve][broker] Remove spamming logs for customized managed ledger (#23862)
• [improve][broker] Support values up to 2^32 in ConcurrentBitmapSortedLongPairSet (#23878)
• [improve][meta] Simplify getting parent path in ZKMetadataStore without using java.io.File (#23996)
• [improve][ml] Use lock-free queue in InflightReadsLimiter since there's no concurrent access (#23962)

Client

• [fix] Avoid NPE when closing an uninitialized SameAuthParamsLookupAutoClusterFailover (#23911)
• [fix] Initialize UrlServiceProvider before trying to use transaction coordinator (#23914)
• [fix][client] Fix LoadManagerReport not found (#23886)
• [fix][client] Fix memory leak in ClientCnx.newLookup when there's TooManyRequestsException (#23971)
• [fix][client] Fix memory leak when message size exceeds max message size and batching is enabled (#23967)
• [fix][client] Orphan producer when concurrently calling producer closing and reconnection (#23853)
• [fix][client] call redeliver 1 msg but did 2 msgs (#23943)
• [fix][client] fix retry topic with exclusive mode. (#23859)
• [improve][client] Avoid logging errors for retriable errors when creating producer (#23935)
• [improve][cli] Support additional msg metadata for V1 topic on peek message cmd (#23978)

Pulsar IO and Pulsar Functions

• [fix][io] Fix pulsar-io:pom not found (#23979)
• [improve] Validate user paths in Functions utils (#22833)
• [improve][fn] Set default tenant and namespace for ListFunctions cmd (#23881)
• [improve][io] Allow skipping connector deployment (#23932)

Others

• [fix] fix for code scanning alert no. 48: Uncontrolled data used in path expression (#23985)
• [fix][build] Use amazoncorretto:21-alpine3.20 JDK build for Alpine 3.20 (#23898)
• [fix][build] Add develops for buildtools (#23992)
• [fix][misc] Honor dynamic log levels in log4j2.yaml (#23847)
• [improve][proxy] Make keep-alive interval configurable in Pulsar Proxy (#23981)

Tests & CI

• [improve][test] Add solution to PulsarMockBookKeeper for intercepting reads (#23875)
• [improve][test] Upgrade Testcontainers to 1.20.4 and docker-java to 3.4.0 (#24003)
• [fix][test] Add reconsumeLater call in RetryTopicTest#testRetryTopicWithMultiTopic. (#23857)
• [fix][test] Fix flaky DelayedDeliveryTest.testEnableTopicDelayedDelivery (#23893)
• [fix][test] Fix flaky test MetadataStoreTest.emptyStoreTest (#23998)
• [fix][test] Fix flaky...

v3.0.10

2025-02-27

Library updates

• [improve] Upgrade to Netty 4.1.117.Final (#23863)
• [fix][sec] Upgrade to Netty 4.1.118 to address CVE-2025-24970 (#23965)
• [fix] Bump org.apache.solr:solr-core from 8.11.3 to 9.8.0 in /pulsar-io/solr (#23899)
• [fix][build] Upgrade json-smart to 2.5.2 (#23966)

Broker

• [fix][broker] Fix bug in RangeCache where different instance of the key wouldn't ever match (#23903)
• [fix][broker] Closed topics won't be removed from the cache (#23884)
• [fix][broker] Continue using the next provider for http authentication if one fails (#23842)
• [fix][broker] Fix acknowledgeCumulativeAsync block when ackReceipt is enabled (#23841)
• [fix][broker] Fix incorrect blockedConsumerOnUnackedMsgs value when maxUnackedMessagesPerConsumer is 1 (#23796)
• [fix][broker] Fix possible mark delete NPE when batch index ack is enabled (#23833)
• [fix][broker] Fix repeatedly acquired pending reads quota (#23869)
• [fix][broker] Fix the retry mechanism in MetadataCache#readModifyUpdateOrCreate (#23686)
• [fix][broker] Make InflightReadsLimiter asynchronous and apply it for replay queue reads (#23901)
• [fix][broker] PIP-399: Fix Metric Name for Delayed Queue (#23712)
• [fix][broker] Remove blocking calls from internalGetPartitionedStats (#23832)
• [fix][broker] Revert "[fix][broker] Cancel possible pending replay read in cancelPendingRead (#23384)" (#23855)
• [fix][broker] Support large number of unack message store for cursor recovery (#9292)
• [fix][broker] fix broker may lost rack information (#23331)
• [cleanup][admin] Do not print full stacktrace when get partitioned metadata not found (#20979)
• [fix][meta] Fix ephemeral Zookeeper put which creates a persistent znode (#23984)
• [fix][meta] Fix ephemeral handling of ZK nodes and fix MockZooKeeper ephemeral and ZK stat handling (#23988)
• [fix][misc] Honor dynamic log levels in log4j2.yaml (#23847)
• [fix][ml] Fix cursor metadata compatability issue when switching the config unackedRangesOpenCacheSetEnabled (#23759)
• [fix][ml] Fix deadlock in PendingReadsManager (#23958)
• [fix][ml] Fix memory leak due to duplicated RangeCache value retain operations (#23955)
• [fix][ml] Fix memory leaks in ManagedCursorInfo and ManagedLedgerInfo decompression and compression (#23960)
• [fix][ml] incorrect non-durable cursor's backlog due to concurrently trimming ledger and non-durable cursor creation (#23951)
• [improve][broker] Avoid PersistentReplicator.expireMessages logic compute backlog twice (#23957)
• [improve][broker] Avoid logging errors when there is a connection issue during subscription. (#23939)
• [improve][broker] Avoid printing log for IncompatibleSchemaException in ServerCnx (#23938)
• [improve][broker] Do not print error logs for NotFound or Conflict errors when using the Admin API (#23928)
• [improve][broker] Don't print error logs for ProducerBusyException (#23929)
• [improve][broker] Improve Consumer.equals performance (#23864)
• [improve][broker] Reduce unnecessary REPLICATED_SUBSCRIPTION_SNAPSHOT_REQUEST (#23839)
• [improve][broker] Remove spamming logs for customized managed ledger (#23862)
• [improve][broker] Support values up to 2^32 in ConcurrentBitmapSortedLongPairSet (#23878)
• [improve][meta] Simplify getting parent path in ZKMetadataStore without using java.io.File (#23996)

Client

• [feat][client] Support forward proxy for the ZTS server in pulsar-client-auth-athenz (#23947)
• [fix] Avoid NPE when closing an uninitialized SameAuthParamsLookupAutoClusterFailover (#23911)
• [fix] Initialize UrlServiceProvider before trying to use transaction coordinator (#23914)
• [fix][client] Fix LoadManagerReport not found (#23886)
• [fix][client] Fix memory leak in ClientCnx.newLookup when there's TooManyRequestsException (#23971)
• [fix][client] Fix memory leak when message size exceeds max message size and batching is enabled (#23967)
• [fix][client] Orphan producer when concurrently calling producer closing and reconnection (#23853)
• [fix][client] call redeliver 1 msg but did 2 msgs (#23943)
• [fix][client] fix retry topic with exclusive mode. (#23859)
• [improve][client] Avoid logging errors for retriable errors when creating producer #23935

Pulsar IO and Pulsar Functions

• [fix][io] Fix pulsar-io:pom not found (#23979)
• [improve] Validate user paths in Functions utils (#22833)
• [improve][io] Allow skipping connector deployment (#23932)

Others

• [improve][proxy] Make keep-alive interval configurable in Pulsar Proxy (#23981)
• [fix] fix for code scanning alert no. 48: Uncontrolled data used in path expression (#23985)
• [fix][build] Add develops for buildtools (#23992)
• [improve] Support overriding java.net.preferIPv4Stack with OPTS (#23846)

Tests & CI

• [improve][test] Add solution to PulsarMockBookKeeper for intercepting reads (#23875)
• [fix][test] Add reconsumeLater call in RetryTopicTest#testRetryTopicWithMultiTopic. (#23857)
• [fix][test] Fix flaky DelayedDeliveryTest.testEnableTopicDelayedDelivery (#23893)
• [fix][test] Fix quiet time implementation in BrokerTestUtil.receiveMessages (#23876)
• [fix][test] fix flaky testNegativeAcksWithBackoff when batch enabled. (#23986)
• [fix][test]Fix flaky test testTopicUnloadAfterSessionRebuild (#23852)
• [fix][ci] Configure Docker data-root to /mnt/docker to avoid running out of disk space (#23909)
• [improve][ci] Increase Maven max heap size to 2048M and tune GCLockerRetryAllocationCount (#23883)
• [improve][ci] Publish build scans to develocity.apache.org (#23851)
• [improve][ci] Skip "OWASP dependency check" when data wasn't found in cache (#23970)
• [improve][ci] Update GitHub Workflows to Ubuntu 22.04 (#20729)

For the complete list, check the full changelog.

v4.0.2

2025-01-20

Known Major Issues in this Release

This release contains a known issue impacting the seek method by timestamp. This issue was reported after the release was published.

• [Bug] Seeking by timestamp causes the subscription to reset to the earliest position (#23910)

Workaround for the issue:

The issue is caused by changes introduced in #22792. To resolve this, disable the functionality by setting the following configuration:

managedLedgerCursorResetLedgerCloseTimestampMaxClockSkewMillis=-1

Workaround Configuration Notes:

• This configuration key needs to be manually added to your broker.conf file as it doesn't exist by default
• For Apache Pulsar Helm Chart Kubernetes deployments, add the following to your values.yaml file under the broker/configData section:

broker:
  configData:
    PULSAR_PREFIX_managedLedgerCursorResetLedgerCloseTimestampMaxClockSkewMillis: "-1"

Library updates

• [fix][sec] Bump commons-io version to 2.18.0 (#23684)
• [fix][sec] Mitigate CVE-2024-53990 by disabling AsyncHttpClient CookieStore (#23725)
• [fix][sec] Upgrade async-http-client to 2.12.4 to address CVE-2024-53990 (#23732)
• [fix][sec] Upgrade golang.org/x/crypto from 0.21.0 to 0.31.0 in pulsar-function-go (#23743)
• [improve] Upgrade lombok to 1.18.36 (#23752)
• [improve] Upgrade to Netty 4.1.116.Final and io_uring to 0.0.26.Final (#23813)
• [improve][monitor] Upgrade OTel to 1.45.0 (#23756)

Broker

• [fix][admin] Fix exception loss in getMessageId method (#23766)
• [fix][admin] Fix exception thrown in getMessageId method (#23784)
• [fix][admin] Listen partitioned topic creation event (#23680)
• [fix][admin] Verify is policies read only before revoke permissions on topic (#23730)
• [fix][broker] Add consumer name for subscription stats (#23671)
• [fix][broker] Avoid block markDeletePosition forward when skip lost entries (#21210)
• [fix][broker] Catch exception for entry payload interceptor processor (#23683)
• [fix][broker] Continue using the next provider for authentication if one fails (#23797)
• [fix][broker] Continue using the next provider for http authentication if one fails (#23842)
• [fix][broker] Fix acknowledgeCumulativeAsync block when ackReceipt is enabled (#23841)
• [fix][broker] Fix bug causing loss of migrated information when setting other localPolicies in namespace (#23764)
• [fix][broker] Fix config replicationStartAt does not work when set it to earliest (#23719)
• [fix][broker] Fix deadlock in Key_Shared PIP-379 implementation (#23854)
• [fix][broker] Fix enableReplicatedSubscriptions (#23781)
• [fix][broker] Fix items in dispatcher.recentlyJoinedConsumers are out-of-order, which may cause a delivery stuck (#23802)
• [fix][broker] Fix possible mark delete NPE when batch index ack is enabled (#23833)
• [fix][broker] Fix the retry mechanism in MetadataCache#readModifyUpdateOrCreate (#23686)
• [fix][broker] Invoke custom BrokerInterceptor's onFilter method if it's defined (#23676)
• [fix][broker] Msg delivery is stuck due to items in the collection recentlyJoinedConsumers are out-of-order (#23795)
• [fix][broker] PIP-399: Fix Metric Name for Delayed Queue (#23712)
• [fix][broker] Remove blocking calls from internalGetPartitionedStats (#23832)
• [fix][broker] Remove failed OpAddEntry from pendingAddEntries (#23817)
• [fix][broker] Revert "[fix][broker] Cancel possible pending replay read in cancelPendingRead (#23384)" (#23855)
• [fix][broker] System topic should not be migrated during blue-green cluster migration (#23767)
• [fix][broker] fix NPE when calculating a topic's backlogQuota (#23720)
• [fix][broker] support missing cluster level fine-granted permissions (#23675)
• [fix][broker] topic policy deadlock block metadata thread. (#23786)
• [fix][ml] Topic load timeout due to ml data ledger future never finishes (#23772)
• [improve][admin] Opt-out of topic-existence check (#23709)
• [improve][broker] Improve SystemTopicBasedTopicPoliciesService reader to reduce GC pressure (#23780)
• [improve][broker] Optimize subscription seek (cursor reset) by timestamp (#22792)
• [improve][broker] Reduce unnecessary REPLICATED_SUBSCRIPTION_SNAPSHOT_REQUEST (#23839)
• [improve][log] Print ZK path if write to ZK fails due to data being too large to persist (#23652)
• [improve][ml] Optimize BlobStoreManagedLedgerOffloader.getOffloadPolicies (#23776)
• [improve][txn] Improve Reader in TransactionBuffer to reduce GC pressure (#23779)

Client

• [Fix][Client] Fix pending message not complete when closeAsync (#23761)
• [fix][client] Cannot access message data inside ProducerInterceptor#onSendAcknowledgement (#23791)
• [fix][client] Fix compatibility between kerberos and tls (#23798)
• [fix][client] Fix enableRetry for consumers using legacy topic naming where cluster name is included (#23753)
• [fix][client] Fix memory leak when publishing encountered a corner case error (#23738)
• [fix][client] Fix reader message filtering issue during blue-green cluster switch (#23693)
• [fix][client] Fix wrong start message id when it's a chunked message id (#23713)
• [fix][client] Make DeadLetterPolicy & KeySharedPolicy serializable (#23718)
• [fix][client] Orphan producer when concurrently calling producer closing and reconnection (#23853)
• [fix][client] Prevent retry topic and dead letter topic producer leaks when sending of message fails (#23824)
• [fix][doc] Refine ClientBuilder#memoryLimit and ConsumerBuilder#autoScaledReceiverQueueSizeEnabled javadoc (#23687)
• [improve][client] Make replicateSubscriptionState nullable (#23757)
• [improve][client] PIP-393: Improve performance of Negative Acknowledgement (#23600)
• [improve][client] PIP-393: Support configuring NegativeAckPrecisionBitCnt while building consumer. (#23804)
• [improve][client] Print consumer stats log if prefetched messages are not zero (#23698)

Pulsar IO and Pulsar Functions

• [improve][fn] Improve closing of producers in Pulsar Functions ProducerCache invalidation (#23734)
• [improve][fn] Improve implementation for maxPendingAsyncRequests async concurrency limit when return type is CompletableFuture<Void> (#23708)
• [improve][io] Bump io.lettuce:lettuce-core from 5.0.2.RELEASE to 6.5.1.RELEASE in /pulsar-io/redis (#23685)

Others

• [fix][common] TopicName: Throw IllegalArgumentException if localName is whitespace only (#23691)
• [fix] Fix issues with Pulsar Alpine docker image stability: remove glibc-compat (#23762)
• [fix][cli] Fix set topic retention policy failed (#23688)
• [fix][cli] Fix set-retention with >2GB size value for topic policy (#23689)
• [fix][misc] Honor dynamic log levels in log4j2.yaml (#23847)
• [improve] Sup...

v3.3.4

2025-01-20

Library updates

• [improve] Upgrade lombok to 1.18.36 (#23752)
• [improve] Upgrade to Netty 4.1.116.Final and io_uring to 0.0.26.Final (#23813)
• [improve][monitor] Upgrade OTel to 1.45.0 (#23756)
• [fix][sec] Bump commons-io version to 2.18.0 (#23684)
• [fix][sec] Mitigate CVE-2024-53990 by disabling AsyncHttpClient CookieStore (#23725)
• [fix][sec] Upgrade async-http-client to 2.12.4 to address CVE-2024-53990 (#23732)
• [fix][sec] Upgrade golang.org/x/crypto from 0.21.0 to 0.31.0 in pulsar-function-go (#23743)

Broker

• [feat][broker] Implement allowBrokerOperationAsync in PulsarAuthorizationProvider to avoid exception thrown (#23663)
• [fix][admin] Fix exception loss in getMessageId method (#23766)
• [fix][admin] Fix exception thrown in getMessageId method (#23784)
• [fix][broker] Add consumer name for subscription stats (#23671)
• [fix][broker] Catch exception for entry payload interceptor processor (#23683)
• [fix][broker] Continue using the next provider for authentication if one fails (#23797)
• [fix][broker] Fix bug causing loss of migrated information when setting other localPolicies in namespace (#23764)
• [fix][broker] Fix config replicationStartAt does not work when set it to earliest (#23719)
• [fix][broker] Fix enableReplicatedSubscriptions (#23781)
• [fix][broker] Fix items in dispatcher.recentlyJoinedConsumers are out-of-order, which may cause a delivery stuck (#23802)
• [fix][broker] Invoke custom BrokerInterceptor's onFilter method if it's defined (#23676)
• [fix][broker] Remove failed OpAddEntry from pendingAddEntries (#23817)
• [fix][broker] Skip to persist cursor info if it failed by cursor closed (#23615)
• [fix][broker] System topic should not be migrated during blue-green cluster migration (#23767)
• [fix][broker] fix NPE when calculating a topic's backlogQuota (#23720)
• [fix][broker] support missing cluster level fine-granted permissions (#23675)
• [fix][broker] support missing tenant level fine-granted permissions (#23660)
• [fix][broker] support missing broker level fine-granted permissions (#23637)
• [fix][broker] Msg delivery is stuck due to items in the collection recentlyJoinedConsumers are out-of-order (#23795)
• [fix][common] TopicName: Throw IllegalArgumentException if localName is whitespace only (#23691)
• [fix][ml] Topic load timeout due to ml data ledger future never finishes (#23772)
• [improve][admin] Opt-out of topic-existence check (#23709)
• [improve][broker][branch-3.3] Optimize PersistentTopic.getLastDispatchablePosition (#22707) (#23826)

Client

• [Fix][Client] Fix pending message not complete when closeAsync (#23761)
• [fix][client] Cannot access message data inside ProducerInterceptor#onSendAcknowledgement (#23791)
• [fix][client] Fix enableRetry for consumers using legacy topic naming where cluster name is included (#23753)
• [fix][client] Fix memory leak when publishing encountered a corner case error (#23738)
• [fix][client] Fix reader message filtering issue during blue-green cluster switch (#23693)
• [fix][client] Make DeadLetterPolicy & KeySharedPolicy serializable (#23718)
• [fix][client] Prevent retry topic and dead letter topic producer leaks when sending of message fails (#23824)
• [improve][client] Make replicateSubscriptionState nullable (#23757)

Pulsar IO and Pulsar Functions

• [improve][fn] Improve closing of producers in Pulsar Functions ProducerCache invalidation (#23734)
• [improve][fn] Improve implementation for maxPendingAsyncRequests async concurrency limit when return type is CompletableFuture (#23708)

Others

• [fix][cli] Fix set topic retention policy failed (#23688)
• [fix][cli] Fix set-retention with >2GB size value for topic policy (#23689)
• [fix] Fix issues with Pulsar Alpine docker image stability: remove glibc-compat (#23762)
• [improve] Install coreutils in docker image to improve compatibility (#23667)

Tests & CI

• [fix][test] Remove useless test code (#23823)
• [fix][test]: Flaky-test: GetPartitionMetadataMultiBrokerTest.testCompatibilityDifferentBrokersForNonPersistentTopic (#23666)

For the complete list, check the full changelog.

v3.0.9

2025-01-20

Library updates

• [fix][sec] Bump commons-io version to 2.18.0 (#23684)
• [fix][sec] Mitigate CVE-2024-53990 by disabling AsyncHttpClient CookieStore (#23725)
• [fix][sec] Upgrade async-http-client to 2.12.4 to address CVE-2024-53990 (#23732)
• [fix][sec] Upgrade golang.org/x/crypto from 0.21.0 to 0.31.0 in pulsar-function-go (#23743)
• [improve] Upgrade lombok to 1.18.36 (#23752)
• [improve] Upgrade to Netty 4.1.116.Final and io_uring to 0.0.26.Final (#23813)

Broker

• [fix][broker] Msg delivery is stuck due to items in the collection recentlyJoinedConsumers are out-of-order (#23795)
• [fix][broker] Add consumer name for subscription stats (#23671)
• [fix][broker] Catch exception for entry payload interceptor processor (#23683)
• [fix][broker] Continue using the next provider for authentication if one fails (#23797)
• [fix][broker] Fix config replicationStartAt does not work when set it to earliest (#23719)
• [fix][broker] Fix enableReplicatedSubscriptions (#23781)
• [fix][broker] Fix items in dispatcher.recentlyJoinedConsumers are out-of-order, which may cause a delivery stuck (#23802)
• [fix][broker] Remove failed OpAddEntry from pendingAddEntries (#23817)
• [fix][broker] Skip to persist cursor info if it failed by cursor closed (#23615)
• [fix][broker] fix NPE when calculating a topic's backlogQuota (#23720)
• [fix][admin] Fix exception loss in getMessageId method (#23766)
• [fix][admin] Fix exception thrown in getMessageId method (#23784)
• [fix][admin] Listen partitioned topic creation event (#23680)
• [improve][admin] Opt-out of topic-existence check (#23709)
• [improve][log] Print ZK path if write to ZK fails due to data being too large to persist (#23652)
• [fix][ml] Topic load timeout due to ml data ledger future never finishes (#23772)

Client

• [Fix][Client] Fix pending message not complete when closeAsync (#23761)
• [fix][client] Cannot access message data inside ProducerInterceptor#onSendAcknowledgement (#23791)
• [fix][client] Fix enableRetry for consumers using legacy topic naming where cluster name is included (#23753)
• [fix][client] Fix memory leak when publishing encountered a corner case error (#23738)
• [fix][client] Fix wrong start message id when it's a chunked message id (#23713)
• [fix][client] Make DeadLetterPolicy & KeySharedPolicy serializable (#23718)
• [fix][client] Prevent retry topic and dead letter topic producer leaks when sending of message fails (#23824)
• [fix][client][branch-3.0] Fix compatibility between kerberos and tls (#23801)
• [improve][client] Make replicateSubscriptionState nullable (#23757)
• [fix][doc] Refine ClientBuilder#memoryLimit and ConsumerBuilder#autoScaledReceiverQueueSizeEnabled javadoc (#23687)

Pulsar IO and Pulsar Functions

• [improve][fn] Improve closing of producers in Pulsar Functions ProducerCache invalidation (#23734)
• [improve][fn] Improve implementation for maxPendingAsyncRequests async concurrency limit when return type is CompletableFuture (#23708)
• [improve][io] Bump io.lettuce:lettuce-core from 5.0.2.RELEASE to 6.5.1.RELEASE in /pulsar-io/redis (#23685)
• [fix][fn][branch-3.0] Fix pulsar-function-go compilation

Others

• [fix][common] TopicName: Throw IllegalArgumentException if localName is whitespace only (#23691)

Tests & CI

• [fix][test] Remove useless test code (#23823)
• [fix][test]: Flaky-test: GetPartitionMetadataMultiBrokerTest.testCompatibilityDifferentBrokersForNonPersistentTopic (#23666)

For the complete list, check the full changelog.

v3.3.3

2024-12-03

Library updates

• [fix][sec] Drop hdfs2 support, Upgrade hadoop3 to 3.4.0 and dnsjava to 3.6.2 to address CVE-2024-25638 (#23411)
• [fix][sec] Replace bcprov-jdk15on dependency with bcprov-jdk18-on (#23532)
• [fix][sec] Upgrade Zookeeper to 3.9.3 to address CVE-2024-51504 (#23581)
• [fix][sec] Upgrade to Netty 4.1.115.Final to address CVE-2024-47535 (#23596)
• [improve] Upgrade OpenTelemetry library to 1.44.1 version (#23656)
• [improve] Upgrade oxia-java to 0.4.10 and fix closing of OxiaMetadataStore (#23653)

Broker

• [fix][broker] Avoid orphan ledgers in BucketDelayedDeliveryTracker (#22802)
• [fix][broker] Broker is failing to create non-durable sub if topic is fenced (#23579)
• [fix][broker] Fix AvgShedder strategy check (#23156)
• [fix][broker] Fix Broker migration NPE while broker tls url not configured (#23534)
• [fix][broker] Fix currently client retries until operation timeout if the topic does not exist (#23530)
• [fix][broker] Fix failed TokenAuthenticatedProducerConsumerTest (#23602)
• [fix][broker] Fix namespace unload might be blocked too long with extensible load manager (#23433) (#23517)
• [fix][broker] Fix ownership loss (#23515)
• [fix][broker] Fix print cluster migration state response (#23535)
• [fix][broker] Fix race-condition causing repeated delete topic (#23522)
• [fix][broker] Fix the broker registering might be blocked for long time (#23371) (#23507)
• [fix][broker] Fix unloadNamespaceBundlesGracefully can be stuck with extensible load manager (#23349) (#23496)
• [fix][broker] Increase readBuffer size for bookkeeper.DLOutputStream (#23548)
• [fix][broker] Topics failed to delete after remove cluster from replicated clusters set and caused OOM (#23360)
• [fix][broker] fix logging with correct error message while loading the topic (#23544)
• [fix][broker] fix null lookup result when brokers are starting (#23642)
• [fix][broker] normalize path (#23438)
• [fix][broker] timeout when broker registry hangs and monitor broker registry (ExtensibleLoadManagerImpl only) (#23382) (#23510)
• [improve][broker] Add log to track issue when handleGetTopicsOfNamespace (#23434)
• [improve][broker] Clear thread local BrokerEntryMetadata instance before reuse (#22752)
• [improve][broker] Decouple pulsar_storage_backlog_age_seconds metric with backlogQuota check (#23619)
• [improve][broker] Exclude system topics from namespace level publish and dispatch rate limiting (#23589)
• [improve][broker] Make cluster metadata init command support metadata config path (#23269)
• [improve][broker] Make cluster metadata teardown command support metadata config path (#23520)
• [improve][broker] PIP-383: Support granting/revoking permissions for multiple topics (#23372)
• [improve][broker] PIP-392: Add configuration to enable consistent hashing to select active consumer for partitioned topic (#23584)
• [improve][broker] Skip unloading when bundle throughput is zero (ExtensibleLoadManagerImpl only) (#23626)
• [improve][broker] Support cleanup replication cluster and allowed cluster when cluster metadata teardown (#23561)
• [improve][broker] re-elect the channel owner if no channel owner is found (#23516) (#23580)
• [improve][broker] replace HashMap with inner implementation ConcurrentLongLongPairHashMap in Negative Ack Tracker. (#23582)

Client

• [fix][admin] Fix lookup get a null result if uses proxy (#23556)
• [fix][client] Fix DLQ producer name conflicts when there are same name consumers (#23577)
• [fix][client] Fix Reader.hasMessageAvailable return wrong value after seeking by timestamp with startMessageIdInclusive (#23502)
• [fix][client] Fix ReaderBuilder doest not give illegalArgument on connection failure retry (#22639)
• [fix][client] Fix deadlock of NegativeAcksTracker (#23651)
• [fix][client] Fix producer/consumer stop to reconnect or Pub/Sub due to IO thread race-condition (#23499)
• [fix][client] Fix race-condition causing doReconsumeLater to hang when creating retryLetterProducer has failed (#23560)
• [fix][client] Fix the javadoc for ConsumerBuilder.isAckReceiptEnabled (#23452)
• [fix][client] Fixed an issue where a cert chain could not be used in TLS authentication (#23644)
• [fix][client] Initializing client-authentication using configured auth params (#23610)
• [fix][client] Make protobuf-java dependency optional in java client libraries (#23632)
• [fix][client] Prevent embedding protobuf-java class files in pulsar-client-admin and pulsar-client-all (#23468)
• [fix][client] The partitionedProducer maxPendingMessages always is 0 (#23593)
• [fix][client] Use dedicated executor for requests in BinaryProtoLookupService (#23378)
• [fix][client] fix incomingMessageSize and client memory usage is negative (#23624)
• [fix][client] fix the beforeConsume() method earlier hit with message listener (#23578)
• [improve][admin] Print error log if handle http response fails (#23563)
• [improve][client] Enhance error handling for non-exist subscription in consumer creation (#23254)
• [improve][client] Increase default Java client connectionMaxIdleSeconds to 60 seconds (#23430)
• [improve][client] Reduce unshaded dependencies and shading warnings in shaded Java client modules (#23647)
• [improve][client] Replace NameUtil#generateRandomName with RandomStringUtils#randomAlphanumeric (#23645)

Pulsar IO and Pulsar Functions

• [fix][fn] ack messages for window function when its result is null (#23618)
• [improve][io] Support update subscription position for sink connector (#23538)
• [improve][io] Upgrade Spring version to 6.1.13 in IO Connectors (#23459)

Others

• [fix][misc] Class conflict during jetcd-core-shaded shading process (#23641)
• [fix][misc] Unable to connect an etcd metastore with recent releases due to jetc-core sharding problem (#23604)
• [fix][ml] Managed ledger should recover after open ledger failed (#23368)
• [fix][proxy] Fix pattern consumer does not work when using Proxy (#23489)
• [fix][ws] Implement missing http header data functions in AuthenticationDataSubscription (#23638)
• [improve] Improve logic for enabling Netty leak detection (#23613)
• [improve] Use single buffer for metrics when noUnsafe use (#23612)
• [improve][misc] Disable OTel by default when running the pulsar-perf tool (#23585)
• [improve][misc] Upgrade Jetty to 9.4.56.v20240826 (#23405)
• [improve][ml] Avoid repetitive nested lock for isMessageDeleted in ManagedCursorImpl ([#23609](htt...

v4.0.1

2024-12-02

Library updates

• [feat][meta] Bump oxia java version from 0.4.5 to 0.4.7 (#23471)
• [feat][misc] Upgrade oxia version to 0.4.9 (#23607)
• [fix][sec] Replace bcprov-jdk15on dependency with bcprov-jdk18-on (#23532)
• [fix][sec] Upgrade Zookeeper to 3.9.3 to address CVE-2024-51504 (#23581)
• [fix][sec] Upgrade to Netty 4.1.115.Final to address CVE-2024-47535 (#23596)
• [improve][monitor] Upgrade OTel to 1.41.0 (#23484)
• [improve] Upgrade OpenTelemetry library to 1.44.1 version (#23656)
• [improve] Upgrade oxia-java to 0.4.10 and fix closing of OxiaMetadataStore (#23653)

Broker

• [feat][broker] Implement allowBrokerOperationAsync in PulsarAuthorizationProvider to avoid exception thrown (#23663)
• [fix][broker] Broker is failing to create non-durable sub if topic is fenced (#23579)
• [fix][broker] Fix Broker migration NPE while broker tls url not configured (#23534)
• [fix][broker] Fix currently client retries until operation timeout if the topic does not exist (#23530)
• [fix][broker] Fix failed TokenAuthenticatedProducerConsumerTest (#23602)
• [fix][broker] Fix ownership loss (#23515)
• [fix][broker] Fix print cluster migration state response (#23535)
• [fix][broker] Fix race-condition causing repeated delete topic (#23522)
• [fix][broker] Increase readBuffer size for bookkeeper.DLOutputStream (#23548)
• [fix][broker] Key_Shared subscription: Reject consumers with incompatible policy (#23449)
• [fix][broker] Revert "[improve][client] Add log when can't add message to the container (#23657)
• [fix][broker] fix logging with correct error message while loading the topic (#23544)
• [fix][broker] fix null lookup result when brokers are starting (#23642)
• [fix][broker] support missing tenant level fine-granted permissions (#23660)
• [fix][broker]: support missing broker level fine-granted permissions (#23637)
• [improve][broker] Decouple pulsar_storage_backlog_age_seconds metric with backlogQuota check (#23619)
• [improve][broker] Exclude system topics from namespace level publish and dispatch rate limiting (#23589)
• [improve][broker] Make cluster metadata teardown command support metadata config path (#23520)
• [improve][broker] PIP-392: Add configuration to enable consistent hashing to select active consumer for partitioned topic (#23584)
• [improve][broker] Reduce memory occupation of InMemoryRedeliveryTracker. (#23640)
• [improve][broker] Reduce memory occupation of the delayed message queue (#23611)
• [improve][broker] Skip unloading when bundle throughput is zero (ExtensibleLoadManagerImpl only) (#23626)
• [improve][broker] Support cleanup replication cluster and allowed cluster when cluster metadata teardown (#23561)
• [improve][broker] re-elect the channel owner if no channel owner is found (#23516)
• [improve][broker] replace HashMap with inner implementation ConcurrentLongLongPairHashMap in Negative Ack Tracker. (#23582)

Client

• [fix][admin] Fix lookup get a null result if uses proxy (#23556)
• [fix][client] Fix ConsumerStats.getRateMsgsReceived javadoc (#21618)
• [fix][client] Fix DLQ producer name conflicts when there are same name consumers (#23577)
• [fix][client] Fix Reader.hasMessageAvailable return wrong value after seeking by timestamp with startMessageIdInclusive (#23502)
• [fix][client] Fix deadlock of NegativeAcksTracker (#23651)
• [fix][client] Fix producer/consumer stop to reconnect or Pub/Sub due to IO thread race-condition (#23499)
• [fix][client] Fix race-condition causing doReconsumeLater to hang when creating retryLetterProducer has failed (#23560)
• [fix][client] Fix the javadoc for startMessageIdInclusive (#23508)
• [fix][client] Fixed an issue where a cert chain could not be used in TLS authentication (#23644)
• [fix][client] Initializing client-authentication using configured auth params (#23610)
• [fix][client] Make protobuf-java dependency optional in java client libraries (#23632)
• [fix][client] Prevent embedding protobuf-java class files in pulsar-client-admin and pulsar-client-all (#23468)
• [fix][client] The partitionedProducer maxPendingMessages always is 0 (#23593)
• [fix][client] Use dedicated executor for requests in BinaryProtoLookupService (#23378)
• [fix][client] fix incomingMessageSize and client memory usage is negative (#23624)
• [fix][client] fix the beforeConsume() method earlier hit with message listener (#23578)
• [improve][admin] Print error log if handle http response fails (#23563)
• [improve][client] Enhance error handling for non-exist subscription in consumer creation (#23254)
• [improve][client] Reduce unshaded dependencies and shading warnings in shaded Java client modules (#23647)
• [improve][client] Replace NameUtil#generateRandomName with RandomStringUtils#randomAlphanumeric (#23645)

Pulsar IO and Pulsar Functions

• [fix][fn] ack messages for window function when its result is null (#23618)
• [improve][io] Support update subscription position for sink connector (#23538)
• [improve][io] Upgrade Spring version to 6.1.13 in IO Connectors (#23459)
• [improve][io] Upgrade Spring version to 6.1.14 in IO Connectors (#23481)

Others

• [feat][monitor] Add offloader stats grafana dashboard (#23479)
• [fix][misc] Class conflict during jetcd-core-shaded shading process (#23641)
• [fix][misc] Unable to connect an etcd metastore with recent releases due to jetc-core sharding problem (#23604)
• [fix][proxy] Fix pattern consumer does not work when using Proxy (#23489)
• [fix][standalone] correctly delete bookie registration znode (#23497)
• [fix][ws] Implement missing http header data functions in AuthenticationDataSubscription (#23638)
• [improve] Improve logic for enabling Netty leak detection (#23613)
• [improve] Use single buffer for metrics when noUnsafe use (#23612)
• [improve][misc] Disable OTel by default when running the pulsar-perf tool (#23585)
• [improve][ml] Avoid repetitive nested lock for isMessageDeleted in ManagedCursorImpl (#23609)
• [improve][offload] Use filesystemURI as the storage path (#23591)
• Enabling DNS retryOnTimeout with TCP in DnsNameResolver (#23590)

Tests & CI

• [cleanup][build] skip generating pom.xml.versionsBackup (#23639)
• [fix][build] Fix error "Element encoding is not allowed here" in pom.xml (#23655)
• [fix][test] Fix DeadLetterTopicTest.testDeadLetterTopicWithInitialSubscriptionAndMultiConsumers (#23552)
• [fix][test] Fix ManagedCursorTest.testForceCursorRecovery (#23518)
• [fix][test] Fix SimpleProducerConsumerT...

v3.0.8

2024-12-02

Library updates

• [fix][sec] Upgrade Zookeeper to 3.9.3 to address CVE-2024-51504 (#23581)
• [fix][sec] Upgrade to Netty 4.1.115.Final to address CVE-2024-47535 (#23596)
• [improve][misc] Upgrade Jetty to 9.4.56.v20240826 (#23405)

Broker

• [fix][broker] Avoid orphan ledgers in BucketDelayedDeliveryTracker (#22802)
• [fix][broker] Broker is failing to create non-durable sub if topic is fenced (#23579)
• [fix][broker] Continue closing even when executor is shut down (#22599)
• [fix][broker] Fix AvgShedder strategy check (#23156)
• [fix][broker] Fix currently client retries until operation timeout if the topic does not exist (#23530)
• [fix][broker] Fix failed TokenAuthenticatedProducerConsumerTest (#23602)
• [fix][broker] Fix ownership loss (#23515)
• [fix][broker] Fix race-condition causing repeated delete topic (#23522)
• [fix][broker] Increase readBuffer size for bookkeeper.DLOutputStream (#23548)
• [fix][broker] Topics failed to delete after remove cluster from replicated clusters set and caused OOM (#23360)
• [fix][broker] fix null lookup result when brokers are starting (#23642)
• [fix][broker] normalize path (#23438)
• [improve][broker] Add log to track issue when handleGetTopicsOfNamespace (#23434)
• [improve][broker] Close TopicPoliciesService to allow Pulsar broker graceful shutdown (#22589)
• [improve][broker] Decouple pulsar_storage_backlog_age_seconds metric with backlogQuota check (#23619)
• [improve][broker] Don't use forkjoin pool by default for deleting partitioned topics (#22598)
• [improve][broker] Exclude system topics from namespace level publish and dispatch rate limiting (#23589)
• [improve][broker] Make cluster metadata init command support metadata config path (#23269)
• [improve][broker] Make cluster metadata teardown command support metadata config path (#23520)
• [improve][broker] PIP-392: Add configuration to enable consistent hashing to select active consumer for partitioned topic (#23584)
• [improve][broker] Support cleanup replication cluster and allowed cluster when cluster metadata teardown (#23561)
• [improve][broker] replace HashMap with inner implementation ConcurrentLongLongPairHashMap in Negative Ack Tracker. (#23582)

Client

• [fix][admin] Fix lookup get a null result if uses proxy (#23556)
• [fix][client] Fix Reader.hasMessageAvailable return wrong value after seeking by timestamp with startMessageIdInclusive (#23502)
• [fix][client] Fix deadlock of NegativeAcksTracker (#23651)
• [fix][client] Fix producer/consumer stop to reconnect or Pub/Sub due to IO thread race-condition (#23499)
• [fix][client] Fix race-condition causing doReconsumeLater to hang when creating retryLetterProducer has failed (#23560)
• [fix][client] Fix the javadoc for ConsumerBuilder.isAckReceiptEnabled (#23452)
• [fix][client] Fixed an issue where a cert chain could not be used in TLS authentication (#23644)
• [fix][client] Initializing client-authentication using configured auth params (#23610)
• [fix][client] Make protobuf-java dependency optional in java client libraries (#23632)
• [fix][client] Prevent embedding protobuf-java class files in pulsar-client-admin and pulsar-client-all (#23468)
• [fix][client] The partitionedProducer maxPendingMessages always is 0 (#23593)
• [fix][client] Use dedicated executor for requests in BinaryProtoLookupService (#23378) (#23461)
• [fix][client] fix incomingMessageSize and client memory usage is negative (#23624)
• [fix][client] fix the beforeConsume() method earlier hit with message listener (#23578)
• [improve][admin] Print error log if handle http response fails (#23563)
• [improve][client] Enhance error handling for non-exist subscription in consumer creation (#23254)
• [improve][client] Increase default Java client connectionMaxIdleSeconds to 60 seconds (#23430)
• [improve][client] Reduce unshaded dependencies and shading warnings in shaded Java client modules (#23647)

Pulsar IO and Pulsar Functions

• [fix][fn] ack messages for window function when its result is null (#23618)
• [fix][sql][branch-3.0] Fix long decimal compatibility in Trino 368. (#23419)
• [fix][sql][branch-3.0] Fix shading configuration for presto-pulsar
• [improve][io] Support update subscription position for sink connector (#23538)
• [improve][io] Upgrade Spring version to 6.1.13 in IO Connectors (#23459)

Others

• [fix] Restored method as deprecated in AbstractMetadataStore (#21950)
• [fix][misc] Class conflict during jetcd-core-shaded shading process (#23641)
• [fix][misc] Unable to connect an etcd metastore with recent releases due to jetc-core sharding problem (#23604)
• [fix][ml] Managed ledger should recover after open ledger failed (#23368)
• [fix][proxy] Fix pattern consumer does not work when using Proxy (#23489)
• [fix][standalone] correctly delete bookie registration znode (#23497)
• [improve] Improve logic for enabling Netty leak detection (#23613)
• [improve] Use single buffer for metrics when noUnsafe use (#23612)
• [fix][ws] Implement missing http header data functions in AuthenticationDataSubscription (#23638)
• [improve][ml] Avoid repetitive nested lock for isMessageDeleted in ManagedCursorImpl (#23609)
• [improve][offload] Use filesystemURI as the storage path (#23591)
• Enabling DNS retryOnTimeout with TCP in DnsNameResolver (#23590)

Tests & CI

• [cleanup][build] skip generating pom.xml.versionsBackup (#23639)
• [fix][build] Add basic support for vscode-java and Eclipse IDE (#23448)
• [fix][build] Fix error "Element encoding is not allowed here" in pom.xml (#23655)
• [fix][test] Address flaky GetPartitionMetadataMultiBrokerTest (#23456)
• [fix][test] Fix DeadLetterTopicTest.testDeadLetterTopicWithInitialSubscriptionAndMultiConsumers (#23552)
• [fix][test] Fix SimpleProducerConsumerTest.testMultiTopicsConsumerImplPauseForManualSubscription (#23546)
• [fix][test] Fix flaky GetPartitionMetadataMultiBrokerTest.testCompatibilityDifferentBrokersForNonPersistentTopic (#23259)
• [fix][test] Fix flaky test ManagedLedgerTest.testDeleteCurrentLedgerWhenItIsClosed (#23437)
• [fix][test][branch-3.0] Fix PrecisTopicPublishRateThrottleTest that broke after #23589 changes
• [fix][test][branch-3.0] Fix cherry-picking issue in 2f6c1a3 where Cleanup import was missing
• [improve][build] Require Java 17 for building Pulsar branch-3.0 (#22875)
• [improve][build] Update maven-wrapper (mvnw) to recent stable version 3.3.2 (#23410)
• [improve][build][branch-3.0] Upgrade docker-maven-plugin to 0.45.1
• [improve][test] Added message properties tests for batch and non-batch...

v4.0.0

2024-10-21

Changes in 4.0.0 release since 3.3.0 release preparation was started.
For the complete list, check the full changelog and full PR list.

What's New in Apache Pulsar 4.0? covers topics such as:

• Enhanced Key_Shared subscription implementation
• Secure Docker runtime based on Alpine Linux and Java 21
• Enhanced Quality of Service controls for multi-tenant deployments
• Rate limiting as a foundation for multi-tenant resource management
• Upgrade paths from previous versions
• Client compatibility and Importance of upgrading Java clients to address CVE-2024-47561

PIPs accepted since 3.0.0 release

PIPs accepted in 4.0.0 release

• PIP-327: Support force topic loading for unrecoverable errors (#21752)
• PIP-337: SSL Factory Plugin to customize SSL Context and SSL Engine generation (#22016)
• PIP-347: add role field in consumer's stat (#22564)
• PIP-348: Trigger offload on topic load stage (#22650)
• PIP-349: Add additionalSystemCursorNames ignore list for TTL check (#22651)
• PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis (#22688)
• PIP-351: Additional options for Pulsar-Test client to support KeyStore based TLS (#22694)
• PIP-352: Event time based topic compactor (#22710)
• PIP-353: Improve transaction message visibility for peek-messages (#22746)
• PIP-354: apply topK mechanism to ModularLoadManagerImpl (#22765)
• PIP-355: Enhancing Broker-Level Metrics for Pulsar (#22778)
• PIP-356: Support Geo-Replication starts at earliest position (#22806)
• PIP-357: Correct the conf name in load balance module. (#22823)
• PIP-358: let resource weight work for OverloadShedder, LeastLongTermMessageRate, ModularLoadManagerImpl. (#22889)
• PIP-359: Support custom message listener executor for specific subscription (#22902)
• PIP-360: Add admin API to display Schema metadata (#22913)
• PIP-363: Add callback parameters to the method: org.apache.pulsar.client.impl.SendCallback.sendComplete. (#22940)
• PIP-364: Introduce a new load balance algorithm AvgShedder (#22946)
• PIP-366: Support to specify different config for Configuration and Local Metadata Store (#23033)
• PIP-367: Propose a Contributor Repository for Pulsar (#23061)
• PIP-368: Support lookup based on the lookup properties (#23075)
• PIP-369: Flag based selective unload on changing ns-isolation-policy (#23116)
• PIP-370: configurable remote topic creation in geo-replication (#23124)
• PIP-374: Visibility of messages in receiverQueue for the consumers (#23235)
• PIP-376: Make topic policies service pluggable (#23248)
• PIP-378: Add ServiceUnitStateTableView abstraction (ExtensibleLoadMangerImpl only) (#23300)
• PIP-379: Key_Shared Draining Hashes for Improved Message Ordering (#23309)
  • Documentation: Preserving order of message delivery by key
• PIP-381: Handle large PositionInfo state (#23328)
• PIP-383: Support granting/revoking permissions for multiple topics (#23355)
• PIP-384: ManagedLedger interface decoupling (#23363)

PIPs accepted in 3.3.0 release

• PIP-315: Configurable max delay limit for delayed delivery (#21798)
• PIP-321: Introduce allowed-cluster at the namespace level (#21648)
• PIP-324: Alpine Docker images (#21716)
• PIP-325: Add command to abort transaction (#21731)
• PIP-326: Create a BOM to ease dependency management (#21747)
• PIP-329: Strategy for maintaining the latest tag to Pulsar docker images (#21872)
• PIP-330: getMessagesById gets all messages (#21873)
• PIP-342: Support OpenTelemetry metrics in Pulsar client (#22178)
• PIP-343: Use picocli instead of jcommander (#22181)
• PIP-344: Correct the behavior of the public API pulsarClient.getPartitionsForTopic(topicName) (#22182)
• PIP-335: Oxia metadata plugin (#22009)
• PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources (#22071)
• PIP-282: Change definition of the recently joined consumers position (#20776)

PIPs accepted in 3.2.0 release

• PIP-323: Complete Backlog Quota Telemetry (#21709)
• PIP-322: Pulsar Rate Limiting Refactoring (#21680)
• PIP-320: OpenTelemetry Scaffolding (#21635)
• PIP-318: Don't retain null-key messages during topic compaction (#21541)
• PIP-315: Configurable max delay limit for delayed del...