v2.4.0

Apache

2.4.0 — 2019-06-30

PIPs

• PIP-26: Added support for delayed message delivery
  #4062, note that messages are only delayed on shared subscriptions.
• PIP-28: Pulsar Proxy Gateway Improvement #3915
• PIP-29: One package for both pulsar client and pulsar admin #3662
• PIP-30: Change authentication provider API to support mutual authentication for Kerberos Authentication #3677
  #3821 #3997
• PIP-32: Go Function API, Instance and LocalRun #3854 #4008 #4174
• PIP-33: Replicated subscriptions to keep subscription state in-sync across multiple geographical regions #4299 #4340 #4354 #4396
• PIP-34: Key_Shared subscription, Architecture for Key_Shared #4079 #4120 #4335 #4372 #4407 #4406 4462
• PIP-36: Support set message size in broker.conf #4247

Fixes

• Fix possible message loss using peer-cluster feature #3426
• Fix pulsar standalone does not read zk port from conf/standalone.conf #3790
• Fix some issues of ZkIsolatedBookieEnsemblePlacementPolicy #3917 #3918
• Fix NPE when unload non-existent topic #3946
• Fix race condition while deleting global topic #4173
• Fix deadlock on skip messages #4411
• Fix NPE when closing batch during a reconnection #4427
• Fix race condition of read-timeout task in managed ledger #4437
• Disable sticky read by default #4526
• Fix race condition between timeout-task and add-call complete #4455

Enhancements

• Optimize message replay for large backlog consumer #3732
• Added support for websocket produce/consume command #3835
• Added support for TTL config in broker.conf #3898
• Reduce memory used in ClientCnx for pending lookups #4104
• Reduce number of hashmap sections for ledger handles cache #4102
• Added backlog and offloaded size in Prometheus stats #4150
• Added support for configure the managed ledger cache eviction frequency #4066
• Added support to avoid payload copy when inserting into managed ledger cache #4197
• Added support to cache unack-messageId into OpenRangeSet #3819
• Added support configure static PulsarByteBufAllocator to handle OOM errors #4196
• Auto refresh new tls certs for jetty webserver #3645
• Create non-persistent topic by pulsar-admin/rest api #3625
• Consumer priority-level in Failover subscription #2954

Security

• Added support for other algorithms in token auth #4528

Namespace Policies

• Added support for tenant based bookie isolation #3933
• Added support for secondary bookie isolation group at namespace #4458
• Added support for secondary bookie-isolation-group #4261
• Added support for replicator rate limit between clusters #4273
• Disable backlog quota check by default #4320

Tiered Storage

• Added support for Pulsar SQL to read data from tiered storage #4045

Pulsar Schema

• Added schema versioning to support multi version messages produce and consume #3876 #3670 #4211 #4325 #4548
• Added TRANSITIVE schema check strategies to support compatibility check over all existing schemas #4214
• Added schema data validator #4360
• Added support for delete schema when deleting a topic #3941
• Added generic record builder #3690

Pulsar IO

• Added IO connector for flume source and sink #3597
• Added IO connector for redis sink #3700
• Added IO connector for solr sink #3885
• Hide kafka-connecter details for easy use debezium connector #3825
• Added IO connector for debezium PostgreSQL source #3924
• Enhancements for RabbitMQ source configuration #3937
• Added IO connector for RabbitMQ sink #3967
• Added IO connector for InfluxDB sink #4017

Pulsar functions

• Added support for authentication #3735 #3874 #4198
• Fix NPE when stats manager not initialized #3891
• Added async state manipulation methods #3798
• Fix fail to update functions in effectively-once mode #3993
• Added labels to function statefulsets and services #4038
• Added support for set key for message when using function publish #4005
• Use negative acknowledge to instead ackTimeout #4103
• Fix backward compatibility with 2.2 auth not working #4241

Java client

• Added negative acks #3703
• Added support for backoff strategy configuration #3848
• Added support for configure TypedMessageBuilder through a Map conf object #4015
• Added interceptor for negative ack send #3962
• Added support for seek operate on reader #4031
• Store key part of KeyValue schema into pulsar message keys #4117
• Added interceptor for ack timeout #4300
• Added support for snappy compression #4259
• Added support for key based batcher #4435

Python client

• Added negative acks #3816
• Added support for snappy compression #4319

Go client

• Added negative acks #3817
• Added support for go schema #3904
• Added support for snappy compression #4319
• Added support for Key_Shared subscription #4465

C++ client

• Added negative acks #3750
• Fix ack timeout when subscribing to regex topic #3897
• Added support for Key_Shared subscription #4366

Adaptors

• In Kafka client wrapper, added some configurations #3753 #3797 #3843 #3887 #3991
• In Apache Flink connector, allow to specify a custom Pulsar producer #3894 and client authentication #3949
• In Apache Flink connector, added support for accept ClientConfigurationData, ProducerConfigurationData, ConsumerConfigurationData #4232
• In Apache Storm connector, fix NPE while emitting next tuple #3991 and some add some enhancements #4280 #4239 #4238 #4236 #4495 #4494

For a complete list of issues fixed, see

https://github.com/apache/pulsar/milestone/20?closed=1

https://github.com/apache/pulsar/releases/tag/v2.4.0