v2.8.0.1

Features

• Add entry formatter to do conversions between MemoryRecords and ByteBuf #280
• Add KafkaEntryFormatter #288
• Collect publish rate #285
• Support continuous offset for KoP #296
• Add kstream aggregation tests #298
• Add advertised listeners as the listeners exposed to client #317
• Simplify peekOffsetFromEntry #328
• Change default offsets topic number partitions to 8 #359
• Upgrade pulsar with LightProto support #337
• Support delete topics using the kafka client #348
• Change pulsar dependency jars to provided scope to reduce nar package size #391
• Perf tuning in decode phase #394
• Support __consumer_offsets topic compaction #401
• Support OAUTHBEARER mechanism for KoP #400
• Add produce message validation #403
• Add KoP's own callback handlers for OAuth 2.0 authentication #405 #406 #409
• Add prometheus metric exporter framework #389
• Add retention policy for offset topic #433
• Add request related metrics #441
• Use setMetadataFromEntryData API to retrieve offset #443
• Memory tuning for decode phase #447
• Add message TTL for Kafka metadata namespace #452
• Add inner topic protection #451 #486
• Prevent queue too large and request timeout #429
• Don't save KafkaHeaderAndRequest in a real queue #457
• Use AtomicBoolean as KafkaTopicManager's state #459
• Remove CompositeBytebuf to avoid on-heap memory copy #454
• Refactor writeAndFlushResponseToClient #465
• Optimize produce request handler #466
• Add metrics related to FETCH request #467
• Add produce/fetch timeout #490
• Use LinkedBlockingQueue as the container of ResponseAndRequest #500
• KoP metrics support label and add all API's latency metric #514
• Add metrics for offset commit #519
• Remove OffsetAcker #521
• Expose consumer stats to prometheus instead of consumer internal stats #520
• Apply getPartitionedTopicMetadata API with correct semantics #522
• Add producer stats to prometheus #536
• Refactor FETCH request handler to reduce GC #538
• Handle group name was failed to be written to ZK gracefully #554
• Enable Offset out-of-range checking #531
• Throw IllegalStateException if group/transaction coordinators fail #567
• Replace SslContextFactory with SslContextFactory.Server #570

Bug Fixed

• Remove unnecessary topics cache for Topic Manager #267
• Fix PulsarEntryFormatter not handle null value correctly #305
• Handle OffsetCommitRequest retentionTime field convert properly #311
• Fix high watermark and log end offset error #330
• Fix wrong offset in send callback #335
• Fix ListOffset returns wrong value #338
• Revert 9a4be21 for close configuring channel auto-read #356
• Fix remove consumer error if the broker restarts #350
• Remove lookup and address cache if can not get broker address when a broker node lost #352
• Remove schedule thread to avoid TimeoutException #358
• Fix can not consume all messages if topic has > 1000 partitions #354
• Avoid unexpected ConflictException when metadata is created #390
• Close offset consumers when channel is inactive #425
• Fix consumers map deadlock bug #426
• Fix AdminManager memory leak #438
• Create cluster if it doesn't exist during metadata initialisation #445
• Fix bundle unload bug #404
• Fix NPE in MessageFetchContext #480
• Fix wrong semantics and usage of KafkaTopicManager#getTopic #473
• Fix publish throttling and LoadManager API for pulsar upgrade #488
• Fix null KafkaTopicConsumerManager not removed #495
• Fix request queue size metric and add response queue metrics #501
• Avoid NPE when removing Producer from the associated PersistentTopic #511
• Fix topic creation may not complete after METADATA response is sent #506
• Fix ByteBuf release bug #518
• Add labels for request queued latency and fix request queue size bug #541
• Make DecodeResult list thread safe #547
• Fix deadlock caused by synchronous asyncFindPosition #556
• Fix direct memory leak bug #542

Transaction

• Make the basic publish and dispatch transaction messages process pass #295
• Fix transaction CI failure caused by thread sync problem #343
• Fix txn offset commit process #365
• Complete the ProducerIdManager component #372
• Fix the NPE if TransactionCoordinator is disabled #398
• Add ACL when creating node to avoid MarshallingError #399
• Transaction state management optimization #383
• Add transaction log key and transaction log value #424
• Add initialize transaction metadata #423