Home

Pulsar Wiki

Pulsar Improvement Proposals

Important

This is a deprecated location for PIPs. All PIP documents have been migrated to the pip directory in the apache/pulsar repository. Please check the description of the Pulsar Improvement Proposal process in the new location.

We encourage you to document any big change or feature or any addition to public user APIs through a design document to be discussed with the community.

Newer PIPs are part of the Pulsar repository at https://github.com/apache/pulsar/tree/master/pip.

Search issues with label:pip

Proposed

• PIP-209: Separate C++/Python clients to own repositories
• PIP-201 : Extensions mechanism for Pulsar Admin CLI tools
• PIP 198: Standardize PR Naming Convention using GitHub Actions
• PIP-194: Pulsar client: seek command add epoch
• PIP-193: Sink preprocessing Function
• PIP-192: New Pulsar Broker Load Balancer
• PIP-191: Support batched message using entry filter
• PIP-189: No batching if only one message in batch
• PIP-188: Cluster migration or Blue-Green cluster deployment support in Pulsar
• PIP-187: Add API to analyse a subscription backlog and provide a accurate value
• PIP-186: Introduce two phase deletion protocol based on system topic
• PIP-184: Topic specific consumer priorityLevel
• PIP-183: Reduce unnecessary REST call in broker
• PIP-182: Provide new load balance placement strategy implementation for ModularLoadManagerStrategy
• PIP-181: Pulsar Shell
• PIP-180: Shadow Topic, an alternative way to support readonly topic ownership
• PIP-178: Multiple snapshots for transaction buffer
• PIP-177: Add the classLoader field for SchemaDefinition
• PIP-176: Refactor Doc Bot
• PIP-175: Extend time based release process
• PIP-174: Provide new implementation for broker dispatch cache
• PIP-173: Create a built-in Function implementing the most common basic transformations
• PIP-162: LTS Releases
• PIP-161: Exclusive Producer: new mode ExclusiveWithFencing
• PIP-157: Bucketing topic metadata to allow more topics per namespace
• PIP-156: Build and Run Pulsar Server on Java 17
• PIP-155: Drop support for Python2
• PIP-154: Max active transaction limitation for transaction coordinator
• PIP-148: Create Pulsar client release notes
• PIP-146: ManagedCursorInfo compression
• PIP-144: Making SchemaRegistry implementation configurable
• PIP-143: Support split bundle by specified boundaries
• PIP-137: Pulsar Client Shared State API
• PIP-136: Sync Pulsar policies across multiple clouds
• PIP-135: Include MetadataStore backend for Etcd
• PIP-130: Apply redelivery backoff policy for ack timeout
• PIP-124: Create init subscription before sending message to DLQ
• PIP-123: Introduce Pulsar metadata CLI tool
• PIP-122: Change loadBalancer default loadSheddingStrategy to ThresholdShedder
• PIP-121: Pulsar cluster level auto failover
• PIP-120: Enable client memory limit by default
• PIP-119: Enable consistent hashing by default on KeyShared dispatcher
• PIP-118: Do not restart brokers when ZooKeeper session expires
• PIP 117: Change Pulsar Standalone defaults
• PIP 116: Create Pulsar Writing Style Guide
• PIP 112: Generate Release Notes Automatically
• PIP-111: Add messages produced by Protocol Handler When checking maxMessagePublishBufferSizeInMB
• PIP 110: Topic metadata
• PIP 108: Add method to help user judge if consumer queue has message PIP
• PIP 107: Introduce the chunk message ID PIP
• PIP 106: Broker extensions to provide operators of enterprise-wide clusters better control and flexibility
• PIP 105: Support pluggable entry filter in Dispatcher
• PIP 104: Add new consumer type: TableView
• PIP 101: Add seek by index feature for consumer
• PIP 100: Pulsar pluggable topic factory
• PIP 97: Asynchronous Authentication Provider
• PIP 94: Support level increment delay for ReconsumerLater interface
• PIP 80: Unified namespace-level admin API
• PIP 79: Reduce redundant producers from partitioned producer
• PIP 59: gPRC Protocol Handler
• PIP 49: Permission levels and inheritance
• PIP 48: hierarchical admin api
• PIP 46: Next-gen Proxy
• PIP 35: Improve topic lookup for topics that have high number of partitions
• PIP 21: Pulsar Edge Component

Accepted

• PIP-265: PR-based system for managing and reviewing PIPs
• PIP-261: Restructure Getting Started section
• PIP-259: Make the config httpMaxRequestHeaderSize of the pulsar web server to configurable
• PIP-249: Pulsar website redesign
• PIP-243: Register Jackson Java 8 support modules by default
• PIP-205: Reactive Java client for Apache Pulsar
• PIP-204: Extensions for broker interceptor
• PIP-195: New bucket based delayed message tracker
• PIP-190: Simplify documentation release and maintenance strategy
• PIP-189: No batching if only one message in batch
• PIP-179: Support the admin API to check unknown request parameters
• PIP-165: Auto release client useless connections
• PIP-160: Make transactions work more efficiently by aggregation operation for transaction log and pending ack store
• PIP-152: Support subscription level dispatch rate limiter setting
• PIP 149: Making the REST Admin API fully async
• PIP 132: Include message header size when check maxMessageSize for non-batch message on the client side
• PIP 131: Resolve produce chunk messages failed when topic level maxMessageSize is set
• PIP 129: Introduce intermediate state for ledger deletion
• PIP 109: Introduce Bot to Improve Efficiency of Developing Docs
• PIP 108: Pulsar Feature Matrix (Client and Function)
• PIP 106: Negative acknowledgment backoff
• PIP 99: Pulsar Proxy Extensions
• PIP 98: Redesign Pulsar Information Architecture
• PIP 96: Message payload processor for Pulsar client
• PIP 95: Smart Listener Selection with Multiple Bind Addresses
• PIP 93: Transaction performance tools
• PIP 92: Topic policy across multiple clusters
• PIP 91: Separate lookup timeout from operation timeout
• PIP 90: Expose broker entry metadata to the client
• PIP 89: Structured document logging
• PIP 87: Upgrade Pulsar Website Framework (Docusaurus)
• PIP 88: Replicate schemas across multiple
• PIP 86: Pulsar Functions: Preload and release external resources
• PIP 85: Add Schema Information to Message in Java Client API
• PIP 85: Expose Pulsar-Client via Function/Connector BaseContext
• PIP 84: Pulsar client: Redeliver command add epoch
• PIP 83: Pulsar client: Message consumption with pooled buffer
• PIP 82: Tenant and namespace level rate limiting
• PIP 81: Split the individual acknowledgments into multiple entries
• PIP 78: Generate Docs from Code Automatically
• PIP 77: Contribute Supernova to Apache Pulsar
• PIP 76: Streaming Offload
• PIP 75: Replace protobuf code generator
• PIP 74: Pulsar client memory limits
• PIP 73: Configurable data source priority for message reading
• PIP 72: Introduce Pulsar Interface Taxonomy: Audience and Stability Classification
• PIP 71: Pulsar SQL migrate SchemaHandle to presto decoder
• PIP 70: Introduce lightweight broker entry metadata
• PIP 69: Schema design for Go client
• PIP 68: Exclusive Producer
• PIP 67: Pulsarctl - An alternative tools of pulsar-admin
• PIP 66: Pulsar Function Mesh
• PIP 65: Adapting Pulsar IO Sources to support Batch Sources
• PIP 64: Introduce REST endpoints for producing, consuming and reading messages
• PIP 63: Readonly Topic Ownership Support
• PIP 62: Move connectors, adapters and Pulsar Presto to separate repositories
• PIP 61: Advertised multiple addresses
• PIP 60: Support Proxy server with SNI routing
• PIP 58 : Support Consumers Set Custom Retry Delay
• PIP 57: Improve Broker's Zookeeper Session Timeout Handling
• PIP 56: Python3 Migration
• PIP 55: Refresh Authentication Credentials
• PIP 54: Support acknowledgement at batch index level
• PIP 53: Contribute DotPulsar to Apache Pulsar
• PIP 52: Message dispatch throttling relative to publish rate
• PIP 51: Tenant policy support
• PIP 50: Package Management
• PIP 47: Time Based Release Plan
• PIP 45: Pluggable metadata interface
• PIP 44: Separate schema compatibility checker for producer and consumer
• PIP 43: producer send message with different schema
• PIP 42: KoP - Kafka on Pulsar
• PIP 41: Pluggable Protocol Handler
• PIP 40: Pulsar Manager
• PIP 39: Namespace Change Events
• PIP 38: Batch Receiving Messages
• PIP 37: Large message size handling in Pulsar
• PIP 33: Replicated subscriptions
• PIP 36: Max Message Size
• PIP 34: Add new subscribe type Key_shared
• PIP 32: Go Function API, Instance and LocalRun
• PIP 31: Transaction Support
• PIP 30: change authentication provider API to support mutual authentication
• PIP 29: One package for both pulsar-client and pulsar-admin
• PIP 28: Pulsar Proxy Gateway Improvement
• PIP 27: Add checklist in github pull request template
• PIP 26: Delayed Message Delivery
• PIP 25: Token based authentication
• PIP 24: Simplify memory settings
• PIP 23: Message Tracing By Interceptors
• PIP 22: Pulsar Dead Letter Topic
• PIP 20: Mechanism to revoke TLS authentication
• PIP 19: Pulsar SQL
• PIP 17: Tiered storage for Pulsar topics
• PIP 16: Pulsar "instance" terminology change
• PIP 15: Pulsar Functions
• PIP 14: Topic compaction
• PIP 13: Subscribe to topics represented by regular expressions
• PIP 12: Introduce builder for creating Producer Consumer Reader
• PIP 11: Short topic names
• PIP 10: Remove cluster for namespace and topic names
• PIP 9: Adding more Security checks to Pulsar Proxy
• PIP 8: Pulsar beyond 1M topics
• PIP 7: Pulsar Failure domain and Anti affinity namespaces
• PIP 6: Guaranteed Message Deduplication
• PIP 5: Event time
• PIP 4: Pulsar End to End Encryption
• PIP 3: Message dispatch throttling
• PIP 2: Non Persistent topic
• PIP 1: Pulsar Proxy

Discarded

• PIP 18: Pulsar Replicator
• PIP 94: Message converter at broker level

Contributor documentation

• Community Meetings
• Release Candidate Validation
• Pulsar Improvement Proposal (PIP)

Committers documentation

• Committer Guide
• Create GPG keys to sign release artifacts
• Release process
• Tools and Tips

Security advisories

• Security advisories