[Bug] Pulsar Proxy Keep-Alive Interval is Hardcoded to 30s and Cannot Be Configured

[eminaktas]

Search before asking

• I searched in the issues and found nothing similar.

Read release policy

• I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

apachepulsar/pulsar-all:3.0.2

Minimal reproduce step

• Start a connection to the Pulsar Broker through Pulsar Proxy.
• Observe the debug logs—keep-alive is scheduled every 30 seconds, regardless of configuration settings.

What did you expect to see?

I expected the keep-alive interval for connections in Pulsar Proxy to be configurable through proxy.conf like as described here

What did you see instead?

The keep-alive interval is hardcoded to 30 seconds, with no way to modify it. The logs confirm this behavior:

2025-02-11T09:18:20,392+0000 [pulsar-proxy-io-2-8] DEBUG org.apache.pulsar.common.protocol.PulsarHandler - [[id: 0x4bf907e2, L:/xxx:6651 - R:/xxx:60882]] Scheduling keep-alive task every 30 s

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[lhotari]

@eminaktas Thanks for the enhancement request. I acknowledge that keep alive cannot be currently configured. To understand the full context, it would be useful to know what impact there is when it's not configurable. What problem are you trying to solve? Why would you need it to be configurable?

[eminaktas]

Hi @lhotari, thank you for your interest in this issue.

We are trying to reduce data usage by increasing the keep-alive intervals. We have managed this on the broker and client sides, but the Proxy is the only component where we couldn’t due to its hardcoded 30-second interval.

After a Pulsar client is created:
• One connection is established through the Proxy to get partition metadata — this connection is affected by the Proxy’s keep-alive.
• Connections to each broker for data transfer use the broker’s keep-alive settings, which work as expected.

Not being able to configure the Proxy’s keep-alive leads to unnecessary data usage, especially with many connections. Making it configurable would help align it with the rest of our Pulsar setup.

Looking forward to your feedback. Thank you!

[lhotari]

We are trying to reduce data usage by increasing the keep-alive intervals. We have managed this on the broker and client sides, but the Proxy is the only component where we couldn’t due to its hardcoded 30-second interval.

@eminaktas Could you elaborate on "trying to reduce data usage"?
Just wondering if increasing keep-alive intervals is an effective way to reduce data usage. What type of problem do you have with data usage? I guess a keep-alive message is a few bytes, but due to network transport layer such as ethernet, you'd always have to send at least one frame. What "data usage" are you trying to reduce? Are you reducing it to lower cloud network transfer costs or for some other reason?

In Pulsar Kubernetes deployments there's usually a lot of wasteful network traffic coming from non-optimal configurations so I'd assume that there would be more effective ways to reduce "data usage". The Apache Pulsar Helmchart doesn't come with optimized configurations that would take everything into account.

One major source of waste is when a proxy connects to a broker in a different AZ. There are optimizations where the lookup is handled in a way where the Pulsar lookup returns the address of the proxy in the same AZ as the target broker. If you are looking to reduce data usage, this would be a much more effective way to achieve your goal.
I'm aware of datastax#163 by @cbornet, but it seems that this improvement was never submitted to Apache Pulsar. That's part of the solution to achieve the solution where the Pulsar lookup in the proxy returns the address of the proxy in the same AZ as the target broker.

[eminaktas]

Just wondering if increasing keep-alive intervals is an effective way to reduce data usage. What type of problem do you have with data usage?

@lhotari Actually, our concern is on the client side. We are focused on decreasing traffic between the client and broker, and the client and proxy. By “client”, I mean devices that may have limited data availability, such as those using mobile networks. Our goal is to reduce data usage for these clients, where even small amounts of continuous traffic, like keep-alive messages, can add up over time. We successfully reduced keep-alive traffic for client-broker connections so far, but the client-proxy connection still remains.

[lhotari]

Just wondering if increasing keep-alive intervals is an effective way to reduce data usage. What type of problem do you have with data usage?

@lhotari Actually, our concern is on the client side. We are focused on decreasing traffic between the client and broker, and the client and proxy. By “client”, I mean devices that may have limited data availability, such as those using mobile networks. Our goal is to reduce data usage for these clients, where even small amounts of continuous traffic, like keep-alive messages, can add up over time. We successfully reduced keep-alive traffic for client-broker connections so far, but the client-proxy connection still remains.

@eminaktas Thanks for sharing, now the requirement of reducing data usage makes sense to me. I'll submit a minimal PR for addressing this.

[lhotari]

@eminaktas PR submitted: #23981

[eminaktas]

@lhotari Thank you 🙏

[cbornet]

I'm aware of datastax#163 by @cbornet, but it seems that this improvement was never submitted to Apache Pulsar. That's part of the solution to achieve the solution where the Pulsar lookup in the proxy returns the address of the proxy in the same AZ as the target broker.

I could upstream that PR if there's interest. Let me know.

[lhotari]

I'm aware of datastax#163 by @cbornet, but it seems that this improvement was never submitted to Apache Pulsar. That's part of the solution to achieve the solution where the Pulsar lookup in the proxy returns the address of the proxy in the same AZ as the target broker.

I could upstream that PR if there's interest. Let me know.

@cbornet Sure, that would be awesome! Having some basic javadocs about how to configure it for achieving the goal to reduce cross-AZ traffic would also be useful so that users could start adopting it.

[lhotari]

I'm aware of datastax#163 by @cbornet, but it seems that this improvement was never submitted to Apache Pulsar. That's part of the solution to achieve the solution where the Pulsar lookup in the proxy returns the address of the proxy in the same AZ as the target broker.

I could upstream that PR if there's interest. Let me know.

@cbornet Sure, that would be awesome! Having some basic javadocs about how to configure it for achieving the goal to reduce cross-AZ traffic would also be useful so that users could start adopting it.

@cbornet Another use case that came up on Pulsar Slack is being able to do portforwarding with kubectl to Pulsar Proxy and make use of it while connecting to brokers. It's not directly possible with your PR, but we could extend the support further.