Skip to content

[improve][client] Add SOCKS5 proxy support for PulsarAdmin and for PulsarClient HTTP lookups#25575

Merged
Jason918 merged 5 commits intoapache:masterfrom
wolfstudy:xiaolongran/support-socket5-for-admin
Apr 28, 2026
Merged

[improve][client] Add SOCKS5 proxy support for PulsarAdmin and for PulsarClient HTTP lookups#25575
Jason918 merged 5 commits intoapache:masterfrom
wolfstudy:xiaolongran/support-socket5-for-admin

Conversation

@wolfstudy
Copy link
Copy Markdown
Member

@wolfstudy wolfstudy commented Apr 24, 2026

Fixes #xyz

Motivation

Historically the PulsarClient Java client supported a SOCKS5 proxy, but the proxy was
wired into the Pulsar binary protocol path only (PulsarChannelInitializer). This led to
two gaps that made Pulsar unusable in network environments where brokers are only
reachable through a SOCKS5 jump host:

  1. PulsarAdmin had no SOCKS5 support at all. PulsarAdminBuilder exposed no
    socks5ProxyAddress(...) / socks5ProxyUsername(...) / socks5ProxyPassword(...)
    methods, and the underlying AsyncHttpConnector never configured a proxy on its
    async-http-client. Users in restricted networks therefore could not run any admin
    operation (topic/tenant/namespace/cluster REST calls) against the broker.

  2. HTTP lookups inside PulsarClient silently ignored the SOCKS5 proxy. When a user
    configured PulsarClient with an http(s):// serviceUrl, or when the failover
    HTTP client kicked in, the HttpClient used by the lookup service built an
    async-http-client without the proxy, so HTTP lookups bypassed SOCKS5 even though the
    user had explicitly asked for it. This was surprising and poorly documented.

This PR closes both gaps while preserving 100% backward compatibility with the
pre-existing PulsarClient behavior.

Modifications

  1. New public API: Socks5ProxyScope enum (pulsar-client-api).
    A selector that tells the builder which kinds of connections should go through the
    configured SOCKS5 proxy:

    • BINARY_ONLY — only Pulsar binary protocol connections (pre-existing
      PulsarClient behavior, and the default for ClientBuilder).
    • HTTP_ONLY — only HTTP/HTTPS traffic (HTTP lookups, failover HTTP clients, and
      PulsarAdmin REST calls). Default for PulsarAdminBuilder.
    • BOTH — both binary and HTTP traffic.

    Helper methods appliesToBinary() / appliesToHttp() keep the call sites tidy.

  2. PulsarAdmin SOCKS5 support.

    • Added socks5ProxyAddress, socks5ProxyUsername, socks5ProxyPassword, and
      socks5ProxyScope builder methods on PulsarAdminBuilder, implemented in
      PulsarAdminBuilderImpl.
    • PulsarAdminBuilderImpl defaults the scope to HTTP_ONLY because admin traffic
      is HTTP-only, so a user who only configures socks5ProxyAddress(...) gets the
      intuitive behavior out of the box.
    • AsyncHttpConnector now builds an async-http-client ProxyServer of type
      SOCKS_V5 when a proxy address is set and the scope includes HTTP. Optional
      BASIC auth is applied when a username is configured.
  3. PulsarClient HTTP SOCKS5 support (backward-compatible).

    • HttpClient (used by the HTTP lookup path and the failover HTTP clients inside
      PulsarClient) now wires a SOCKS5 ProxyServer onto its async-http-client
      exactly the same way AsyncHttpConnector does.
    • The default scope for ClientBuilder stays BINARY_ONLY, so existing users who
      configured socks5ProxyAddress(...) on PulsarClient continue to see the old
      behavior — HTTP lookups do NOT go through the proxy. Users who want the new
      behavior opt in explicitly via socks5ProxyScope(Socks5ProxyScope.HTTP_ONLY) or
      Socks5ProxyScope.BOTH.
    • PulsarChannelInitializer.initSocks5IfConfig(...) now additionally checks
      socks5ProxyScope.appliesToBinary(), so users can disable the proxy for the
      binary protocol while keeping it for HTTP.
  4. Config plumbing.

    • New field socks5ProxyScope on ClientConfigurationData (defaults to
      BINARY_ONLY, overridden to HTTP_ONLY by PulsarAdminBuilderImpl).
    • New setters on both ClientBuilder and PulsarAdminBuilder with full Javadoc
      explaining the backward-compatibility contract.

Verifying this change

This change added tests and can be verified as follows:

  • Unit tests (PulsarAdminBuilderImplTest): verify that every new builder method
    on PulsarAdminBuilder correctly stores its value into ClientConfigurationData,
    including the default HTTP_ONLY scope, address/username/password round-trips, and
    scope overrides.
  • End-to-end test (new): boots an in-process SOCKS5 proxy, starts a mocked broker
    admin endpoint, and drives a real PulsarAdmin instance through
    socks5ProxyAddress(...). The test asserts that the admin call succeeds only when
    it is routed through the proxy (the proxy increments a counter) and fails when the
    proxy is taken down, proving that PulsarAdminBuilder actually honours the SOCKS5
    configuration end-to-end. This covers the reviewer's request to verify that SOCKS5
    "can be configured in the PulsarAdminBuilder".
  • Existing tests for PulsarChannelInitializer / binary-protocol SOCKS5 still
    pass unchanged, which is the regression signal that BINARY_ONLY is indeed the
    default on ClientBuilder.

Manual verification: run PulsarAdmin and PulsarClient against a local ssh -D
SOCKS5 tunnel with every Socks5ProxyScope value and confirm that binary/HTTP
traffic is (or is not) routed through the tunnel as documented.

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Highlighted public-API changes:

  • New enum org.apache.pulsar.client.api.Socks5ProxyScope (public, stable).
  • New methods on org.apache.pulsar.client.api.ClientBuilder:
    • ClientBuilder socks5ProxyScope(Socks5ProxyScope scope)
  • New methods on org.apache.pulsar.client.admin.PulsarAdminBuilder:
    • PulsarAdminBuilder socks5ProxyAddress(InetSocketAddress address)
    • PulsarAdminBuilder socks5ProxyUsername(String username)
    • PulsarAdminBuilder socks5ProxyPassword(String password)
    • PulsarAdminBuilder socks5ProxyScope(Socks5ProxyScope scope)

All additions are purely additive. No existing signatures are removed or changed, and
the default Socks5ProxyScope.BINARY_ONLY on ClientBuilder guarantees that
PulsarClient instances that only call socks5ProxyAddress(...) keep the exact
pre-existing behavior (binary protocol only, HTTP lookups unproxied).

Signed-off-by: xiaolongran <xiaolongran@tencent.com>
@wolfstudy wolfstudy self-assigned this Apr 24, 2026
Copy link
Copy Markdown
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add a end-to-end test to verify that the Socks proxy can be configured in the PulsarAdminBuilder. That doesn't seem to be possible at the moment.

@lhotari
Copy link
Copy Markdown
Member

lhotari commented Apr 24, 2026

In addition, it would be useful to create a separate PR to add Socks proxy support for https serviceUrls. The Socks proxy in the Pulsar client is only used for the Pulsar binary protocol at the moment. In that case, there would be a need to specify whether the socks proxy should be used for both http lookups as well as Pulsar binary protocol connections to brokers. The Pulsar client has also http clients in failover implementations.

@wolfstudy
Copy link
Copy Markdown
Member Author

In addition, it would be useful to create a separate PR to add Socks proxy support for https serviceUrls. The Socks proxy in the Pulsar client is only used for the Pulsar binary protocol at the moment. In that case, there would be a need to specify whether the socks proxy should be used for both http lookups as well as Pulsar binary protocol connections to brokers. The Pulsar client has also http clients in failover implementations.

Thanks lhotari. That is a great suggestion. Adding a toggle switch here to control different behaviors is indeed a better implementation approach. I will readjust and revise accordingly.

@wolfstudy wolfstudy changed the title [improve][admin] Support SOCKS5 proxy for PulsarAdmin HTTP client [improve][client] Add SOCKS5 proxy support for PulsarAdmin and for PulsarClient HTTP lookups Apr 24, 2026
Signed-off-by: xiaolongran <xiaolongran@tencent.com>
Signed-off-by: xiaolongran <xiaolongran@tencent.com>
Signed-off-by: xiaolongran <xiaolongran@tencent.com>
@wolfstudy
Copy link
Copy Markdown
Member Author

@lhotari Please take a look again, thanks

Signed-off-by: xiaolongran <xiaolongran@tencent.com>
Copy link
Copy Markdown
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wolfstudy wolfstudy requested a review from Jason918 April 27, 2026 12:35
Copy link
Copy Markdown
Contributor

@hanmz hanmz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown
Contributor

@Jason918 Jason918 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Great work!

@Jason918 Jason918 merged commit 03ed3af into apache:master Apr 28, 2026
80 of 82 checks passed
@Jason918 Jason918 added this to the 4.3.0 milestone Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants