Support multiple Broker URL aliases (by supplied connection context)

[youurayy]

Currently (Pulsar 2.4.1), there is the possibility to use the Proxy component to hardcode alternate Broker URLs to be then used in discovery and sent back to clients which cannot use the primary Broker URLs (e.g. when these are Kubernetes Cluster IP type).

However this has the following drawbacks:

• only one alternate type is supported, e.g. I have to choose whether I support my AWS-internal or my AWS-external clients, but I cannot support both with the same Proxy
• there's no autodiscovery and the user has to manually manage this
• one whole component (the Proxy) is necessary to support this, this could be removed

The solution could go along these lines:

1. when connecting to a Pulsar Broker (i.e. pulsar://), user should be able to set a "context" (or the type of my origin) e.g. "external", "internal", "tunnel", etc. (perhaps via pulsar@context-name:// ? or via the client API). This context info is sent to the Broker with the initial request.
2. the Broker has multiple context lookup "providers" defined, i.e.:
   • hardcoded URL provider (comes from the config; currently provided by the Proxy component)
   • discovered URL provider; e.g. AWS k8s Pulsar culster will provide 3 URLs:
     • 1. Cluster IP:port from within Kubernetes
     • 2. AWS internal (private) node IP:port (if the k8s NodePort service for Broker is defined)
     • 3. AWS external (public) node IP:port (if the k8s NodePort service for Broker is defined)
3. when a redirection to another Broker is necessary, the Broker matches the client-provided context with a defined provider (by name, e.g. "aws-external"), and returns the correct other-Broker's URL for the client to (correctly) redirect to.

[tisonkun]

Closed as stale. Please open a new issue if it's still relevant in maintained versions.