Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proxy & Client] Configure Netty DNS resolver to match JDK DNS caching setting, share DNS resolver instance in Proxy #15219

Merged
merged 4 commits into from
Apr 20, 2022

Conversation

lhotari
Copy link
Member

@lhotari lhotari commented Apr 19, 2022

Motivation

By default, the Netty DNS resolver caches DNS entries until the max TTL for the DNS entry. This could be problematic if DNS resolution settings change more often than the max TTL.

Modifications

  • make Netty DNS resolver settings match the JDK DNS caching settings
    • with the exception that the max TTL is 60 seconds if DNS max TTL is the default (forever)
  • reuse the DNS resolver instance on the Proxy

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments
@lhotari lhotari added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/client area/proxy doc-not-needed Your PR changes do not impact docs labels Apr 19, 2022
@lhotari lhotari added this to the 2.11.0 milestone Apr 19, 2022
@lhotari lhotari self-assigned this Apr 19, 2022
Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

this change affects also the Broker and the Function worker, that is any component that uses the Pulsar Client.

Copy link
Member

@michaeljmarshall michaeljmarshall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

lhotari added a commit to datastax/pulsar that referenced this pull request Apr 20, 2022
…g setting, share DNS resolver instance in Proxy (apache#15219)

- make Netty DNS resolver settings match the JDK DNS caching settings
  - with the exception that the max TTL is 60 seconds if DNS max TTL is the default (forever)
- reuse the DNS resolver instance on the Proxy
@lhotari lhotari merged commit 39b1862 into apache:master Apr 20, 2022
lhotari added a commit to datastax/pulsar that referenced this pull request Apr 21, 2022
…g setting, share DNS resolver instance in Proxy (apache#15219)

* Align Netty DNS resolver cache settings with Java DNS cache settings

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments

* Share Netty DNSNameResolver in proxy

* Remove overriding of ConnectionPool.close since it's not necessary

* Address review comment: remove ProxyConnectionPool

(cherry picked from commit 39b1862)
lhotari added a commit to datastax/pulsar that referenced this pull request Apr 21, 2022
…g setting, share DNS resolver instance in Proxy (apache#15219)

* Align Netty DNS resolver cache settings with Java DNS cache settings

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments

* Share Netty DNSNameResolver in proxy

* Remove overriding of ConnectionPool.close since it's not necessary

* Address review comment: remove ProxyConnectionPool

(cherry picked from commit 39b1862)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Apr 28, 2022
…g setting, share DNS resolver instance in Proxy (apache#15219)

* Align Netty DNS resolver cache settings with Java DNS cache settings

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments

* Share Netty DNSNameResolver in proxy

* Remove overriding of ConnectionPool.close since it's not necessary

* Address review comment: remove ProxyConnectionPool

(cherry picked from commit 39b1862)
lhotari added a commit that referenced this pull request Apr 29, 2022
…g setting, share DNS resolver instance in Proxy (#15219)

* Align Netty DNS resolver cache settings with Java DNS cache settings

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments

* Share Netty DNSNameResolver in proxy

* Remove overriding of ConnectionPool.close since it's not necessary

* Address review comment: remove ProxyConnectionPool

(cherry picked from commit 39b1862)
@lhotari lhotari added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Apr 29, 2022
lhotari added a commit that referenced this pull request Apr 29, 2022
…g setting, share DNS resolver instance in Proxy (#15219)

* Align Netty DNS resolver cache settings with Java DNS cache settings

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments

* Share Netty DNSNameResolver in proxy

* Remove overriding of ConnectionPool.close since it's not necessary

* Address review comment: remove ProxyConnectionPool

(cherry picked from commit 39b1862)
@lhotari lhotari added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Apr 29, 2022
lhotari added a commit that referenced this pull request Apr 29, 2022
…g setting, share DNS resolver instance in Proxy (#15219)

* Align Netty DNS resolver cache settings with Java DNS cache settings

- Netty DNS resolver caches forever by default
  - this could cause problems with Kubernetes deployments

* Share Netty DNSNameResolver in proxy

* Remove overriding of ConnectionPool.close since it's not necessary

* Address review comment: remove ProxyConnectionPool

(cherry picked from commit 39b1862)
lhotari added a commit that referenced this pull request Jun 1, 2022
…g setting, share DNS resolver instance in Proxy (#15219)

- make Netty DNS resolver settings match the JDK DNS caching settings
  - with the exception that the max TTL is 60 seconds if DNS max TTL is the default (forever)
- reuse the DNS resolver instance on the Proxy

(cherry picked from commit f5adc17)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker area/client area/function area/proxy cherry-picked/branch-2.8 Archived: 2.8 is end of life cherry-picked/branch-2.9 Archived: 2.9 is end of life cherry-picked/branch-2.10 doc-not-needed Your PR changes do not impact docs release/2.8.4 release/2.9.3 release/2.10.1 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants