Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DNS Cache that respects TTL of resolved DNS records #28

Closed
lidel opened this issue Apr 14, 2021 · 1 comment
Closed

Add DNS Cache that respects TTL of resolved DNS records #28

lidel opened this issue Apr 14, 2021 · 1 comment

Comments

@lidel
Copy link
Member

lidel commented Apr 14, 2021

Problem

When DNS over HTTPS resolver is set up, every query produces HTTP request.

To illustrate, when using https://resolver.cloudflare-eth.com/dns-query for crypto TLDs, this adds around 1 second to EVERY resolution event (only the very first one should be that slow):

$ t ipfs name resolve /ipns/brave.crypto
/ipfs/QmWrdNJWMbvRxxzLhojVKaBDswS4KNVM7LvjsN7QbDrvka
  0.25s user 0.03s system 21% cpu 1.274 total

$ t ipfs name resolve /ipns/brave.crypto                                                                                                                                                    /ipfs/QmWrdNJWMbvRxxzLhojVKaBDswS4KNVM7LvjsN7QbDrvka
  0.21s user 0.02s system 20% cpu 1.100 total

$ t ipfs name resolve /ipns/brave.crypto                                                                                                                                                   
/ipfs/QmWrdNJWMbvRxxzLhojVKaBDswS4KNVM7LvjsN7QbDrvka
  0.23s user 0.02s system 23% cpu 1.058 total

To illustrate, using non-DoH name leverages OS-level resolver as a cache, and only first lookup is expensive:

$ t ipfs name resolve /ipns/en.wikipedia-on-ipfs.org                                                                                                                                        
/ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
  0.23s user 0.05s system 80% cpu 0.343 total

$ t ipfs name resolve /ipns/en.wikipedia-on-ipfs.org                                                                                                                                        /ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
  0.04s user 0.01s system 173% cpu 0.029 total

$ t ipfs name resolve /ipns/en.wikipedia-on-ipfs.org                                                                                                                                        /ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
  0.03s user 0.02s system 160% cpu 0.032 total

Solution

The problem is not limited to DoH – there is no point in hitting OS-level default resolver if we could use cached record.
We need to add a global DNS Cache to resolve.go that makes sure we don't do redundant lookups until TTL from resolved record expires.

cc @aschmahmann @vyzo what would be the least painful way to solve this globally?

@vyzo
Copy link
Contributor

vyzo commented Apr 14, 2021

Following sync discussion, we decided that this is not something we can easily too without creating interface impedance mismatch.
Specifically, the default go resolver interface doesn't provide the TTL.
So instead, we decided to implement the cache inside the resolver implementations, where we have the RR record itself (and thus the TTL).

Implementation for DoH: libp2p/go-doh-resolver#3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants