Add option to cache and prefetch records

[Fail-Safe]

As a future enhancement, it would be nice if the NextDNS client could cache records. This would be similar to the way in which dnsmasq and dnscrypt-proxy operate. Having this option would remove the need to have dnsmasq or unbound (or the like) sitting between client-devices and the NextDNS client in order to reduce lookup latency for common queries.

Additionally, if this feature was implemented, an added bonus would be if the NextDNS client could prefetch common queries like unbound does so well. That would further benefit lower latency for client-devices. I recognize this would interfere with the "-report-client-info" option as the requestor for prefetched records would be the NextDNS client itself. To some, this might be an acceptable tradeoff if latency is a concern.

[rs]

The caching part is planned. Prefetching might be tricky to do right, and I doubt an unbound-like prefetch would benefit local LAN without enough clients to amortize it. A knot-resolver type of prefetch with prediction could be interesting, but far more complex.

We'll start with simple caching :)

[rs]

Fixed by 0e2dc7a. Please play the new cache-size configuration option.

[lunim]

How long are results cached for?

If the whitelist/blacklist is updated is the cache cleared? If not, it might be handy to have the client invalidate the cache if a configuration change is made.

[rs]

It is an AST cache, it will keep the most popular entries until the size of the cache is reached, replacing least popular entries by new ones then. When a query is performed, if the minimum TTL of the cached entry is still valid, the entry is used, otherwise, a request is performed to the upstream. If the upstream can't be reached, the stale cache entry is used instead of returning a SERVFAIL.

Blocked entries have a TTL of 5 min set by NextDNS, they will then be invalidated after 5 min. Note that your OS is also performing the same kind of caching, so clearing one cache and not the other won't have much effect.

[FernandoMiguel]

@rs -cache-size is in byte size, not TTL?
I would prefer to say to cache 30 seconds , for example

[rs]

Please read my message above.

It does not make sense to force the eviction of a cached entry if there is still space available in the cache. Those entries can be useful in case of upstream reachability issue. The AST eviction algorithm will make sure that least popular entries give up space for most popular ones.

[gam-phon]

Great work. This is great, thank you so much for providing this feature.

I have a question since cache will depends on original TTL for unblocked by NEXTDNS dashboard. If there is a website that has TTL for 1 year for example and already cached in the client of NEXTDNS then later from NEXTDNS dashboard I have blocked it by category or specifically. I think it will not be blocked until it got purged from NEXTDNS client, right?

[Fail-Safe]

@rs I am thrilled to see the progress on this! Many thanks!

I did log two new issues based on testing I have been doing. If you would prefer I log issues in this thread instead of opening new issues, let me know and I will move the details over to this thread.

[rs]

@gam-phon that is correct. We can provide a max-ttl option to cap the TTL of entries to workaround that.

[rs]

@Fail-Safe new issues are better.

[lunim]

Blocked entries have a TTL of 5 min set by NextDNS, they will then be invalidated after 5 min. Note that your OS is also performing the same kind of caching, so clearing one cache and not the other won't have much effect.

Thanks for the update. I've noticed that devices on my network respond much faster than 5 minutes when unblocking a site. Does restarting the NextDNS client flush the cache?

[rs]

Our clients flushes the DNS cache of the host yes. It's actually a side effect, but a convenient one.

[jwelter1971]

Any way to get statistics of how the cache is performing?

[rs]

Yes, can you open a ticket?

[jwelter1971]

Done, see #154