Cache for custom upstream configurations

[Iceman248]

Issue Details

• Version of AdGuard Home server:
  v0.107.0-b.11, issue started on v0.106.3, still issue after upgrading to beta
• How did you install AdGuard Home:
  Docker using Unraid Community Apps
• How did you setup DNS configuration:
  DNS assigned via DHCP from router, AGH goes straight to NextDNS as upstream

Expected Behavior

many queries showing fractional ms times

Actual Behavior

Typical ping times for everything, nothing being cached, NextDNS showing many times over more queries as a result

Additional Information

Starting happened in the last 24 hours

[ameshkov]

It'd be better if you provided an example, so far I see no issues with the cache.

Here's how to check it:

1. Examine the query log and find queries that were not cached.
2. Find the previous queries (which should've been cached) and check the TTL in the response.
3. If the TTL has not been expired by the time of №1, something went wrong.

[Iceman248]

I just discovered that the issue is only happening with clients that I configured in the Client Settings. I only did so to identify the device at NextDNS, so I needed to add the custom DNS server, plus it cleans up the labeling in the query log, but for some reason the caching doesn't work on configured clients. I unconfigured one client and caching works again, but I'd much rather use it. I can't see this being an intentional feature.

[ameshkov]

Hmm, this may indeed be a bug. Have you configured a custom upstream for these clients?

[sachinwadhwa]

I have noticed similar issue regardless of custom upstream (I have both clients with and without custom upstreams) but caching only works for clients which are not in client settings. See example as "graph.facebook.com"

[Iceman248]

I do have a custom upstream, but it's the same one but just a different identifier in it for each of my clients, but as it was just pointed out, it happens regardless of whether a custom upstream is used or not.

[EugeneOne1]

@ameshkov, we actually have a piece of logic that restricts caching custom upstream results:

dnsproxy/proxy/proxy.go

Line 481 in 0f48436

cacheWorks := p.cache != nil && d.CustomUpstreamConfig == nil

This is the result of fixing AdguardTeam/AdGuardHome#1301. Should we leave it as is or implement per-client caching?

[ameshkov]

@EugeneOne1 needs to be extracted to a new issue in DNS proxy.

Indeed, we cannot simply use the global cache here, we should come up with a solution though.

[Iceman248]

Thanks. What is the reason that custom upstream clients can't use the global cache? Is that a permanent situation? What is the limitation with that?

[ameshkov]

Usually, custom upstreams do additional filtering. For instance, if a client is a Kids device, people may choose to use an upstream that blocks adult websites. Using a global cache would be inconsistent in this case.

[Iceman248]

Okay, I see. What about for one of the options in the client settings to choose whether or not that client uses the global cache?

[emlimap]

At the moment I run blocky in the middle between AGH & DOH/DoT upstreams for the sole purpose of caching, and would like to get rid of it when dnsproxy/AGH get caching support for custom up streams.

Also, blocky supports custom upstream on per client basis. Not sure how they implement caching for this as I use it with just default upstream. https://0xerr0r.github.io/blocky/configuration/#upstream-configuration

Source code for blocky https://github.com/0xERR0R/blocky

[Iceman248]

I'm confused after reading the other linked threads. Was this actually resolved or not?

[Mizzick]

@Iceman248, I have fixed this on the side of the dnsproxy. If you want to test this client's cache urgently with AdGuard Home, please have a look on the latest edge build version "v0.108.0-a.764+c8f1112d". Nonetheless it yet doesn't have an UI to set the settings, but we have added new properties in YAML config clients.persistent.upstreams_cache_enabled and clients.persistent.upstreams_cache_size which describe cache configuration for each client custom upstream configuration.