Memory footprint of DNS cache & prefetch - How much RAM for e.g. 10,000 entries?

[gspannu]

1. As a rough guide, what is the memory footprint of storing DNS records in cache?
2. Is the memory required for prefetch (prefetchMaxItemsCount) different to the main cache (maxItemsCount)?
   Or is it shared or is double the memory required?

So, if my settings were like below (as a rough guide) - what would be my memory usage? Approx...

caching:
  maxItemsCount: 10000
  prefetching: true
  prefetchMaxItemsCount: 10000

caching:
  maxItemsCount: 1000
  prefetching: true
  prefetchMaxItemsCount: 1000

Just looking to get some basic maths for estimating my deployment.

[t-e-s-tweb]

Depends on Allow and Deny lists too. Running it just for DNS without any lists is only 13MB.

[gspannu]

I presume you mean it is 13 MB for a cache of 10,000 entries?

Allow/Deny lists, other bits will consume additional memory.

[t-e-s-tweb]

No, its when you start it. it caches as the queries are made. i have set it up for 10000 and and 1000 prefech. Max memory I have seen is 450MB with multiple allow deny lists.

However blocky still uses very less ram compared to other similar software.

[gspannu]

Thanks for the clarification.
What I was looking for is the memory usage (specifically used for caching) of say 10,000 items?

@0xERR0R : What would be the approximate memory usage as per the code? Is the cache being stored as plain text or is it being compressed in RAM, etc...?

[kwitsch]

"plain text" is really misleading regarding DNS messages. 😅

It is stored as struct without the OPT section.
For prefetching purposes there are some additional values per cache entry and some additional overhead from the beginning (prefetching logic doesn't increase its footprint for additional cache entries).

The entries aren't compressed since the initial overhead is low and it would increase response times.

But then again your question is hard to answer since DNS messages differ in size and you don't specify what messages are stored(1 to 1 or 1 to n mapping, additional section,...).

If you want to calculate a really rough estimate use: MaxUdpSize * 2 * CacheEntryCount

[gspannu]

Thanks for the detailed explanation...

a really rough estimate use: MaxUdpSize * 2 * CacheEntryCount

👍

[kwitsch]

If you are really concerned about your memory footprint you should consider refraining from DOH and use DOT or UDP/TCP instead since this reduces the per request overhead which matters during the prefetch process.

Additionally setting a min TTL for cache entries could reduce the load and memory spikes since it reduces cache roundtrips.