How to use FusionCache with prefetching?

[hvgNET]

(This is my first time using GitHub, so if I'm doing something wrong, please don't hesitate to tell me)

Hello,

I am considering using FusionCache for a project and I am currently testing it out.
I have a scenario that I don't know how to solve with my current knowledge about FusionCache, hence this question.

The scenario is that I want to prefill the cache with data, but then let the GetOrSet factory stuff handle cache misses, timeouts, cache refreshes and all that good stuff after the cache have been filled.
I want to prefill the cache from a database and the important detail about this scenario is that i only want to call the database once, when prefilling.

Example:

I have 10.000.000 key value pairs in a db, and i know the ids upfront.
I have a method that takes a single id called GetValue(int id), which accesses the db.

I could prefill the cache by doing GetOrSet({id}, _ => GetValue(id)) for each of the ids.
But to my understanding this would go to the db 10M times, which is the reason I want to go to the db once.

So conceptually I would like to do something like

• Call a method called GetAllValues()
• Use Set to set them individually for each of them
• Call GetOrSet({id}, _ => GetValue(id)) for each of them

But that just seems... a bit hacky to me as i neither want to get nor set the value, i just want to set the factory up

I hope it makes sense, else I am happy to provide further information or input :)

[jodydonetti]

Hi @hvgNET and thanks for considering FusionCache.

Sorry for the delay in answering but covid finally got me.

In general your idea seems the right one to me:

• load all 10.000.000 items from the db in one go
• save them all in the cache individually with a Set call for each of them (with a reasonable cache duration)

Then, when you need one of them during the lifetime of your application, call GetOrSet({id}, _ => GetValue(id)) so that if the value is still in the cache you'll get it back immediately and, if not in the cache anymore, it will be loaded from the db and saved in the cache.

There's this passage that I don't understand though:

But that just seems... a bit hacky to me as i neither want to get nor set the value, i just want to set the factory up

Why do you say it feels hacky? Just to be clear, you don't need to call GetOrSet for each item when you are prefetching the entire thing at the beginning, you only need to call GetOrSet later on when you need some of those values and, in that moment, the GetOrSet means "get me X from the cache and, if it is not there, load it and set it".

Maybe you are thinking about calling GetOrSet for each key in the initial prefetching phase? If that is the case, that is not necessary.

So, to recap:

• INITIAL PREFETCHING: load all items from db -> for each item call Set with a reasonable duration
• DURING YOUR NORMAL APP USAGE: call GetOrSet so that if the value is expired from the cache it will be loaded again

Doing it this way will allow you to have the cache already filled at start + automatically keep some of those items in the cache, based on actual usage.

Let me know if I got this right and if I can help you more.

Hope this helps.

[jodydonetti]

ps: if you plan on also using a 2nd layer (distributed cache) keep in mind that the initial prefetching may not be necessary, since the data may already be in the distributed cache because of previous runs.

[jodydonetti]

I'm converting this into a discussion.