Automatically expiring items based on lifetime

[poisa]

I'm researching cache libraries for an upcoming project and think that Carlos would be a good fit. There are two things that I can't figure out how to do with it yet and hopefully you can point me in the right direction.

1st requirement: I need a disk >>> network cache that should always fetch from the network if the item has already been sitting in the cache a set amount of time. Each item should have a different expiration timeout. For example, some JSON files can sit in the cache for 1 hour whereas images can sit for 1 month. Feels like I could achieve this with a custom disk level but I wouldn't like to reinvent the wheel as I'm sure the disk level that ships with Carlos will be much sturdier (the amount of unit tests in Carlos is positively biasing my decision!).

2nd requirement: I need an "offline" mode so that when I set it, cache items will only be fetched from the first level (disk) and will never fallback to the last level (network). This feels like it can be achieved by setting the network cache to be "conditional". Is that right? I'm not sure I fully understand the docs for conditioning caches.

Many thanks in advance for any help you can give me!

[vittoriom]

Hi @poisa, and thanks for your interest in Carlos!

let's see if I can answer your questions:

1st requirement: I'm afraid we don't offer a time-based expiration of items yet. This is a big limitation of Carlos at the moment that should be addressed with #89. I already had some ideas in mind how to design the feature, but didn't have time unfortunately to work on it. So short answer, yes you would need to build something on top on DiskCacheLevel (that you can reuse anyway), that keeps track of the time each item was fetched. Not a blocker, but some amount of work on your part.
I would do it like this:

• Instantiate a NetworkFetcher
• Instantiate a DiskCacheLevel
• Transform the disk cache with a condition. In the condition closure you can check for the type of the object through the key (for example the last part of the URL or some similar logic to differ between images and JSON), and once you determine the type of the object, you could even inspect the file on disk (by leveraging the knowledge of how DiskCacheLevel stores files based on the hash of the key) to check its last modified timestamp. You can apply your custom expiration rules here and return false if needed, so that the whole disk level will fail
• Build a composition of this newly created disk level with the network fetcher created at step 1

This will behave as described, although you would need to create the transformer/condition at step 3 manually.

2nd requirement: As you said, you can condition the network fetcher with a check for reachability. The problem is that this, together with the solution for the 1st requirement, would cause a cache miss if the item is older than your expiration time and the user is offline. To avoid this, you could directly apply the reachability check to the condition mentioned above, so that you don't check for the expiration time of an item if the user is offline (since we already know we are not going to fetch from the network anyway).

Another more convoluted option would be to switch the cache created for the 1st requirement, and the original disk level created at step 2 of the solution for the 1st requirement, and choose between the two depending on the reachability status. This way, if you are offline you directly route the request to the disk level only; if you are online, you route the request to the composed cache obtained as the solution of the 1st requirement. This is "more complex" but slightly better IMHO because you separate the concerns of the reachability rule and the expiration rule.

This would look like this:

The code would look something like this:

let oneHour = 60*60
let oneMonth = 30*24*oneHour
let disk = DiskCacheLevel<NSURL, NSData>()
let network = NetworkFetcher()
let conditionedDisk = disk.conditioned { URL in
  if isImage(URL) {
    return Future(!isOlderThan(URL, oneMonth))
  } else if isJSON(URL) {
    return Future(!isOlderThan(URL, oneHour))
  } else {
    return Future(true)
  }
}
let onlineCache = conditionedDisk >>> network
let onlineCheck = switchLevels(onlineCache, cacheB: disk, switchClosure: { URL in
  if isOnline() {
    return .CacheA
  } else {
    return .CacheB
  }
})

I hope this answers your questions, and you will be able to make an informed decision about whether to use Carlos or not!

[vittoriom]

I updated the answer since I first wrote it with more information, a diagram, and some pseudo code.

[poisa]

Wow, thanks for the prompt and detailed answer! With the info you provided I think I will be able to use Carlos in my upcoming project.
In my ideal cache interface I would love to do something along the lines of:

cache.set("test String", forKey: "key", withTimeout: 60)
// or
cache.setTimeout(60)

In my case not all JSON files will have the same expiration timeout but I think for the time being I can come up with some class that given a URL will return the appropriate timeout.

For the second requirement I like the second option better (switching levels) since, like you hinted, each cache will do less and the intent will be clearer.

I'll be working with this in the next few weeks (don't be surprised it I pop up around here with another question!).

Thanks!

[vittoriom]

No problem if you come up with more questions! I'll be happy to answer them.
I'm glad the answer clarified some things a bit more.

Regarding the ideal interface, I thought about the global timeout (cache.setTimeout(60)) but that's quite complex when you think that this should work with every level that we ship out of the box, plus every custom level the user might build, plus any composition of the above.
Maybe the first option could be easier (cache.set("test string", forKey: "key", withTimeout: 60)). I still have to think about it some more time and when the "Eureka" will come, I'll write a draft for it :)

Have a nice weekend!