Write up LDF Cache Clearing Problem

[no-reply]

The ActiveTriples::LinkedDataFragments implementation doesn't provide a way to clear individual resources from the cache. This makes invalidation hard. It's not obvious how to handle this in the current implementation.

This ticket is to get a more complete write-up of the problems into an issue on the LDF repository and raise it for discussion in the Applied Linked Data WG.

[hackartisan]

I think this is related to the work Lynette has been doing to define / configure cache expiration behavior. See her branch of Linked Data Fragments https://github.com/ActiveTriples/linked-data-fragments/tree/cache_config/config

working stemming from this issue ActiveTriples/linked-data-fragments#31 and follow-up discussions on applied linked data group.

[no-reply]

Engaging with the discussion on ActiveTriples/linked-data-fragments#31 seems like the right move. I think I can write up a proposal based on what's discussed there already.

[no-reply]

Approaches for cache control:

• Use a marmotta-like configuration as started in: https://github.com/ActiveTriples/linked-data-fragments/tree/cache_config/config
  • How can we support this kind of interface for other backends?
  • Does marmotta's approach meet our requirements?
• Use HTTP Cache-Control headers.
  • We could probably support this in its entirety with: https://github.com/rtomayko/rack-cache
  • Is this too much client control?
• Create a static caching strategy that clients don't have control over.
  • Will this paint us into a corner?
  • Is there a one-size fits all caching strategy we can define?

In any of these three cases, questions that need to be answered include:

• What are the pre-warming needs?
  • Do search needs for an authority require a full pre-load of that authority?
  • Or can search be dependant on an external service?
• What happens when the cache "invalidates"?
  • do we clear data from the backend,
  • or simply require re-fetch on the next retrieval if the remote is available?
• What is an acceptable TTL?
• Does the client ever need to manually invalidate the cache?

I think that's the general shape of the problem.