Servicing stale cached responses and immediate purging

[krizhanovsky]

Depends on #515

Servicing stale responses from the cache

Implement cache_use_stale option, which doubles Nginx's proxy_cache_use_stale. 3rd case in #506 mentions the stale responses flags, so the issues are dependent. imeout, error and HTTP status codes must be implemented. The new configuration option synthax:

cache_use stale 1*(error | timeout | status_code)

, where status_code is numerical HTTP status codes, error implies any 50x 0r 40x error codes. If a status code specified, then error isn't allowed. Examples:

cache_use stale error timeout 304;

cache_use stale 404 502 500;

stale-while-revalidate and stale-if-error headers from RFC 5861 also must be supported.

Implementation

The caching workflow must be adjusted:

1. Lookup the cache for the response
2. If found a good response, then just forward it.
3. If a stale response is found, then link it with the request (probably just use req->resp) and mark (probably as a flag in the response or request) that the response is stale. (The cache entry must be locked or reference counted, see TDBv0.2: Cache background revalidation and eviction #515)
4. If a response found in the cache is stale, then go to the logic as if there wasn't cache hit - forward the request to a backend
5. If the backend replies with an error code specified in cache_use_stale or timeout occured, then do not link the error response with req->resp, but send the stale cache response to a client
6. If backend replies with an acceptable response, then unlock/unreference the cache entry and forward the response to a client

We should keep not more than 1, the latest one, maybe stale though, response in the cache. All older stale versions of the response must be purged. So with adding a cache entry wich already has stale version, we need to remove the stale version. The stale version can be removed directly by just linkedreq->resp and we don't need to traverse the TDB index.

Immediate purging

#501 required two strategies for cache purging: immediate and invalidate. While we're actually always service stale responses due to #788, we store all the stale responses in the cache. So the immediate strategy must be implemented when TDB removal is done in #515.

Testing & documentation

Please update https://github.com/tempesta-tech/tempesta/wiki/Caching-Responses and develop appropriate functional tests or create an issue.

[krizhanovsky]

While #515 is blocked by the full index rework in #1869, we still can implement an elementary TDB removal and finish this task.

Need to implement int tdb_htrie_remove(TdbHdr *dbh, uint64_t key, bool (*eq_cb)(void *), void *data) for HTrie, where eq_cb and data is a callback, which can compare stored record against a required one (i.e. URL + host) since HTrie is indexed by a hash value, which is prune for collisions. The function must be called for cache_purge immediate.

Once we add a new cache entry to HTrie, we should scan the target bucket and call tdb_htrie_remove() for all the invalid (outdated) entries (i.e. implement a basic cache eviction) and an entry matching the new one (replace an old cache entry with a new one).

We should never remove index nodes (the same approach is used in the new HTrie version). Records removing can and should be done under the bucket lock. If a bucket becomes empty, then do not reclaim it. Remove only the data (variable length data, which is web cache entries are).

I think we can use reference counters (either in TfwCacheEntry or TdbVRec for data reclamation in tdb_rec_put(): once a reference to a record from a bucket is removed (refcounter is decreased) and there are no more readers and updaters, then we free the memory. The reclaimed memory (data blocks) should be added into a per-cpu free blocks list, which must be firstly checked on new data insertion.