Autocache using redis

[gabestein]

Motivation

So we don't rely on file system based storage that slows down or crashes the app for large queries.

Requirements

• A ValKey service is added to our dev, self-host, and production environments.
• Next is configured to store data cached with unstable_cache in ValKey.
• A maximum entry size is set to prevent extremely large queries from being cached.

Acceptance Criteria

• Include the output of mask tf plan in the PR, and run mask tf apply only after the PR is approved.
• Justify the maximum entry size with some napkin math.

Extra work

• Instrument ValKey to report cache hits/misses somewhere. Neither Honeycomb, nor Sentry support ValKey as a data source. So this might be tricky.

[tefkah]

See example here: https://github.com/vercel/next.js/blob/canary/examples/cache-handler-redis/README.md

One requirement id add is to allow us to set some maximum cache size, both for individual entries and for the total size of the cache stored. If the total size of the to-be-cached data > MAX_ENTRY_SIZE, it should not be stored. i think redis itself can probably handle the evicting of cached data if a certain limit is reached.

One suggestion id make is that we use ValKey rather than straight redis. I have a bit more faith in it continuing to exist in its current form than redis, and switching the backend later would be a hassle. but on the other hand redis is much more standardized ofc.

[tefkah]

Also, and this might be too much: it would be nice if we could trace the cache hit rate somewhere, likely honeycomb. Don't really know the status of our honeycomb setup, but while we do have a (kind of jank admittedly) way of logging out whether there was a hit or not, it would be nice to also keep track of how many times a certain cache was used before it was busted

[3mcd]

I updated the ticket to include ValKey as our cache db, but I want some feedback on whether or not the lack of instrumentation is a deal breaker for ValKey, since Redis seems supported by both Sentry and Honeycomb out of the box

[kalilsn]

I don't think the automatic instrumentation support should guide our decision here. We can manually capture the cache metrics we're interested in as open telemetry span attributes in the debug logging code in the memoize function. If we want to capture metrics from ValKey itself, the AWS cloudwatch metrics should be plenty for now. Eventually the valkey collector should let us collect those with opentelemetry too: open-telemetry/opentelemetry-collector-contrib#37005.

That said, I'd love to hear why not use redis - I know they changed their license but I don't really understand the implications.

Also, given the questions about honeycomb/sentry here, I want to point out that we are using sentry pretty heavily right now and honeycomb not at all. So maybe we don't need honeycomb? Sentry can ingest our opentelemetry spans and metrics so we wouldn't even have to change much.

[3mcd]

That all makes sense to me!

We should do a cross comparison to make sure we understand Honeycomb's intended role in our observability stack. I think Honeycomb was selected because it can take advantage of high-dimensional telemetry, e.g. able to construct views that consider many different dimensions (not just time-series). But I only partly know what I'm talking about here so we should regroup on this soon, maybe our next symposium meeting.

We'll stick with ValKey for this ticket. If the final implementation doesn't include metrics capture we can create a follow up ticket.