Solid Cache should be the default caching backend for Rails 8

[dhh]

Like with Solid Queue, Solid Cache gives us a database-agnostic backend for Rails.cache that works well as an out-of-the-box default in production – without any configuration needed or dependencies (like Redis) required.

The tables should be setup out of the box with "rails new", but you should be able to avoid this using --skip-solid-cache or just --skip-solid.

Work outstanding:

• Add size-based trimming to prevent SC from filling up tiny DBs.
• Release Solid Cache 1.0

cc @djmb

[frederikspang]

Could this be a switch like —-cache=memory,solid,redis like we have for --database and —-javascript - or are we going for a “SolidCache or setup yourself” kinda direction?

[inkyvoxel]

Would —skip-solid skip all the ‘Solid’ features e.g. Cache and Queues? (I’m sure there’s a Snake hiding somewhere here too)

[byroot]

As mentioned in person in Amsterdam, I don't think SolidCache is a good default because:

• It's not LRU but FIFO, so it needs a lot of space to thrive
• It needs a specialized configured backing DB to perform well.

Given that a very large part of users (especially people trying Rails for the very first time) are using a managed database with fairly low limits on the number of rows and total storage, it would be a big footgun.

Even for users going the rented server + Kamal route, hosting a memcached with default settings will likely perform better and be less work than setting up a second MySQL or Postgres server with a tweaked config.

And even for users going with SQLite, it means they are on a single server, so FileStore should be adequate.

I agree it's annoying we can't have a cache store setup by default, but I don't think SolidCache is the solution.

If anything I'd be more in favor of enabling the FileStore as a default (even though it makes cache.delete inconsistent if you have multiple servers)

[dhh]

Here are the problems I'd like to solve:

• The cache store should be auto-limit out of the box. A default file store cache will fill the disk until it's out of space. Solid Cache can be set with a conservative limit, like, say 1 week cache for starters.
• The cache store should be multi-machine by default. Adding two dynos for the same app should share the same cache. That also rules out the file store, but would allow something like Redis.
• The cache store should not require any additional moving parts than what's included by Rails by default. That makes Redis a tough fit.

But let's validate whether these design goals are compatible with the performance of envelopes of common, low-end VMs.

I'd be surprised if you run into DB-related issues, given a low default Solid Cache limit, before you'd run out of other resources at the low end. If anything, the reverse may well be true. That more effective and longer lived caches makes your app perform better, even if it does lean on a constrained DB.

But this would be good to test! So let's open this issue to people who'd like to help us discern those factors. Try to setup Solid Cache on small VMs, run a bunch of benchmarks against an app that uses caching, and let's see where things might fall over.

Appreciate the concerns you raise, @byroot! It's absolutely possible to add bad caching to the mix and make things worse. So we need to avoid that.

[byroot]

• A default file store cache will fill the disk until it's out of space.

That's a good point It's something that we could try to improve though. Memsize would be complicated, but we could cap the number of entries, and LRU. Wouldn't be exactly ideal but doable.

The cache store should be multi-machine by default. Adding two dynos for the same app should share the same cache. That also rules out the file store

Yes and no. It only rule it out of you delete or overwrite existing keys, which IMO isn't great design since most caches are eventually consistent, but that's a much longer debate 😄 .

There is also the question of whether once you have multiple machine you aren't already past the point where it's OK to setup a dedicated cache service. I agree that Redis being a swiss army knife it requires a careful config, but Memcached is beyond trivial to setup.

So that's where I don't quite follow the direction. As I believe we both agree, defaults should try to optimize for the most common use case, people who know what they are doing shouldn't be afraid to change the defaults.

So I'm trying to put myself in the shoes of someone starting a new Rails app, that's not gonna handle a huge load right away.

In my mind they are either starting with a PaaS (e.g. Heroku, Fly, Render, etc):

• Databases there have a limited number of row and disk space.
• They can't be tuned for SolidCache
• It's very expensive to go on a higher tier.
• That's where they are likely to have multiple dynos relatively early, because of how weak each dyno is.

Or they are starting with some cheap VPS or bare metal (e.g. capistrano or Kamal), in which case:

• They will likely stick with a single machine for a long time, making a local store acceptable.
• Even when they'll move to multiple machines, Memcached will be much easier to setup, monitor etc than one extra MySQL or PostgreSQL database.

That's why I don't see SolidCache fitting the bill for being the default. It's a bit too situational, and can turn into a footgun if it fills the limited database.

[dhh]

What are those limits? On say Heroku? If you're running a tiny app on a tiny dyno, you'll presumably also have few users, and thus not much data to cache? So I think these things go together, but let's explore.

I think the current situation is not good. There's no default, persistent cache that won't fill up your disk. That's a problem we should fix. Solid Cache fixes that, but maybe in a way that demands too much of tiny DBs?

On small systems, though, I reckon you're more likely to be constrained with memory (Redis) and CPU (no cache) before disk (DB). Will explore some testing to validate this hypothesis.

[byroot]

What are those limits? On say Heroku?

From: https://elements.heroku.com/addons/heroku-postgresql

Name	Row Limit	Size Limit	Price
Mini	10k	1G	$5
Basic	10M	10G	$9
Standard 0	None	64GB	$50

I think the current situation is not good. [...] That's a problem we should fix.

I totally agree with the premise, I just don't see a solution 😢.

[dhh]

Okay. If you're on that tiny tier, you may well not want to use that space for caching. But you could also just not. By default, Rails doesn't actually cache anything.

I don't want to design primarily for such a poor setup. 1GB/10K seems like limits that probably made sense for Heroku in 2009, and then just weren't ever updated since.

A $7 DO Droplet has 25GB of storage, which could be used by a database.

We are never going to satisfy all the constraints. But a default setup that's multi-machine, uses disks over RAM, and supports both auto-trimming and encryption out of the box seems superior to what we have now.

Then we can document what to do if you still want a large cache but live under the severe constraints of the smallest possible cloud VMs. Either way, even at 10K/1GB, you're going to be fine for a long time.

[dhh]

But would love if anyone is ALSO interested in improving the filestore with trimming controls and encryption. Would be nice to have great options for both files, DB, and Redis/memcached.

[byroot]

I don't want to design primarily for such a poor setup. [...] even at 10K/1GB, you're going to be fine for a long time.

Note that my apprehension isn't so much about being suitable for such small setups, but more about failure mode. My big worry is someone create a new Rails app, deploy it to one of these platforms (there are tons of tutorials for that), it works fine for a while until suddenly the DB is full and everything falls apart.

If it was only taking down the cache I wouldn't mind, but here it would also take down the app primary features.

It's really about "unknown unknowns", when a user opt-in to a feature, we can consider they are responsible for making sure it will work for their setup. When it's the default it's more our responsibility to make sure it won't bite them.

improving the filestore [...] encryption

I have a big refactoring of Active Support Cache in the back of my mind for a couple years now, to solve a few perf problems but also make this sort of stuff easier. Not sure if / when I'll get around to work on it though.

[dhh]

Yeah filling up the DB with caching entries is a no go. Let's make this contingent on having a space-based limit in Solid Cache rather than just the current time-based limit. Then we can ensure that we only start the original setting at 100-250mb, leaving lots of room for data on a 1GB-capped DB. We will get that sorted before proceeding 👌

[dhh]

But I would ALSO love to see a better file store with both auto-trimming and encryption.

[igorkasyanchuk]

I wish I could have solid_cache, file_store_cache, and redis_cache in one app.

e.g.

Rails.cache_storage(:redis).cache do ... end 
Rails.cache_storage(:memcached).cache do ... end 
# or
config.controller.cache_storage = :file