Enable Rails 7.1 Marshalling format

[byroot]

This replace the workaround added in #24142 and also fixes #27622 so #28521 can be reverted.

That new format isn't backward compatible, so normally you are supposed to deploy Rails 7.1 without that new format enabled first, but this can be sidestepped by invalidating old cache entries, which this PR does by changing the cache prefix.

cc @ClearlyClaire @Gargron

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (43d800a) 84.78% compared to head (f1394ed) 84.75%.
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #28609      +/-   ##
==========================================
- Coverage   84.78%   84.75%   -0.03%     
==========================================
  Files        1039     1039              
  Lines       28259    28175      -84     
  Branches     4560     4548      -12     
==========================================
- Hits        23960    23881      -79     
+ Misses       3139     3136       -3     
+ Partials     1160     1158       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ClearlyClaire]

Looks fine to me. One downside is that it'll leave old keys in the cache, but most should only persist for 10 minutes, some persist for up to 1 day, and a handful for up to a week.

I also wonder if we shouldn't take the opportunity to use MessagePack?

[byroot]

One downside is that it'll leave old keys in the cache, but most should only persist for 10 minutes, some persist for up to 1 day, and a handful for up to a week.

I guess it depends how the Redis is configured. By default the maxmemory-policy is volatile-lru so these unreachable keys will simply be evicted.

[byroot]

As mentioned, I added the workaround back, so this can ship. I'll see about cleaning up the settings gem if I find the time.

[ClearlyClaire]

As mentioned, I added the workaround back, so this can ship. I'll see about cleaning up the settings gem if I find the time.

Looks good to me. I think I prefer to have the config set directly in config/application.rb to make sure there is no cache format change at runtime, but I can do that in a separate PR.

Likewise, I think it would be a good thing to find and fix the places where ActiveRecord is initialized early, but that can be done separately, and you've given us plenty of information about how to find those cases! Thanks again

[byroot]

I think I prefer to have the config set directly in config/application.rb to make sure there is no cache format change at runtime

I've amended my commit to do that.

[renchap]

Looks good to me. Yay for removing ugly hacks.

This will cause empty caches in many places when deploying this commit, but I think we did it before and did not see really bad performance following it.

I also wonder if we shouldn't take the opportunity to use MessagePack?

This is probably a good idea, probably using the paquito gem? But this should probably be in another PR, as I expect we will have some additional work to do to ensure all our shapes are serialisable.