Fetch fill lock

[danmayer]

improve support for meta flags and fetch_with_lock

This starts to drive more usage of optional meta flags for specific features and cleans up the existing interface.

• This adds support for fetch with lock leveraging the meta protocol flags. Currently only for the singular case as the multi case will be a bit more complicated and I wanted to ensure this looks good before tackling it.
• The meta flags additional data is now sparse with only what is needed
• additional testing around meta flags has been added.

The benchmarks are for now in another repo where we are integrating and testing this branch...

This benchmark includes many competing threads which will hit the race condition and thundering herd around a fill lock. It shows this new method vs a more naive lock implemented outside of the dalli client. Outside of just benchmark performance it records the total count of the cache being refilled, the lock prevents many duplicate fills which can stress underlying systems like the database query. As the number of workers on a hot key gross the importance of the fill lock is more clear. The Dalli fetch with locking is this new method, it is the most efficient in terms of refilling, the only lower total fills was dalli_store_lock_fills but only because it was so slow it completed far less iterations.

❯❯❯$ BENCH_TIME=10 BENCH_TARGET=fetch_with_lock RUBY_YJIT_ENABLE=1 BENCH_CACHE_URL=127.0.0.1:11211 bundle exec ruby scripts/cache_bench
yjit: true
writing pairs to cache
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
    DalliStore fetch   887.000 i/100ms
DalliStore fetch with lock
                         1.182k i/100ms
         Dalli fetch     2.224k i/100ms
Dalli fetch with locking
                         2.094k i/100ms
Calculating -------------------------------------
    DalliStore fetch     12.542k (±55.6%) i/s   (79.73 μs/i) -     70.073k in  10.437755s
DalliStore fetch with lock
                         11.059k (±51.6%) i/s   (90.42 μs/i) -     74.466k in  10.228236s
         Dalli fetch     35.534k (±30.4%) i/s   (28.14 μs/i) -    251.312k in  10.236320s
Dalli fetch with locking
                         30.177k (±36.8%) i/s   (33.14 μs/i) -    188.460k in  10.019552s

Comparison:
         Dalli fetch:    35533.7 i/s
Dalli fetch with locking:    30176.5 i/s - same-ish: difference falls within error
    DalliStore fetch:    12542.1 i/s - 2.83x  slower
DalliStore fetch with lock:    11059.2 i/s - 3.21x  slower

dalli_fetch_fills: (bench: 19, threads: 226, total: 245)
dalli_fetch_lock_fills: (bench: 22, threads: 61, total: 83)
dalli_store_fills: (bench: 24, threads: 201, total: 225)
dalli_store_lock_fills: (bench: 18, threads: 40, total: 58)

This is a benchmark with a single thread just looping fetch while a key will expire and need to be refilled often. Note that the with lock is basically same-ish difference when there is no competition:

❯❯❯$ BENCH_TIME=10 BENCH_TARGET=fetch_with_lock_no_competition RUBY_YJIT_ENABLE=1 BENCH_CACHE_URL=127.0.0.1:11211 bundle exec ruby scripts/cache_bench
yjit: true
writing pairs to cache
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +YJIT +PRISM [arm64-darwin23]
Warming up --------------------------------------
    DalliStore fetch     1.988k i/100ms
DalliStore fetch with lock
                         2.314k i/100ms
         Dalli fetch     4.167k i/100ms
Dalli fetch with lock
                         3.725k i/100ms
Calculating -------------------------------------
    DalliStore fetch     16.986k (±29.5%) i/s   (58.87 μs/i) -    149.100k in  10.238837s
DalliStore fetch with lock
                         19.610k (±26.1%) i/s   (50.99 μs/i) -    178.178k in  10.074813s
         Dalli fetch     41.038k (± 5.3%) i/s   (24.37 μs/i) -    412.533k in  10.084243s
Dalli fetch with lock
                         35.613k (±11.5%) i/s   (28.08 μs/i) -    353.875k in  10.058282s

Comparison:
         Dalli fetch:    41037.8 i/s
Dalli fetch with lock:    35612.9 i/s - same-ish: difference falls within error
DalliStore fetch with lock:    19610.2 i/s - 2.09x  slower
    DalliStore fetch:    16985.9 i/s - 2.42x  slower

[danmayer]

github actions needed this update to run

[danmayer]

rubocop update required moving this outside of private

[danmayer]

at the moment bitflags is always present as it is a bit outside of a legacy flag... we could try to refactor it a away it is a bit awkward as a legacy flag, but there is meta flag option -f to request it so we could get rid of it, some of the dalli methods expect it so it might require a bit larger refactoring.

[nickamorim]

Do we need an early return here since there is a default lock TTL and interval?

[danmayer]

this just sets to an empty hash opposed to nil as it is annoying to keep checking for if the value is in the hash or if the options don't exist at all.

[nickamorim]

Not sure why we need the !val.nil? || val != '' check. Would you mind explaining plz?

[danmayer]

when you get a W or Z flag returned it will create a memcached entry with '' empty string as the value. So if you call get it will return with '' which isn't what you expect for a regular get, but when you passed the N flag you will expect the client to ignore and not use the '' value. Given re-ordering this set of if statements, it can now just be if there is a z response ignore the val.

[nickamorim]

Suggested change

	raise ArgumentError, 'lock_ttl must be a positive integer' if !lock_ttl.is_a?(Integer) && lock_ttl <= 1
	raise ArgumentError, 'lock_ttl must be a positive integer' if !lock_ttl.is_a?(Integer) && lock_ttl < 1

Since a lock TTL of 1s is valid