YJIT: implement cache for recently encoded/decoded contexts

[maximecb]

The decoded_from and one-entry caches seemed slightly hacky, particularly given that decoded_from broke equality comparison on Context objects.

Since we know from Kokubun's previous work that there is significant duplication among contexts, I thought it could make sense to implement a fixed-size cache of recently encoded/decoded contexts. This turns out to work quite well.

master branch (with var-len ctx and single-entry cache):

yjit_alloc_size:          19,953,035
context_data_bytes:          649,671
num_contexts_encoded:        134,390
bytes_per_context:              4.83

After, cache size 128:

yjit_alloc_size:          19,585,604
context_data_bytes:          294,313
num_contexts_encoded:        134,268
bytes_per_context:              2.19

After, cache size 256:

yjit_alloc_size:          19,546,263
context_data_bytes:          264,494
num_contexts_encoded:        134,361
bytes_per_context:              1.97

After, cache size 512:

yjit_alloc_size:          19,550,879 # ~2% memory savings vs master
context_data_bytes:          239,697
num_contexts_encoded:        134,445
bytes_per_context:              1.78

So this is pretty cool. With minimal changes we can get some amount of deduplication of contexts. If we keep making the cache bigger, at some point, memory usage starts going up again because of the size of the cache itself. This seems to happen at cache size 512 for lobsters. That being said, I'd be inclined to keep a size of 512 because the bigger the app, the less the cache overhead matters.

[k0kubun]

nice!