feat: O(1) selector tables

[charles-cooper]

What I did

implement O(1) jumptables

How I did it

two methods, hash table with probing and perfect hashing using a two-level technique.

the first method divides the selectors into buckets, uses method_id % n_buckets as a "guess" to where to enter the selector table and then jumps there and performs the familiar linear search for the selector ("probing"). to avoid too large buckets, the jumptable generator searches a range of n_buckets; the average worst case for 80-100 methods is 3 items per bucket and the worst worst case is 4 items per bucket (presumably if you get really unlucky), see _bench_sparse() in vyper/codegen/jumptable_utils.py. the average bucket size is 1.6 methods.

the second method uses a perfect hashing technique. finding a single magic which produces a perfect hash is infeasible for large N (exponential, and seems to run off a cliff around 10 methods). to "get around" this, the methods are divided into buckets of roughly size 10, and a magic is computed per bucket. several n_buckets are tried, trying to minimize n_buckets. the code size overhead of each bucket is roughly 5 bytes per bucket, which works out to ~20% per method, see _bench_dense() in vyper/codegen/jumptable_utils.py. then, the function selector is looked up in two steps - it loads the magic for the bucket given by method_id % n_buckets, and then uses the magic to compute the location of the function selector (and associated metadata) in the data section. from there it loads the function metadata, performs the calldatasize, callvalue and method id checks and jumps into the function.

there is a gas vs code size tradeoff, as can be seen in the following table:

notably, the sparse (gas optimizing) version clocks in at 69 gas in the best case (~109 gas in the "average" case), while the dense version clocks in at ~8 bytes per method.

some additions needed to be made to the assembler to handle pure data blocks.

How to verify it

all existing tests pass, also see new selector table tests in tests/parser/test_selector_table.py which should test different configurations of selector tables somewhat exhaustively

Commit message

this commit replaces the existing linear entry point search with an O(1)
implementation. there are two methods depending on whether optimizing
for code size or gas, hash table with probing and perfect hashing using
a two-level technique.

the first method divides the selectors into buckets, uses
`method_id % n_buckets` as a "guess" to where to enter the selector
table and then jumps there and performs the familiar linear search for
the selector ("probing"). to avoid too large buckets, the jumptable
generator searches a range from ~`n_buckets * 0.85` to
`n_buckets * 1.15` to minimize worst-case probe depth; the average worst
case for 80-100 methods is 3 items per bucket and the worst worst case
is 4 items per bucket (presumably if you get really unlucky), see
`_bench_sparse()` in `vyper/codegen/jumptable_utils.py`. the average
bucket size is 1.6 methods.

the second method uses a perfect hashing technique. finding a single
magic which produces a perfect hash is infeasible for large `N`
(exponential, and in practice seems to run off a cliff around 10
 methods). to "get around" this, the methods are divided into buckets of
roughly size 10, and a magic is computed per bucket. several `n_buckets`
are tried, trying to minimize `n_buckets`. the code size overhead of
each bucket is roughly 5 bytes per bucket, which works out to ~20% per
method, see `_bench_dense()` in `vyper/codegen/jumptable_utils.py`.
then, the function selector is looked up in two steps - it loads the
magic for the bucket given by `method_id % n_buckets`, and then uses the
magic to compute the location of the function selector (and associated
metadata) in the data section. from there it loads the function
metadata, performs the calldatasize, callvalue and method id checks and
jumps into the function.

there is a gas vs code size tradeoff between the two methods - roughly
speaking, the sparse method requires ~69 gas in the best case (~109 gas
in the "average" case) and 12-22 bytes of code per method, while the
dense method requires ~212 gas across the board, and ~8 bytes of code
per method.

to accomplish this implementation-wise, the jumptable info is generated
in a new helper module, `vyper/codegen/jumptable_utils.py`. some
refactoring had to be additionally done to pull the calldatasize,
callvalue and method id checks from external function generation out
into a new selector section construction step in
`vyper/codegen/module.py`.

additionally, a new IR "data" directive was added, and an associated
assembly directive. the data segments in assembly are moved to the end
of the bytecode to ensure that data bytes which happen to look like
`PUSH` instructions do not mangle valid bytecode which comes after the
data section.

Description for the changelog

Cute Animal Picture

[pcaversaccio]

that's right, we have to do linear search after jumping to the guessed position. but unlike hashmaps constructed at runtime, we have some control over worst-case probe depth because we can search for a good hash function to minimize probe depth - benchmarking shows that typical worst case is 3 items per bucket (and average 1.6 per bucket), so we have a (statistical) bound on the probing depth. so it's something like O(1-3), which is still O(1).

(probably if you were to benchmark with larger and larger selector tables, i'd guess the worst case bucket grows at some rate like log_10(N), but it's pretty much indistinguishable from O(1) since we are only typically dealing with selector tables up to ~100 items in practice.)

oh yeah, right, thanks for the clarifications!

[pcaversaccio]

As per offline discussion, it might make sense to test for non-deterministic behaviour of the selector table. See a similar (not equivalent iiuc) bug in Solidity version 0.8.21:

Vyper does already check the call graph stability (see here: #3370)

[charles-cooper]

As per offline discussion, it might make sense to test for non-deterministic behaviour of the selector table.

yea this makes sense, although it's not super clear to me the best way to test this, maybe it would be good for this to be addressed in another PR as this PR is already getting quite large

[pcaversaccio]

yea this makes sense, although it's not super clear to me the best way to test this, maybe it would be good for this to be addressed in another PR as this PR is already getting quite large

I agree, but this should be addressed before the 0.3.10 release. I opened an issue #3530 to track it.

[pcaversaccio]

Can't approve via GitHub, so I approve via comment :) LGTM

[antazoey]

@charles-cooper Is it possible for len(bytecode_sequence) to be < push_len here and if that is the case, would that cause isses?

[charles-cooper]

yes, but it shouldn't cause issues, this is just a way of handling disassembly of the data section