Implement read decryption offloading

[mkeeter]

Analogous to #1066 , this PR moves read decryption into the rayon thread pool. It uses exactly the same infrastructure (particularly DeferredQueue), and is implemented in a very similar way.

I need to do some benchmarking before this goes in, but wanted to open it for review early.

[mkeeter]

I did some benchmarking on c12, using the following fio script:

[global]
filename=/dev/nvme0n1
iodepth=25
ioengine=aio
time_based
runtime=60
numjobs=1
direct=1
stonewall=1

[randwrite-4M]
bs=4M
rw=randwrite

[randread-4K]
bs=4K
rw=randread

[randread-1M]
bs=1M
rw=randread

[randread-4M]
bs=4M
rw=randread

(the 4M writes are to put some data on the virtual disk, which is otherwise blank)

I see the following performance (in MiB/s)

Read size	main	offload-read-decryption
4K	50.5	42.1
1M	164	216
4M	156	218

It looks like the overhead of moving decryption off-thread isn't worth it for small reads, which isn't totally surprising.

Doing a more detailed sweep, I see this:

The crossover point looks like about 8 KiB, so I'm going to tweak the code to decrypt immediately below that size.

[mkeeter]

Running with the change above (8f3ba46) to only defer > 8 KiB read decryption, I'm only seeing a small improvement (42.7 -> 44.2 MiB/s). This is still below the performance on main (50.6 MiB/s), so I think more investigation is needed.

[mkeeter]

Re-running on Crucible main (099c201), I see 44.5 MiB/s, so I suspect the 50.6 MiB/s reading was a fluke. This means that there's no performance regression, so we can move forward with the PR.

[faithanalog]

I am not getting the exact same numbers you are on my test setup, but I am getting a similar ratio of improvement. about 40-45% better for 16k reads, 75-80% better for 4M reads.