feat: immutable data structure for memdb

[maan2003]

currently memdb clones all the database items on each begin_transaction, this is really slow for even slightly large database.

this uses immutable data structures for memdb, so clone in begin_transaction is O(1)

[dpc]

🔥

The merge-base changed after approval.

The merge-base changed after approval.

The merge-base changed after approval.

[maan2003]

rebased

[m1sterc001guy]

Is there another flaky test? Hard for me to tell if its due to these changes or not

[dpc]

00:04:44 ================= RESULTS ==================
00:04:44 REISSUE: min: 0.5s, avg: 0.7s, median: 0.6s, p90: 0.9s, max: 1.4s, sum: 19.9s
00:04:44 LN SEND: min: 1.3s, avg: 1.6s, median: 1.5s, p90: 2.2s, max: 2.3s, sum: 31.4s
00:04:44 LN RECV: min: 1.1s, avg: 1.6s, median: 1.6s, p90: 1.7s, max: 1.7s, sum: 31.9s
00:04:44 FM PAY: min: 1.4s, avg: 1.9s, median: 1.6s, p90: 1.7s, max: 6.7s, sum: 37.1s
00:04:44 RESTORE: 100.985712469s
00:04:44 thread 'main' panicked at /build/source/devimint/src/tests.rs:284:5:
00:04:44 assertion failed: fm_pay_stats.max.as_secs_f64() < fm_pay_stats.p90.as_secs_f64() * factor
00:04:44 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

@maan2003 That assertion ... looks :sus: . Seems to me that you might have improved (along possibly with the executor improvements) common case so much, that is now much faster than a maximum one? Anyway, I'll leave it to you. :)

[maan2003]

this can't affect devimint latency tests, in devimint test we only test using rocksdb

likely just due to executor improvments

[dpc]

this can't affect devimint latency tests, in devimint test we only test using rocksdb

likely just due to executor improvments

I don't know. Failed again in mq, while I don't see failures like that on other PRs.

[maan2003]

also failed for another PR
https://github.com/fedimint/fedimint/actions/runs/7977983266/job/21782122000#step:10:4064

[dpc]

also failed for another PR
https://github.com/fedimint/fedimint/actions/runs/7977983266/job/21782122000#step:10:4064

Hmm. So what do we do? Change the factor? Seems like usual case got much faster, while max can sometimes take couple of seconds.