Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

recv-style abi #1002

Merged
merged 6 commits into from
May 22, 2024
Merged

recv-style abi #1002

merged 6 commits into from
May 22, 2024

Conversation

coolreader18
Copy link
Collaborator

Description of Changes

Still need to update the csharp wasm bindings. Also, unsure about that exhausted out param. Might be superfluous.

Expected complexity level and risk

3

Testing

smoketests pass - I think those cover all the API usage so I'm confident it doesn't crash or anything. bot-testing might be good to gauge perf improvement/impact.

@coolreader18 coolreader18 added the abi-break A PR that makes an ABI breaking change label Mar 21, 2024
gefjon
gefjon previously requested changes Mar 21, 2024
Copy link
Contributor

@gefjon gefjon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please write a document describing the design of the new ABI.

crates/bindings-sys/src/lib.rs Outdated Show resolved Hide resolved
crates/bindings-sys/src/lib.rs Show resolved Hide resolved
crates/bindings-sys/src/lib.rs Outdated Show resolved Hide resolved
crates/bindings-sys/src/lib.rs Outdated Show resolved Hide resolved
crates/bindings-sys/src/lib.rs Outdated Show resolved Hide resolved
crates/bindings/src/lib.rs Show resolved Hide resolved
crates/bindings/src/lib.rs Show resolved Hide resolved
crates/cli/src/tasks/mod.rs Outdated Show resolved Hide resolved
Comment on lines 51 to 48
// For now, just send buffers over a certain fixed size.
// This is kept in sync with `DEFAULT_BUFFER_CAPACITY` in `crates/bindings/src/lib.rs`
const ITER_CHUNK_SIZE: usize = 0x20_000;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could sneak this into spacetimedb-primitives as it is shared between bindings and db.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, true. errnos should probably go there as well.

crates/core/src/host/wasmtime/mod.rs Show resolved Hide resolved
@coolreader18 coolreader18 force-pushed the noa/recv_abi branch 2 times, most recently from 9294c74 to 5086358 Compare March 21, 2024 20:00
@bfops bfops added release-0.9 test-with-bots Recommend to test under higher CCU labels Mar 25, 2024
Comment on lines 88 to 100
trait ToBsatn {
fn to_bstan_extend(&self, buf: &mut Vec<u8>) -> Result<(), BsatnError>;
}
impl ToBsatn for RowRef<'_> {
fn to_bstan_extend(&self, buf: &mut Vec<u8>) -> Result<(), BsatnError> {
self.to_bsatn_extend(buf)
}
}
impl ToBsatn for RelValue<'_> {
fn to_bstan_extend(&self, buf: &mut Vec<u8>) -> Result<(), BsatnError> {
self.to_bsatn_extend(buf)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd call this trait ToBsatnExtend and the types could implement them directly and the inherent methods could be removed.

Copy link
Collaborator Author

@coolreader18 coolreader18 Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I might do that as a follow-up - ideally it'd probably go in the sats crate or something but I'd like to try making it more generalized if we do that.

Copy link
Contributor

@cloutiertyler cloutiertyler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove unnecessary ABI changes such as removing the underscore (see inline comments).

crates/bindings-sys/src/lib.rs Outdated Show resolved Hide resolved
@cloutiertyler
Copy link
Contributor

benchmarks please

Copy link

github-actions bot commented Mar 29, 2024

Criterion benchmark results

Criterion benchmark report

YOU SHOULD PROBABLY IGNORE THESE RESULTS.

Criterion is a wall time based benchmarking system that is extremely noisy when run on CI. We collect these results for longitudinal analysis, but they are not reliable for comparing individual PRs.

Go look at the callgrind report instead.

empty

db on disk new latency old latency new throughput old throughput
sqlite 💿 - 419.6±2.33ns - -
sqlite 🧠 - 418.9±2.44ns - -
stdb_raw 💿 720.8±0.78ns 709.4±1.72ns - -
stdb_raw 🧠 689.1±1.91ns 683.9±1.47ns - -

insert_1

db on disk schema indices preload new latency old latency new throughput old throughput

insert_bulk

db on disk schema indices preload count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 2048 256 - 511.1±0.81µs - 1956 tx/sec
sqlite 💿 u32_u64_str unique_0 2048 256 - 132.6±0.21µs - 7.4 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 2048 256 - 417.3±0.51µs - 2.3 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 2048 256 - 119.8±0.71µs - 8.2 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 2048 256 - 445.3±0.21µs - 2.2 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 2048 256 - 116.2±0.61µs - 8.4 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 2048 256 - 366.0±0.35µs - 2.7 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 2048 256 - 102.8±0.49µs - 9.5 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 2048 256 723.0±0.27µs 717.4±0.61µs 1383 tx/sec 1393 tx/sec
stdb_raw 💿 u32_u64_str unique_0 2048 256 624.8±0.66µs 627.7±0.62µs 1600 tx/sec 1593 tx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 2048 256 414.2±0.42µs 402.8±1.20µs 2.4 Ktx/sec 2.4 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 2048 256 365.4±0.60µs 363.8±0.43µs 2.7 Ktx/sec 2.7 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 2048 256 496.3±0.36µs 564.8±0.44µs 2014 tx/sec 1770 tx/sec
stdb_raw 🧠 u32_u64_str unique_0 2048 256 406.0±0.70µs 473.3±0.55µs 2.4 Ktx/sec 2.1 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 2048 256 311.5±0.19µs 369.7±0.22µs 3.1 Ktx/sec 2.6 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 2048 256 266.5±0.24µs 330.4±0.35µs 3.7 Ktx/sec 3.0 Ktx/sec

iterate

db on disk schema indices new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 - 19.8±0.20µs - 49.4 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 - 18.2±0.03µs - 53.8 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 - 18.1±0.26µs - 53.9 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 - 17.0±0.04µs - 57.6 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 18.7±0.00µs 18.7±0.00µs 52.3 Ktx/sec 52.3 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 15.9±0.00µs 15.8±0.00µs 61.6 Ktx/sec 61.7 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 18.6±0.00µs 18.6±0.00µs 52.4 Ktx/sec 52.4 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 15.8±0.00µs 15.8±0.00µs 61.7 Ktx/sec 61.8 Ktx/sec

find_unique

db on disk key type preload new latency old latency new throughput old throughput

filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 2048 256 - 64.5±0.24µs - 15.1 Ktx/sec
sqlite 💿 u64 index 2048 256 - 63.1±0.14µs - 15.5 Ktx/sec
sqlite 🧠 string index 2048 256 - 62.7±0.57µs - 15.6 Ktx/sec
sqlite 🧠 u64 index 2048 256 - 58.8±0.14µs - 16.6 Ktx/sec
stdb_raw 💿 string index 2048 256 5.6±0.00µs 5.6±0.00µs 175.3 Ktx/sec 175.2 Ktx/sec
stdb_raw 💿 u64 index 2048 256 5.5±0.00µs 5.6±0.00µs 177.5 Ktx/sec 175.9 Ktx/sec
stdb_raw 🧠 string index 2048 256 5.5±0.00µs 5.6±0.00µs 176.5 Ktx/sec 175.9 Ktx/sec
stdb_raw 🧠 u64 index 2048 256 5.5±0.00µs 5.5±0.00µs 178.8 Ktx/sec 177.0 Ktx/sec

serialize

schema format count new latency old latency new throughput old throughput
u32_u64_str bflatn_to_bsatn_fast_path 100 3.9±0.01µs 4.0±0.01µs 24.2 Mtx/sec 23.9 Mtx/sec
u32_u64_str bflatn_to_bsatn_slow_path 100 3.7±0.01µs 3.7±0.05µs 25.6 Mtx/sec 25.6 Mtx/sec
u32_u64_str bsatn 100 2.6±0.01µs 2.5±0.00µs 36.5 Mtx/sec 38.6 Mtx/sec
u32_u64_str json 100 5.1±0.07µs 5.1±0.07µs 18.7 Mtx/sec 18.8 Mtx/sec
u32_u64_str product_value 100 677.4±21.70ns 656.1±1.18ns 140.8 Mtx/sec 145.4 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_fast_path 100 1315.1±14.97ns 1344.1±3.73ns 72.5 Mtx/sec 71.0 Mtx/sec
u32_u64_u64 bflatn_to_bsatn_slow_path 100 2.9±0.00µs 3.0±0.10µs 32.7 Mtx/sec 32.2 Mtx/sec
u32_u64_u64 bsatn 100 1775.7±32.34ns 1734.4±31.81ns 53.7 Mtx/sec 55.0 Mtx/sec
u32_u64_u64 json 100 3.1±0.06µs 3.4±0.04µs 30.3 Mtx/sec 27.9 Mtx/sec
u32_u64_u64 product_value 100 563.5±1.00ns 611.6±0.44ns 169.2 Mtx/sec 155.9 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_fast_path 100 1109.5±1.90ns 1111.5±14.92ns 86.0 Mtx/sec 85.8 Mtx/sec
u64_u64_u32 bflatn_to_bsatn_slow_path 100 2.9±0.00µs 2.9±0.01µs 32.7 Mtx/sec 32.6 Mtx/sec
u64_u64_u32 bsatn 100 1767.5±27.87ns 1737.8±34.83ns 54.0 Mtx/sec 54.9 Mtx/sec
u64_u64_u32 json 100 3.3±0.07µs 3.5±0.02µs 28.6 Mtx/sec 27.1 Mtx/sec
u64_u64_u32 product_value 100 598.9±0.38ns 604.9±0.78ns 159.2 Mtx/sec 157.7 Mtx/sec

stdb_module_large_arguments

arg size new latency old latency new throughput old throughput
64KiB 73.7±8.15µs 89.3±3.21µs - -

stdb_module_print_bulk

line count new latency old latency new throughput old throughput
1 46.6±5.13µs 43.9±6.16µs - -
100 363.4±60.84µs 349.2±5.57µs - -
1000 2.3±0.45ms 1868.1±22.13µs - -

remaining

name new latency old latency new throughput old throughput
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 - 46.0±0.12µs - 21.2 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 - 40.6±0.10µs - 24.1 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 - 38.4±0.07µs - 25.4 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 - 35.1±0.14µs - 27.8 Ktx/sec
stdb_module/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 2.9±0.01ms 2.8±0.04ms 341 tx/sec 354 tx/sec
stdb_module/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 1851.4±20.67µs 1823.0±1.58µs 540 tx/sec 548 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=2048/count=256 1113.3±1.92µs 1102.4±0.72µs 898 tx/sec 907 tx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 729.0±0.53µs 723.0±0.44µs 1371 tx/sec 1383 tx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=2048/count=256 786.9±1.02µs 911.4±0.88µs 1270 tx/sec 1097 tx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=2048/count=256 535.4±0.29µs 663.8±0.48µs 1867 tx/sec 1506 tx/sec

@coolreader18 coolreader18 added this pull request to the merge queue May 22, 2024
Merged via the queue into master with commit 471f4ff May 22, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
abi-break A PR that makes an ABI breaking change release-1.0 test-with-bots Recommend to test under higher CCU
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants