tests

Helper utilities for ensuring that musli is safe to use and behaves as expected.

This does include a fairly sophisticated benchmarking suite.

Benchmarking

To run the benchmarks included in this crate, use the separate tools crate.

cargo run -p tools -- --bench --force

Note: --force will overwriting existing violing plots.

This will ensure that only crates with comparable feature sets are grouped together, and at the end generate a benchmarks-new/index.md report.

The tests tool

This comes with the tests binary which can do the following things:

Run serialization for a long time against a lot of structured random data. Both with and without miri. When run with miri, the size of the datasets is drastically reduced.

cargo run -p tests --features musli-wire
cargo +nightly miri run -p tests --features musli-wire

Run deserialization against randomly generated bytes. Both with and without miri. When run with miri, the --features musli-wire size of the datasets is drastically reduced.

cargo run -p tests --features musli-wire -- --random
cargo +nightly miri run -p tests --features musli-wire -- --random

Note you can add the running of optional crates by enabling its corresponding feature, such as --features bincode-serde.

To select which framework(s) to fuzz, enabled their corresponding features. If you just want to fuzz serde_json, you'd run:

cargo run -p tests --features serde_json

Just beware that enabling certain dependencies limits the models being used.

Potential security issues reported through this crate

• dlhn: Allocating and initializing large arrays based on untrusted input (DoS): dlhn#11.
• rkyv: Undefined Behavior: stacked borrows violation rkyv#436.
• bincode: bincode 2 overallocates on untrusted inputs bincode#764¹.

Other issues discovered by this crate

• bincode: bincode_derive does not like generic parameters with defaults: bincode#763
• simd_json: Incomplete serialization for zero-element tuple variant: simd_json#416.

Size comparison

To perform a size comparison of a bunch of generated structures:

cargo run -p tests -- --size

This reports all the sizings as a JSON, which is intended to be consumed by the benchmarking tools crate.

Footnotes

1. Note that upstream sees this as an enhancement, while I consider this a security issue. It affects the default configuration of bincode and the mitigation proposed should be in place to protect any configuration. ↩