Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new gas metering method: mutable global + local gas function #34

Merged
merged 59 commits into from
Nov 20, 2022
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
926fcd5
fix misprints in doc comments
agryaznov Sep 26, 2022
448e16b
added global gas tracker variable and local gas fn
agryaznov Oct 3, 2022
445a5d3
all exported functions of the module to accept a new param and to set…
agryaznov Oct 3, 2022
b0e1f29
make module support both gas metering methods
agryaznov Oct 4, 2022
af54747
tests fixed for the old metering method
agryaznov Oct 4, 2022
dc3845a
better naming
agryaznov Oct 4, 2022
daa48b3
MutableGlobal metering method implemented, tests for the old method pass
agryaznov Oct 5, 2022
cfd8381
gas_metering::tests updated and pass
agryaznov Oct 6, 2022
a557147
all tests udpdated and pass
agryaznov Oct 6, 2022
c08caa4
emacs backup files to .gitignore
agryaznov Oct 6, 2022
ff59648
docs updated
agryaznov Oct 6, 2022
144e272
clippy fix
agryaznov Oct 6, 2022
b0ef59b
iff = if and only if
agryaznov Oct 6, 2022
4389779
more clippy
agryaznov Oct 6, 2022
7f08437
docs misprints fixes
agryaznov Oct 7, 2022
afd36f2
refactored to have Backend trait and two implementations in separate …
agryaznov Oct 11, 2022
bd20b95
docs updated
agryaznov Oct 11, 2022
a54eab0
fixed old benches (updating them is coming next)
agryaznov Oct 11, 2022
074422c
added bench for an instrumented wasm-coremark
agryaznov Oct 11, 2022
2fa2527
updated benches: added them for both gas_metering instrumentations
agryaznov Oct 11, 2022
b6cad4e
benches contest first ver
agryaznov Oct 12, 2022
3e674d4
added debug prints to the bench
agryaznov Oct 12, 2022
af67b82
refactored to better fit frontend-backend pattern
agryaznov Oct 12, 2022
e87387e
docs update
agryaznov Oct 12, 2022
10b862e
updated benches
agryaznov Oct 12, 2022
baf7627
design updated on feedback
agryaznov Oct 14, 2022
0dce7c6
re-structured sub-modules
agryaznov Oct 14, 2022
ec6bd43
docs improved
agryaznov Oct 15, 2022
5651469
addressed latest feedback comments
agryaznov Oct 18, 2022
7280e8a
re-writed the local gas function
agryaznov Oct 18, 2022
bbb7ffb
coremark benches show ~20% performance improvement
agryaznov Oct 18, 2022
3dc5513
fix ci: test + clippy
agryaznov Oct 19, 2022
ad12c85
save before re-factoring prepare_in_wasm()
agryaznov Oct 19, 2022
bea3bc4
bare_call_16 shows 16% worse perf
agryaznov Oct 28, 2022
a9eaff1
+ fibonacci recursive bench
agryaznov Oct 29, 2022
ee7b2ef
refactored benchmarks
agryaznov Oct 30, 2022
d7457a8
+ factorial recursive bench
agryaznov Oct 30, 2022
9f4b07e
benches on wasmi fixtures show no perf improvement, coremark runs ~20…
agryaznov Oct 31, 2022
3a7363f
charge gas for local gas func isntructions execution
agryaznov Nov 1, 2022
2e3d34f
replaced benchmark which requires multi_value feature
agryaznov Nov 1, 2022
cdc5ba3
save: optimized gas func a bit (benches work, fixture tests fail)
agryaznov Nov 2, 2022
d887e72
1033% overhead on many_blocks.wasm when mut_global gas_metering toget…
agryaznov Nov 10, 2022
f2510b1
size overhead test for both gas metering methods + stack limiter
agryaznov Nov 11, 2022
77cc080
added more benches
agryaznov Nov 11, 2022
4d711bd
improved print_size_overhead test
agryaznov Nov 11, 2022
b3980e8
test for comparing size overheads of two gas_metering injectors
agryaznov Nov 12, 2022
1c986d0
before optimization: benches + size overhead
agryaznov Nov 14, 2022
4bbe7f4
optimization try-1: inline part of gas func instructions: +benches +s…
agryaznov Nov 12, 2022
1290935
optimization try-2: inline hot path of gas fn: +benches +size overheads
agryaznov Nov 14, 2022
87ee309
opt try-3: count for gas fn cost on the caller side: +benches +size o…
agryaznov Nov 14, 2022
3583140
revert to initial version but with static gas fn cost on the caller s…
agryaznov Nov 16, 2022
ba653eb
tests fixed
agryaznov Nov 16, 2022
810abd1
use newest wasmi 0.20: +benches +docs updated
agryaznov Nov 16, 2022
6f43fbc
use if-else block instead of Return: +benches
agryaznov Nov 18, 2022
38d8edb
fix tests
agryaznov Nov 18, 2022
0142940
Merge branch 'master' into ag-gas-metering
agryaznov Nov 18, 2022
832eeb8
clippy fix
agryaznov Nov 18, 2022
66e8833
addressed review comments
agryaznov Nov 18, 2022
f17b1e1
Update changelog
athei Nov 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ target
.DS_Store
.idea
.vscode
*~
4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,12 @@ rand = "0.8"
wat = "1"
wasmparser = "0.90"
wasmprinter = "0.2"
wasmi = "0.18"

[features]
default = ["std"]
std = ["parity-wasm/std"]
sign_ext = ["parity-wasm/sign_ext"]

[lib]
bench = false
athei marked this conversation as resolved.
Show resolved Hide resolved
283 changes: 279 additions & 4 deletions benches/benches.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,16 @@ use std::{
path::PathBuf,
};
use wasm_instrument::{
gas_metering, inject_stack_limiter,
parity_wasm::{deserialize_buffer, elements::Module},
gas_metering::{self, host_function, mutable_global, Backend, ConstantCostRules},
inject_stack_limiter,
parity_wasm::{deserialize_buffer, elements::Module, serialize},
};

fn fixture_dir() -> PathBuf {
let mut path = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
path.push("benches");
path.push("fixtures");
path.push("wasm");
path
}

Expand All @@ -36,7 +38,12 @@ where
fn gas_metering(c: &mut Criterion) {
let mut group = c.benchmark_group("Gas Metering");
any_fixture(&mut group, |module| {
gas_metering::inject(module, &gas_metering::ConstantCostRules::default(), "env").unwrap();
gas_metering::inject(
module,
host_function::Injector::new("env", "gas"),
&ConstantCostRules::default(),
)
.unwrap();
});
}

Expand All @@ -47,5 +54,273 @@ fn stack_height_limiter(c: &mut Criterion) {
});
}

use std::time::{Duration, SystemTime, UNIX_EPOCH};
use wasmi::{
self,
core::{Value, F32},
Caller, Config, Engine, Extern, Func, Instance, Linker, StackLimits, Store,
};
fn prepare_module<P: Backend>(backend: P, input: &[u8]) -> (wasmi::Module, Store<u64>) {
let module = deserialize_buffer(input).unwrap();
let instrumented_module =
gas_metering::inject(module, backend, &ConstantCostRules::default()).unwrap();
let input = serialize(instrumented_module).unwrap();
// Prepare wasmi
let engine = Engine::new(&bench_config());
let module = wasmi::Module::new(&engine, &mut &input[..]).unwrap();
// Init host state with maximum gas_left
let store = Store::new(&engine, u64::MAX);

(module, store)
}

fn add_gas_host_func(linker: &mut Linker<u64>, store: &mut Store<u64>) {
// Create gas host function
let host_gas = Func::wrap(store, |mut caller: Caller<'_, u64>, param: u64| {
*caller.host_data_mut() -= param;
});
// Link the gas host function
linker.define("env", "gas", host_gas).unwrap();
}

fn add_gas_left_global(instance: &Instance, mut store: Store<u64>) -> Store<u64> {
instance
.get_export(&mut store, "gas_left")
.and_then(Extern::into_global)
.unwrap()
.set(&mut store, Value::I64(i64::MAX))
.unwrap();
store
}

fn gas_metered_coremark(c: &mut Criterion) {
let mut group = c.benchmark_group("Coremark, instrumented");
// Benchmark host_function::Injector
let wasm_filename = "coremark_minimal.wasm";
let bytes = read(fixture_dir().join(wasm_filename)).unwrap();
group.bench_function("with host_function::Injector", |bench| {
let backend = host_function::Injector::new("env", "gas");
let (module, mut store) = prepare_module(backend, &bytes);
// Link the host functions with the imported ones
let mut linker = <Linker<u64>>::new();
add_gas_host_func(&mut linker, &mut store);
// Create clock_ms host function.
let host_clock_ms = Func::wrap(&mut store, || {
SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_millis() as u64
});
// Link the time measurer for the coremark wasm
linker.define("env", "clock_ms", host_clock_ms).unwrap();

let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();

bench.iter(|| {
let run = instance
.get_export(&mut store, "run")
.and_then(Extern::into_func)
.unwrap()
.typed::<(), F32>(&mut store)
.unwrap();
// Call the wasm!
run.call(&mut store, ()).unwrap();
})
});

group.bench_function("with mutable_global::Injector", |bench| {
let backend = mutable_global::Injector::new("gas_left");
let (module, mut store) = prepare_module(backend, &bytes);
// Add the gas_left mutable global
let mut linker = <Linker<u64>>::new();
// Create clock_ms host function.
let host_clock_ms = Func::wrap(&mut store, || {
SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_millis() as u64
});
// Link the time measurer for the coremark wasm
linker.define("env", "clock_ms", host_clock_ms).unwrap();

let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();
let mut store = add_gas_left_global(&instance, store);

bench.iter(|| {
let run = instance
.get_export(&mut store, "run")
.and_then(Extern::into_func)
.unwrap()
.typed::<(), F32>(&mut store)
.unwrap();
// Call the wasm!
run.call(&mut store, ()).unwrap();
})
});
}

/// Converts the `.wat` encoded `bytes` into `.wasm` encoded bytes.
pub fn wat2wasm(bytes: &[u8]) -> Vec<u8> {
wat::parse_bytes(bytes).unwrap().into_owned()
}

/// Returns a [`Config`] useful for benchmarking.
fn bench_config() -> Config {
let mut config = Config::default();
config.set_stack_limits(StackLimits::new(1024, 1024 * 1024, 64 * 1024).unwrap());
config
}

fn gas_metered_recursive_ok(c: &mut Criterion) {
let mut group = c.benchmark_group("recursive_ok, instrumented");
const RECURSIVE_DEPTH: i32 = 8000;
let wasm_bytes = wat2wasm(include_bytes!("fixtures/wat/recursive_ok.wat"));

group.bench_function("with host_function::Injector", |bench| {
let backend = host_function::Injector::new("env", "gas");
let (module, mut store) = prepare_module(backend, &wasm_bytes);
// Link the host function with the imported one
let mut linker = <Linker<u64>>::new();
add_gas_host_func(&mut linker, &mut store);
let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();

let bench_call = instance.get_export(&store, "call").and_then(Extern::into_func).unwrap();
let mut result = [Value::I32(0)];

bench.iter(|| {
bench_call
.call(&mut store, &[Value::I32(RECURSIVE_DEPTH)], &mut result)
.unwrap();
assert_eq!(result, [Value::I32(0)]);
})
});

group.bench_function("with mutable_global::Injector", |bench| {
let backend = mutable_global::Injector::new("gas_left");
let (module, mut store) = prepare_module(backend, &wasm_bytes);
// Add the gas_left mutable global
let mut linker = <Linker<u64>>::new();
let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();
let mut store = add_gas_left_global(&instance, store);

let bench_call = instance.get_export(&store, "call").and_then(Extern::into_func).unwrap();
let mut result = [Value::I32(0)];

bench.iter(|| {
bench_call
.call(&mut store, &[Value::I32(RECURSIVE_DEPTH)], &mut result)
.unwrap();
assert_eq!(result, [Value::I32(0)]);
})
});
}

fn gas_metered_fibonacci_recursive(c: &mut Criterion) {
let mut group = c.benchmark_group("fibonacci_recursive, instrumented");
const FIBONACCI_REC_N: i64 = 10;
let wasm_bytes = wat2wasm(include_bytes!("fixtures/wat/fibonacci.wat"));

group.bench_function("with host_function::Injector", |bench| {
let backend = host_function::Injector::new("env", "gas");
let (module, mut store) = prepare_module(backend, &wasm_bytes);
// Link the host function with the imported one
let mut linker = <Linker<u64>>::new();
add_gas_host_func(&mut linker, &mut store);
let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();

let bench_call = instance
.get_export(&store, "fib_recursive")
.and_then(Extern::into_func)
.unwrap();
let mut result = [Value::I32(0)];

bench.iter(|| {
bench_call
.call(&mut store, &[Value::I64(FIBONACCI_REC_N)], &mut result)
.unwrap();
});
});

group.bench_function("with mutable_global::Injector", |bench| {
let backend = mutable_global::Injector::new("gas_left");
let (module, mut store) = prepare_module(backend, &wasm_bytes);

// Add the gas_left mutable global
let mut linker = <Linker<u64>>::new();
let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();
let mut store = add_gas_left_global(&instance, store);

let bench_call = instance
.get_export(&store, "fib_recursive")
.and_then(Extern::into_func)
.unwrap();
let mut result = [Value::I32(0)];

bench.iter(|| {
bench_call
.call(&mut store, &[Value::I64(FIBONACCI_REC_N)], &mut result)
.unwrap();
});
});
}

fn gas_metered_fac_recursive(c: &mut Criterion) {
let mut group = c.benchmark_group("factorial_recursive, instrumented");
let wasm_bytes = wat2wasm(include_bytes!("fixtures/wat/factorial.wat"));

group.bench_function("with host_function::Injector", |b| {
let backend = host_function::Injector::new("env", "gas");
let (module, mut store) = prepare_module(backend, &wasm_bytes);
// Link the host function with the imported one
let mut linker = <Linker<u64>>::new();
add_gas_host_func(&mut linker, &mut store);
let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();
let fac = instance
.get_export(&store, "recursive_factorial")
.and_then(Extern::into_func)
.unwrap();
let mut result = [Value::I64(0)];

b.iter(|| {
fac.call(&mut store, &[Value::I64(25)], &mut result).unwrap();
assert_eq!(result, [Value::I64(7034535277573963776)]);
})
});

group.bench_function("with mutable_global::Injector", |b| {
let backend = mutable_global::Injector::new("gas_left");
let (module, mut store) = prepare_module(backend, &wasm_bytes);

// Add the gas_left mutable global
let mut linker = <Linker<u64>>::new();
let instance = linker.instantiate(&mut store, &module).unwrap().start(&mut store).unwrap();
let mut store = add_gas_left_global(&instance, store);
let fac = instance
.get_export(&store, "recursive_factorial")
.and_then(Extern::into_func)
.unwrap();
let mut result = [Value::I64(0)];

b.iter(|| {
fac.call(&mut store, &[Value::I64(25)], &mut result).unwrap();
assert_eq!(result, [Value::I64(7034535277573963776)]);
})
});
}

criterion_group!(benches, gas_metering, stack_height_limiter);
criterion_main!(benches);
criterion_group!(
name = coremark;
config = Criterion::default()
.sample_size(10)
.measurement_time(Duration::from_millis(250000))
.warm_up_time(Duration::from_millis(1000));
targets =
gas_metered_coremark,
);
criterion_group!(
name = wasmi_fixtures;
config = Criterion::default()
.sample_size(10)
.measurement_time(Duration::from_millis(250000))
.warm_up_time(Duration::from_millis(1000));
targets =
gas_metered_recursive_ok,
gas_metered_fibonacci_recursive,
gas_metered_fac_recursive,
);
criterion_main!(coremark, wasmi_fixtures);
Binary file added benches/fixtures/wasm/coremark_minimal.wasm
Binary file not shown.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
35 changes: 35 additions & 0 deletions benches/fixtures/wat/factorial.wat
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
(module
;; Iterative factorial function, does not use recursion.
(func (export "iterative_factorial") (param i64) (result i64)
(local i64)
(local.set 1 (i64.const 1))
(block
(br_if 0 (i64.lt_s (local.get 0) (i64.const 2)))
(loop
(local.set 1 (i64.mul (local.get 1) (local.get 0)))
(local.set 0 (i64.add (local.get 0) (i64.const -1)))
(br_if 0 (i64.gt_s (local.get 0) (i64.const 1)))
)
)
(local.get 1)
)

;; Recursive trivial factorial function.
(func $rec_fac (export "recursive_factorial") (param i64) (result i64)
(if (result i64)
(i64.eq (local.get 0) (i64.const 0))
(then (i64.const 1))
(else
(i64.mul
(local.get 0)
(call $rec_fac
(i64.sub
(local.get 0)
(i64.const 1)
)
)
)
)
)
)
)