Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge 'Coroutinize commit log' from Calle Wilund
No real refactoring, just move the various methods to coroutines. Because coroutines are neat. Broken down into one method per change to make review easier. And hoping I get tipped per change. Grand idea being that using coroutines will eventually make real refactoring easier. Unit tests + relevant dtest. As discussed below, simply coroutinizing the code will, at least in the fast path, cause the slightly naive compiler to generate multiple unused coroutine frames, dropping raw performance a bit. The last two patches in this series addresses this, by breaking the fast path into non-coroutine subroutines (no futures involved) and one coroutine main loop. Results, as collected by `perf_simple_query` are: Master (before changes): ``` { "parameters" : { "concurrency" : 100, "concurrency,partitions,cpus,duration" : "100,10000,1,30", "cpus" : 1, "duration" : 30, "partitions" : 10000 }, "stats" : { "allocs_per_op" : 52.237303521776113, "instructions_per_op" : 47403.34422198555, "mad tps" : 670.12528706749436, "max tps" : 140817.0800358199, "median tps" : 139391.58369995767, "min tps" : 133663.0095463676, "tasks_per_op" : 13.189605506751203 }, "test_properties" : { "type" : "write" }, "versions" : { "scylla-server" : { "commit_id" : "1f51bc67fd", "date" : "20210712", "run_date_time" : "2021-07-13 10:26:46", "version" : "4.6.dev" } } } ``` This PR (coroutines + fast path optimization patches): ``` { "parameters" : { "concurrency" : 100, "concurrency,partitions,cpus,duration" : "100,10000,1,30", "cpus" : 1, "duration" : 30, "partitions" : 10000 }, "stats" : { "allocs_per_op" : 52.208628061750559, "instructions_per_op" : 47300.501878330339, "mad tps" : 707.70233700674726, "max tps" : 139618.0661493362, "median tps" : 137891.11290420164, "min tps" : 127551.83433347062, "tasks_per_op" : 13.172121395660733 }, "test_properties" : { "type" : "write" }, "versions" : { "scylla-server" : { "commit_id" : "1d4b6f50bd", "date" : "20210719", "run_date_time" : "2021-07-19 09:27:09", "version" : "4.6.dev" } } } ``` I.e. both allocations/op and instruction count seem to be on par. Closes #8954 * github.com:scylladb/scylla: commitlog: Make allocate_when_possible a template commitlog: break fast path alloc into non-fut/corout + outer loop commitlog: Drop stream/subscription from replayer commitlog: coroutinize commitlog::read_log_file commitlog: coroutinize commitlog::create_commitlog commitlog: coroutinize commitlog::add_entries commitlog: coroutinize commitlog::add_entry commitlog: coroutinize commitlog::add commitlog: change entry_writer usage to reference commitlog: coroutinize segment_manager::clear commitlog: coroutinize segment_manager::do_pending_deletes commitlog: coroutinize segment_manager::delete_file commitlog: coroutinize segment_manager::shutdown commitlog: coroutinize segment_manager::shutdown_all_segments commitlog: coroutinize segment_manager::sync_all_segments commitlog: coroutinize segment_manager::clear_reserve_segments commitlog: coroutinize segment_manager::active_segment commitlog: coroutinize segment_manager::new_segment commitlog: coroutinize segment_manager::allocate_segment commitlog: coroutinize segment_manager::rename_file commitlog: coroutinize segment_manager::init commitlog: coroutinize segment_manager::list_descriptors commitlog: coroutinize segment_manager::replenish_reserve commitlog: coroutinize segment::shutdown commitlog: coroutinize segment::close commitlog: coroutinize segment::batch_cycle commitlog: coroutinize segment::do_flush commitlog: coroutinize segment::flush commitlog: coroutinize segment::cycle commitlog: coroutinize allocate_when_possible commitlog: coroutinize segment::allocate
- Loading branch information