Skip to content

/rustfs

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark the throughput of using SPDK #21

Merged
merged 42 commits into from Apr 20, 2019

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@xxks-kkk
@xxks-kkk
Copy link
Member

commented Apr 20, 2019

Sequential write using dd on SSD vs. sequential write using Rust + SPDK framework

xxks-kkk added some commits Dec 15, 2018

@xxks-kkk
Start to implement benchmarks
48a6459
@xxks-kkk
Add a simple dd command; subject to change
9ffad32
@xxks-kkk
Start to use TOML
081835c
@xxks-kkk
Finish seq write benchmark using dd
0efb270
@xxks-kkk
Prepare to implement SPDK-based Rust benchmark
d2d0d20
@xxks-kkk
Port hello_nvme_bdev_rust_wrapper code to language.rs; now ready to m… 
…odify
c9b1e8a
@xxks-kkk
async for loop seem to work
fd2c03d
@xxks-kkk
Move strip function to utils package
4741376
@xxks-kkk
Add new wrapper function to be used by SPDK write benchmark measurement
96b534d
@xxks-kkk
Add convert helper function
92de15a
@xxks-kkk
integrate logger into the code; more info see: https://rust-lang-nurs… 
…ery.github.io/rust-cookbook/development_tools/debugging/log.html and https://rust-lang-nursery.github.io/rust-cookbook/development_tools/debugging/config_log.html
8b4af91
@xxks-kkk
Add convert_time helper function
9a8398b
@xxks-kkk
Modify bdev.conf to use the NVMe device on thoothukudi-lom
b878e1c
@xxks-kkk
Finish preliminary sequential write benchmark for SPDK+rust; still ne… 
…eds to polish
b2bfd0d
@xxks-kkk
Add generate_string function
9aef73a
@xxks-kkk
Instead of writing zeros, we now write random string from /dev/urandom
539369b
@xxks-kkk
Fix a bug that pass in the wrong size as parameter to bdev::write and… 
… expose one more bug
50e78aa
@xxks-kkk
Add fill_fixed & follow the suggestion in issue 578 of SPDK: spdk/spd… 
…k#578
7d7130a
@xxks-kkk
Bump rust version to satisfy the latest dependency requirement
ce9008e
@xxks-kkk
Able to run cargo test under spdk-rs; fix one small bug
4cdac24
@xxks-kkk
Add test case to env.rs
091da3f
@xxks-kkk
A new number
147e354
@xxks-kkk
sequential write seems to work
d944ada
@xxks-kkk
Add check 1 to rust_seq; enable to colorful print on shell; Add test … 
…case framework to language.rs
8d37ec0
@xxks-kkk
refactor rust_seq to support check2
4018733
@xxks-kkk
Add generate_file_random()
ae8904a
@xxks-kkk
We need to rethink how we want to implement check 2) effectively
5d64bcc
@xxks-kkk
Add fill_from_file function and corresponding test
0399287
@xxks-kkk
Add print_of_type helper function
e14ad73
@xxks-kkk
Add get_checksum helper function
9c4b894
@xxks-kkk
Implement check2 but there is a bug when we try to unwrap() the read_… 
…buffer.read(); seems like the bug happens in the fill_from_file() implementation: a character may possibly get truncated?
152bb85
@xxks-kkk

This comment has been minimized.

Sign in to view
Copy link
Member Author

commented on 50e78aa Feb 8, 2019

*(self.to_raw() as *mut u8).offset(read_start as isize) = '\0' as u8; typical usage.

xxks-kkk added some commits Feb 8, 2019

@xxks-kkk
rewrite fill_from_file function with better implementation; bug still… 
… exists
cfa8f03
@xxks-kkk
now we try alternative approach to workaround bug; however a new segf… 
…ault error happens
8559dbf
@xxks-kkk
Fix segfault by replacing unsafe fill_fixed() with rust-friendly impl… 
…ementation
ac7a934
@xxks-kkk
Fix the bug to the point where now the only thing is checksum different
bbe3eb3
@xxks-kkk
Now we have two issues: 1. cargo run breaks (that leads to bus error)… 
…. To be honest, that's the frustrating part of development: your used-to-work code seems to break magically and we don't have travis to do periodically build test due to the speciality of SPDK (this issue should be resolved in QEMU otherwise we have to restart the host machine) 2. checksum difference: there is only one character off between the write and read text
a996990
@xxks-kkk
cleanup utils lib.rs
9028b77
@xxks-kkk
Cleanup spdk-rs to address all the warnings raised by rustc
7649352
@xxks-kkk
Add argument parse to run different benchmarks under language.rs
dbe0849
@xxks-kkk
start to refactor
194cb86
@xxks-kkk
can run but the number may not make sense
4905615
@xxks-kkk
Update throughput benchmark
1cd66de

@xxks-kkk xxks-kkk merged commit f1d1d1c into master Apr 20, 2019

@xxks-kkk xxks-kkk deleted the overhead-benchmark-refactor branch Apr 20, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.