Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make IOPerDrive Configurable #7

Merged
merged 1 commit into from
Sep 18, 2023
Merged

Conversation

moting9
Copy link
Contributor

@moting9 moting9 commented Sep 14, 2023

"ioperdrive" is fixed as 4 in code, but when there are multiple disks such as 12 disks in a server, the concurrent read result is abnormal.

  • this PR make "--ioperdrive" configurable by user.
  • and change default value from 4 to 2.

Here is test result in Intel Xeon 8480+ with 12 pcs KIOXIA Corporation NVMe SSD Controller Cx6

./dperf -f 10GiB -i 4 /mnt/d{1..12} --verbose

┌──────────┬───────────┬───────────┬───┐
│ PATH │ READ │ WRITE │ │
│ /mnt/d10 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d5 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d1 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d6 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d7 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d9 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d2 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d11 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d3 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d4 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d8 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d12 │ 1.6 GiB/s │ 1.1 GiB/s │ ✓ │
└──────────┴───────────┴───────────┴───┘
┌───────────┬────────────┐
│ TotalREAD │ TotalWRITE │
│ 19 GiB/s │ 14 GiB/s │
└───────────┴────────────┘

./dperf -f 10GiB /mnt/d{1..12} --verbose

┌──────────┬───────────┬───────────┬───┐
│ PATH │ READ │ WRITE │ │
│ /mnt/d5 │ 2.1 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d1 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d2 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d3 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d6 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d8 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d10 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d4 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d11 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d12 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d9 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d7 │ 2.0 GiB/s │ 1.1 GiB/s │ ✓ │
└──────────┴───────────┴───────────┴───┘
┌───────────┬────────────┐
│ TotalREAD │ TotalWRITE │
│ 24 GiB/s │ 13 GiB/s │
└───────────┴────────────┘

./dperf -f 10GiB -i 1 /mnt/d{1..12} --verbose

┌──────────┬───────────┬───────────┬───┐
│ PATH │ READ │ WRITE │ │
│ /mnt/d10 │ 3.0 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d12 │ 2.9 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d4 │ 2.9 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d11 │ 2.8 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d7 │ 2.8 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d3 │ 2.8 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d1 │ 2.8 GiB/s │ 1.0 GiB/s │ ✓ │
│ /mnt/d2 │ 2.8 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d8 │ 2.8 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d6 │ 2.7 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d9 │ 2.7 GiB/s │ 1.1 GiB/s │ ✓ │
│ /mnt/d5 │ 2.7 GiB/s │ 1.1 GiB/s │ ✓ │
└──────────┴───────────┴───────────┴───┘
┌───────────┬────────────┐
│ TotalREAD │ TotalWRITE │
│ 34 GiB/s │ 13 GiB/s │
└───────────┴────────────┘

@harshavardhana
Copy link
Member

A drive regardless of 1 thread or 4 threads should show overall performance to be the same with minor changes. Not to the extent of 1GiB/sec .

This PR however is valid for other tools not here, dperf is designed to be run once and captured. IF concurrent dperf I/O cannot be performed on the drive then there is no way it can perform for other applications.

That's why I see this to be problematic PR that can give wrong results.

cmd/cmd.go Outdated Show resolved Hide resolved
Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
@harshavardhana harshavardhana merged commit 31b8560 into minio:master Sep 18, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants