Permalink
Cannot retrieve contributors at this time
Fetching contributors…
Cannot retrieve contributors at this time
| In the meantime, the very useful --report-interval switch has been added to | |
| development versions of sysbench, and I've had access to one additional system. | |
| If I thought that the internal RAID was bad, that's only because I have not yet | |
| experienced an external enclosure from HP attached via FibreChannel (P2000 G3 | |
| MSA, QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA). | |
| Unfortunately, I don't have detailed information about the configuration of | |
| this enclosure, except that it's a RAID6 volume, with 10 or 12 disks, I | |
| believe. | |
| Witness this horrendous tanking of write throughput: | |
| [ 2s] reads: 0.00 MB/s writes: 0.07 MB/s fsyncs: 0.00/s response time: 0.616ms (95%) | |
| [ 4s] reads: 0.00 MB/s writes: 14.10 MB/s fsyncs: 0.00/s response time: 0.481ms (95%) | |
| [ 6s] reads: 0.00 MB/s writes: 15.28 MB/s fsyncs: 0.00/s response time: 0.458ms (95%) | |
| [ 8s] reads: 0.00 MB/s writes: 14.65 MB/s fsyncs: 0.00/s response time: 0.464ms (95%) | |
| [ 10s] reads: 0.00 MB/s writes: 15.32 MB/s fsyncs: 0.00/s response time: 0.447ms (95%) | |
| [ 12s] reads: 0.00 MB/s writes: 15.18 MB/s fsyncs: 0.00/s response time: 0.460ms (95%) | |
| [ 14s] reads: 0.00 MB/s writes: 15.18 MB/s fsyncs: 0.00/s response time: 0.471ms (95%) | |
| [ 16s] reads: 0.00 MB/s writes: 14.06 MB/s fsyncs: 0.00/s response time: 0.468ms (95%) | |
| [ 18s] reads: 0.00 MB/s writes: 0.43 MB/s fsyncs: 0.00/s response time: 3.933ms (95%) | |
| [ 20s] reads: 0.00 MB/s writes: 0.00 MB/s fsyncs: 0.00/s response time: 985.122ms (95%) | |
| [ 22s] reads: 0.00 MB/s writes: 0.01 MB/s fsyncs: 0.00/s response time: 1435.164ms (95%) | |
| [ 24s] reads: 0.00 MB/s writes: 0.00 MB/s fsyncs: 0.00/s response time: 1194.568ms (95%) | |
| [ 26s] reads: 0.00 MB/s writes: 0.00 MB/s fsyncs: 0.00/s response time: 1112.091ms (95%) | |
| [ 28s] reads: 0.00 MB/s writes: 0.01 MB/s fsyncs: 0.00/s response time: 1443.350ms (95%) | |
| [ 30s] reads: 0.00 MB/s writes: 0.00 MB/s fsyncs: 0.00/s response time: 1078.972ms (95%) | |
| Operations performed: 0 reads, 53413 writes, 0 Other = 53413 Total | |
| Read 0b Written 208.64Mb Total transferred 208.64Mb (6.8007Mb/sec) | |
| 1740.98 Requests/sec executed | |
| For comparison, this is the SmartArray P400 RAID6 that I initially complained about: | |
| [ 2s] reads: 0.00 MB/s writes: 6.34 MB/s fsyncs: 0.00/s response time: 0.219ms (95%) | |
| [ 4s] reads: 0.00 MB/s writes: 5.35 MB/s fsyncs: 0.00/s response time: 0.217ms (95%) | |
| [ 6s] reads: 0.00 MB/s writes: 5.48 MB/s fsyncs: 0.00/s response time: 0.208ms (95%) | |
| [ 8s] reads: 0.00 MB/s writes: 5.30 MB/s fsyncs: 0.00/s response time: 0.228ms (95%) | |
| [ 10s] reads: 0.00 MB/s writes: 5.81 MB/s fsyncs: 0.00/s response time: 0.226ms (95%) | |
| [ 12s] reads: 0.00 MB/s writes: 6.01 MB/s fsyncs: 0.00/s response time: 0.223ms (95%) | |
| [ 14s] reads: 0.00 MB/s writes: 5.39 MB/s fsyncs: 0.00/s response time: 0.212ms (95%) | |
| [ 16s] reads: 0.00 MB/s writes: 5.21 MB/s fsyncs: 0.00/s response time: 0.225ms (95%) | |
| [ 18s] reads: 0.00 MB/s writes: 5.16 MB/s fsyncs: 0.00/s response time: 0.224ms (95%) | |
| [ 20s] reads: 0.00 MB/s writes: 5.97 MB/s fsyncs: 0.00/s response time: 0.217ms (95%) | |
| [ 22s] reads: 0.00 MB/s writes: 4.28 MB/s fsyncs: 0.00/s response time: 0.228ms (95%) | |
| [ 24s] reads: 0.00 MB/s writes: 7.44 MB/s fsyncs: 0.00/s response time: 0.191ms (95%) | |
| [ 26s] reads: 0.00 MB/s writes: 5.30 MB/s fsyncs: 0.00/s response time: 0.250ms (95%) | |
| [ 28s] reads: 0.00 MB/s writes: 5.45 MB/s fsyncs: 0.00/s response time: 0.258ms (95%) | |
| [ 30s] reads: 0.00 MB/s writes: 5.27 MB/s fsyncs: 0.00/s response time: 0.254ms (95%) | |
| Operations performed: 0 reads, 42890 writes, 0 Other = 42890 Total | |
| Read 0b Written 167.54Mb Total transferred 167.54Mb (5.5773Mb/sec) | |
| 1427.80 Requests/sec executed | |
| Slow, but at least it's consistent. | |
| And that's what I would expect, and which a decent RAID controller manages to | |
| provide (LSI Logic / Symbios Logic MegaRAID SAS 1078): | |
| [ 2s] reads: 0.00 MB/s writes: 56.65 MB/s fsyncs: 0.00/s response time: 0.117ms (95%) | |
| [ 4s] reads: 0.00 MB/s writes: 37.15 MB/s fsyncs: 0.00/s response time: 0.221ms (95%) | |
| [ 6s] reads: 0.00 MB/s writes: 35.92 MB/s fsyncs: 0.00/s response time: 0.225ms (95%) | |
| [ 8s] reads: 0.00 MB/s writes: 34.15 MB/s fsyncs: 0.00/s response time: 0.239ms (95%) | |
| [ 10s] reads: 0.00 MB/s writes: 33.19 MB/s fsyncs: 0.00/s response time: 0.221ms (95%) | |
| [ 12s] reads: 0.00 MB/s writes: 34.02 MB/s fsyncs: 0.00/s response time: 0.229ms (95%) | |
| [ 14s] reads: 0.00 MB/s writes: 36.61 MB/s fsyncs: 0.00/s response time: 0.233ms (95%) | |
| [ 16s] reads: 0.00 MB/s writes: 37.62 MB/s fsyncs: 0.00/s response time: 0.232ms (95%) | |
| [ 18s] reads: 0.00 MB/s writes: 35.75 MB/s fsyncs: 0.00/s response time: 0.228ms (95%) | |
| [ 20s] reads: 0.00 MB/s writes: 35.42 MB/s fsyncs: 0.00/s response time: 0.233ms (95%) | |
| [ 22s] reads: 0.00 MB/s writes: 34.63 MB/s fsyncs: 0.00/s response time: 0.233ms (95%) | |
| [ 24s] reads: 0.00 MB/s writes: 34.83 MB/s fsyncs: 0.00/s response time: 0.230ms (95%) | |
| [ 26s] reads: 0.00 MB/s writes: 36.84 MB/s fsyncs: 0.00/s response time: 0.229ms (95%) | |
| [ 28s] reads: 0.00 MB/s writes: 36.15 MB/s fsyncs: 0.00/s response time: 0.232ms (95%) | |
| Operations performed: 0 reads, 284087 writes, 0 Other = 284087 Total | |
| Read 0b Written 1.0837Gb Total transferred 1.0837Gb (36.99Mb/sec) | |
| 9469.55 Requests/sec executed | |
| The command line used was: sysbench --test=fileio --max-time=30 | |
| --max-requests=10000000 --file-num=1 --file-extra-flags=direct | |
| --file-total-size=8G --file-block-size=4k --file-fsync-all=off | |
| --file-fsync-freq=0 --file-fsync-mode=fdatasync --num-threads=1 | |
| --file-test-mode=ag4 --report-interval=2 run | |
| I have not yet uploaded my patched version of the development sysbench, but I'm | |
| planning to do so, and I'd be really interested if someone could run it on a | |
| really high-end storage system. |