Report merged reads and writes per disk #756

Closed
bobrik opened this Issue Feb 4, 2016 · 7 comments

Comments

Projects
None yet
2 participants
@bobrik

bobrik commented Feb 4, 2016

I have spinning disks and during raid-10 rebuild I noticed this:

image

With a theoretical maximum of ~200 IOPS having 2000+ IOPS looks a bit surreal. I dug a bit and found out that these are merged reads and the do not mean much for monitoring, since you don't measure how hard you hit your disks.

I suggest reporting merged reads and writes:

It also looks like time spent in IO for spinning disks could be more than 1 second per 1 second of real time. This also looks like a science fiction, you can't have one disk head in many locations at once, that's witchcraft^W quantum computing.

I suggest also reporting 14th column from the document above, that seems to hold the right metric.

@giampaolo

This comment has been minimized.

Show comment
Hide comment
@giampaolo

giampaolo Feb 4, 2016

Owner

So you're suggesting to add 5 - reads merged and 9 - writes merged.
What about 14 - weighted time spent doing I/Os (ms)? Why do you think it would be useful? Is it what is it being suggested in #523?

Owner

giampaolo commented Feb 4, 2016

So you're suggesting to add 5 - reads merged and 9 - writes merged.
What about 14 - weighted time spent doing I/Os (ms)? Why do you think it would be useful? Is it what is it being suggested in #523?

@bobrik

This comment has been minimized.

Show comment
Hide comment
@bobrik

bobrik Feb 4, 2016

Yes, let's add 5 and 9.

#523 seems to do what I want to calculate %util.

bobrik commented Feb 4, 2016

Yes, let's add 5 and 9.

#523 seems to do what I want to calculate %util.

@giampaolo

This comment has been minimized.

Show comment
Hide comment
@giampaolo

giampaolo Feb 13, 2016

Owner

Why do you need 5 and 9? They definitively exist on Linux only and I would avoid them if possible. Isn't 10 ms_spent_doing_io enough?

Owner

giampaolo commented Feb 13, 2016

Why do you need 5 and 9? They definitively exist on Linux only and I would avoid them if possible. Isn't 10 ms_spent_doing_io enough?

giampaolo added a commit that referenced this issue Feb 13, 2016

@bobrik

This comment has been minimized.

Show comment
Hide comment
@bobrik

bobrik Feb 14, 2016

I need them because I care about actual writes on disk, not about write() syscalls. With spinning disks these are even more important.

Does FreeBSD (or other systems) provide you with raw or merged writes?

bobrik commented Feb 14, 2016

I need them because I care about actual writes on disk, not about write() syscalls. With spinning disks these are even more important.

Does FreeBSD (or other systems) provide you with raw or merged writes?

@giampaolo

This comment has been minimized.

Show comment
Hide comment
@giampaolo

giampaolo Feb 14, 2016

Owner

It is not possible to determine merged r/w on BSD and Windows (didn't check OSX and Solaris yet). They either don't have this capability of merging 2 adjacent r/w calls into one or the kernel does not collect/expose this info.
OK, I'm convinced this is needed, even if it's Linux only.

Owner

giampaolo commented Feb 14, 2016

It is not possible to determine merged r/w on BSD and Windows (didn't check OSX and Solaris yet). They either don't have this capability of merging 2 adjacent r/w calls into one or the kernel does not collect/expose this info.
OK, I'm convinced this is needed, even if it's Linux only.

@giampaolo giampaolo closed this in 8cc496d Feb 14, 2016

@giampaolo

This comment has been minimized.

Show comment
Hide comment
@giampaolo

giampaolo Feb 14, 2016

Owner

As for the other stats:
12 - I/Os currently in progress: I'm gonna add it as part of #523.

As for:

13 - time spent doing I/Os (ms)
14 - weighted time spent doing I/Os (ms)

...I may consider adding them if we can come up with some real-world app use case. Right now I'm not sure (don't understand) how can they be useful.

Owner

giampaolo commented Feb 14, 2016

As for the other stats:
12 - I/Os currently in progress: I'm gonna add it as part of #523.

As for:

13 - time spent doing I/Os (ms)
14 - weighted time spent doing I/Os (ms)

...I may consider adding them if we can come up with some real-world app use case. Right now I'm not sure (don't understand) how can they be useful.

@bobrik

This comment has been minimized.

Show comment
Hide comment
@bobrik

bobrik Feb 14, 2016

@giampaolo real world example for 14 is util% from iostat, #523 aims to provide that.

bobrik commented Feb 14, 2016

@giampaolo real world example for 14 is util% from iostat, #523 aims to provide that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment