add total time field to disk_io_counters()

[joemiller]

the total-time field is necessary to calculate a util% metric (0 - 100%) similar to what iostat -x prints.

[joemiller]

fixed the failing test

[joemiller]

ping

[giampaolo]

I don't understand the usefulness of this. What's the number returned by the kernel? Isn't it supposed to just be 'read_time' + 'write_time'? If that's the case then it is not necessary to complicate psutil API. If it's not the case then we need to know what that numbers refers to exactly (and whether we can extract it on other platforms as well, not only Linux).

PS: sorry for the late reply but I've been travelling a lot lately and didn't have time to work on my personal projects

[joemiller]

It's been a while since I thought about this so I need to try to recall the details.

Basically though, I could not find a useful way to get disk utilization (0-100%) with read_time + write_time. Only tot_time seemed to be useful to get anywhere near the %util as reported by iostat.

Here is an example of how different the numbers are as well. The tot_time is much different than read+write_times. I'm not sure if there is some HZ/tick conversion happening on one but not the other. I was not able to back-in to getting tot_time from read+write with anything that I tried when I wrote this patch.

>>> import psutil
>>> psutil.disk_io_counters()
sdiskio(read_count=4747007, write_count=111142907, read_bytes=43785395200, write_bytes=2134750146560, read_time=651313, write_time=573974127, tot_time=15754343)
>>> 651313 + 573974127
574625440
>>> 574625440 - 15754343
558871097

[joemiller]

Here's the complete func I use to calculate a continuous util%, for additional context.

class DiskIopsUsage(Metric):
    """Get disk device utilization. Similar to the util% output by `iostat -x`.
        Returns a float between 0 - 100%.

        The utilization of the busiest disk is reported. If there are 3 disks,
        with util% of 10%, 20%, and 90%, the value returned will be 90.

        @TODO: calc the average utilization across all disks? If so, filter out dummy disks.
        @TODO: or, allow user to specify a specific disk.
    """
    name = 'disk_iops'
    def __init__(self, **kwargs):
        super(DiskIopsUsage, self).__init__(**kwargs)

        self.prev = None
        self.prev_ts = None
        self.get()  # initialize self.prev

    def get(self):
        utilization = 0.0
        new = psutil.disk_io_counters()
        new_ts = time.time()
        if self.prev:
            tick_diff = float((new.tot_time - self.prev.tot_time)) / 1000  # ms to seconds
            time_diff = new_ts - self.prev_ts
            utilization = (tick_diff / time_diff) * 100.0
        self.prev = new
        self.prev_ts = new_ts
        return utilization

[joemiller]

Here is a small test that illustrates what I was seeing when originally writing this PR:

import psutil
import time

prev = None
prev_ts = None

while True:
    utilization_tot = 0.0
    utilization_readwrite = 0.0
    new = psutil.disk_io_counters()
    new_ts = time.time()
    print new
    if prev:
        tick_diff = float((new.tot_time - prev.tot_time)) / 1000  # ms to seconds
        time_diff = new_ts - prev_ts
        utilization_tot = (tick_diff / time_diff) * 100.0

        tick_diff = float(((new.read_time + new.write_time) - (prev.read_time + prev.write_time))) / 1000  # ms to seconds
        time_diff = new_ts - prev_ts
        utilization_readwrite = (tick_diff / time_diff) * 100.0
    prev = new
    prev_ts = new_ts
    print utilization_tot
    print utilization_readwrite
    time.sleep(1)

Run test.py in one terminal, and start a dd in another: eg: dd if=/dev/zero of=/tmp/test.file bs=1M count=5000

using tot_time I am able to get near the 100% utilization I expect and that shows in iostat, but the numbers are far too high if I use read+write_time. There may be a time unit or tick conversion I am missing here.

$ venv/bin/python test.py
sdiskio(read_count=4747414, write_count=111365799, read_bytes=43790232576, write_bytes=2143709179904, read_time=651886, write_time=579435198, tot_time=15797308)
0.0
0.0
sdiskio(read_count=4747420, write_count=111365800, read_bytes=43790269440, write_bytes=2143709184000, read_time=651889, write_time=579435198, tot_time=15797311)
0.299530669318
0.299530669318
sdiskio(read_count=4747421, write_count=111365802, read_bytes=43790273536, write_bytes=2143709192192, read_time=651890, write_time=579435198, tot_time=15797312)
0.0997764826225
0.0997764826225
sdiskio(read_count=4747421, write_count=111365802, read_bytes=43790273536, write_bytes=2143709192192, read_time=651890, write_time=579435198, tot_time=15797312)
0.0
0.0
sdiskio(read_count=4747422, write_count=111367555, read_bytes=43790277632, write_bytes=2143784177664, read_time=651891, write_time=579469123, tot_time=15797571)
25.8694264538
3388.59521959
sdiskio(read_count=4747424, write_count=111374199, read_bytes=43790285824, write_bytes=2144071692288, read_time=651899, write_time=579608839, tot_time=15798641)
106.87623742
13956.238689
sdiskio(read_count=4747448, write_count=111379854, read_bytes=43791195136, write_bytes=2144323678208, read_time=652050, write_time=579753018, tot_time=15799664)
102.150146805
14411.8579555
sdiskio(read_count=4747516, write_count=111385787, read_bytes=43793779712, write_bytes=2144589328384, read_time=652551, write_time=579899984, tot_time=15800685)
101.893967548
14716.9419318
sdiskio(read_count=4747541, write_count=111391266, read_bytes=43794770944, write_bytes=2144834965504, read_time=653032, write_time=580037460, tot_time=15801687)
99.983337461
13765.869547
sdiskio(read_count=4747573, write_count=111395838, read_bytes=43796077568, write_bytes=2145039990784, read_time=653353, write_time=580177890, tot_time=15802693)
100.416443236
14049.4182921
sdiskio(read_count=4747606, write_count=111400850, read_bytes=43797314560, write_bytes=2145257091072, read_time=653804, write_time=580335775, tot_time=15803750)
105.593231179
15817.6062932
sdiskio(read_count=4747671, write_count=111404901, read_bytes=43799804928, write_bytes=2145436815360, read_time=654312, write_time=580477924, tot_time=15804780)
102.845840502
14244.3486102
^CTraceback (most recent call last):

[fasionchan]

I want this feature as well. IO UTIL is an important metric I need to collection. Field name tot_time is not so specified, I suggest to rename it to busy_time or io_time.

So why read_time + write_time >= busy_time? This is because disk devices can operate several read/write operations parallelly. That is, device is busy no matter how many operations it is operating. However, write_time will be n times if n operations.

IO UTIL metric, measures how much time the device is busy. Solution on Linux is just show above. That is similar solutions on other platforms. For instance, on FreeBSD, we can use busy_time field in devstat struct(sys/devicestat.h)。

Hope busy_time to been included. I may share some of my code.

[giampaolo]

Agreed this is necessary. We first need to figure out on what platforms this is retrievable though, especially OSX.

[fasionchan]

All right. I will help to find out solutions for other platforms.

[giampaolo]

FreeBSD implementation: #769
Linux: edd6c22

[giampaolo]

It appears this info is available only on Linux and BSD. I only added FreeBSD implementation though because on OpenBSD and NetBSD I couldn't find a way to get the right metrics. Closing this out.