shutil.disk_usage - Lacking documentation

[LordAntonHvornum]

BPO	38825
Nosy	@giampaolo, @tarekziade, @tylercrompton, @tirkarthi, @normanlorrain

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-11-16.22:19:58.065>
labels = ['easy', '3.9', '3.10', '3.11', 'type-feature', 'docs']
title = 'shutil.disk_usage - Lacking documentation'
updated_at = <Date 2021-07-12.21:07:41.352>
user = 'https://bugs.python.org/LordAntonHvornum'

bugs.python.org fields:

activity = <Date 2021-07-12.21:07:41.352>
actor = 'Norman Lorrain'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2019-11-16.22:19:58.065>
creator = 'Lord Anton Hvornum'
dependencies = []
files = []
hgrepos = []
issue_num = 38825
keywords = ['easy']
message_count = 6.0
messages = ['356771', '356788', '356796', '396520', '397267', '397357']
nosy_count = 7.0
nosy_names = ['giampaolo.rodola', 'tarek', 'docs@python', 'tylercrompton', 'Lord Anton Hvornum', 'xtreak', 'Norman Lorrain']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38825'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• gh-83006: Document behavior of shutil.disk_usage for non-mounted filesystems on Unix #107031
• [3.12] gh-83006: Document behavior of shutil.disk_usage for non-mounted filesystems on Unix (GH-107031) #107047
• [3.11] gh-83006: Document behavior of shutil.disk_usage for non-mounted filesystems on Unix (GH-107031) #107048

[LordAntonHvornum]

https://docs.python.org/3.8/library/shutil.html#shutil.disk_usage

There's no mention that this helper function simply calls os.statvfs() in the background. Something that is quite troubling when you're trying to get disk_usage (or disk-information) about a non-mounted drive.

It's clear as day if you see the code: https://github.com/python/cpython/blob/master/Lib/shutil.py#L1249

But if you're a novice user, this will puzzle your brain.
Because the end result will be that disk_usage() returns the same information for all these cases:

shutil.disk_usage('/dev/sda')
shutil.disk_usage('/dev/sdb')
shutil.disk_usage('/dev/sdc')

Which translates to:

os.statvfs('/dev/sd?')'

--

All I'm asking, is that we add a little note stating:
"On mounted filesystems" or a reference to os.statvfs() where it's clearly stated that it runs on mounted filesystems.

Thanks in advance.

[tirkarthi]

I modified psutil to shutil since I feel it's a typo. Please revert back if it's incorrect.

[LordAntonHvornum]

xtreak: You are correct, that was a typo. My apologies.

[normanlorrain]

This issue is a bit dated, but here goes:

On Unix, the function statvfs() is called. path should be a path on a mounted filesystem (it should not be a device).

On Windows, the function GetDiskFreeSpaceExW() is called. path is any directory on a disk volume.

Looking through the rest of the page, no other functions get into such specifics; maybe this is outside the scope of the document. Maybe the issue should be closed.

[tylercrompton]

Not even the kernel knows how much space is available on a nonmounted partition. It doesn't know much beyond the fact that it exists and where it exists. There exist tools that can analyze nonmounted partitions, but these will vary by filesystem and operating system. If someone wants to implement solutions for the most common filesystem-OS combinations, then that might be helpful.

But considering that no one has done that or even asked for it yet, I'm not sure that doing so is worth the effort. So I agree that documenting it is the best approach. It should be documented such that the limitations are mentioned, not necessarily how it's implemented. If explaining which system calls are used helps explain the limitations, then so be it. Additionally, as far as I'm concerned, there's no reason to restrict other Python implementations from implementing functionality for nonmounted disks. So the limitations should be described as being CPython-specific, akin to what is done for readline.set_auto_history.

[normanlorrain]

Something like this:

.. impl-detail::
Where applicable (e.g. Unix) *path* must point to somewhere on a
**mounted** filesystem partition.

On Sun, 11 Jul 2021 at 16:18, Tyler Crompton <report@bugs.python.org> wrote:

Tyler Crompton tyler@tylercrompton.com added the comment:

Not even the kernel knows how much space is available on a nonmounted
partition. It doesn't know much beyond the fact that it exists and where it
exists. There exist tools that can analyze nonmounted partitions, but these
will vary by filesystem and operating system. If someone wants to implement
solutions for the most common filesystem-OS combinations, then that might
be helpful.

But considering that no one has done that or even asked for it yet, I'm
not sure that doing so is worth the effort. So I agree that documenting it
is the best approach. It should be documented such that the limitations are
mentioned, not necessarily how it's implemented. If explaining which system
calls are used helps explain the limitations, then so be it. Additionally,
as far as I'm concerned, there's no reason to restrict other Python
implementations from implementing functionality for nonmounted disks. So
the limitations should be described as being CPython-specific, akin to what
is done for readline.set_auto_history.

----------
nosy: +tylercrompton

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue38825\>

---

[hauntsaninja]

Looks like behaviour is now documented, thanks all!