Spaces at the end of file and directory names are silently ignored on windows

[timhoffm]

Bug report

Bug description:

>>> import os
>>> os.path.exists("test")
False
>>> os.path.exists("test ")
False
>>> os.mkdir("test ")
>>> os.path.exists("test")
True
>>> os.path.exists("test ")
True

The explorer also shows the created directory as test, i.e. without trailing spaces.

The win32 docs say about trailing spaces

Do not end a file or directory name with a space or a period. Although the underlying file system may support such names, the Windows shell and user interface does not.

One could discuss whether this is user-responsibility, but silently dropping trailing spaces feels wrong. Either create the directory (though not recommended), or raise if there is a space at the end.

Note: Behaviour gets even more weird when combining trailing spaces with alternate data streams, e.g. "test : foo" (matplotlib/matplotlib#27748), which brought me here. But before going into this more difficult discussion, I'd like to learn the general position on handling trailings spaces in windows filenames.

CPython versions tested on:

3.12

Operating systems tested on:

Windows

[zooba]

This is normal Windows behaviour. Every Windows API that normalises paths (most of them, though most allow using the \\?\ prefix to bypass normalisation) will strip trailing dots and spaces from path segments.

We won't make any change to support it by default, but we do try to make sure that if you use a \\?\ prefix then anything we might do with the path will work. But it's the caller's responsibility to get it right - we don't try to implement POSIX rules around Windows paths.

[zooba]

At its most general, we pass path strings directly to Windows APIs. The best way to get Python to handle these in a particular way is to get Windows to handle them in a particular way.

[terryjreedy]

Should the Using doc Windows section include the pair of sentences quoted above?

[zooba]

Yeah, somewhere we ought to write down our policy on this. I haven't figured out where best to put it though... seems kinda like Dev Guide is the place?

[timhoffm]

Yeah, somewhere we ought to write down our policy on this. I haven't figured out where best to put it though... seems kinda like Dev Guide is the place?

Isn't this rather user-facing? I think adding to https://docs.python.org/3/library/os.path.html somewhere in the vicinity of this note would be helpful:

Something like:

These modules mimic OS specific behavior (often by calling system APIs) and therefore may differ slightly in behavior across systems. For example, Windows typically trims trailing spaces from file and directory names.

[zooba]

The policy on how we choose what to control vs what to pass through is specific to contributors.

We should also document somewhere that, at least for Windows, we prefer to expose the OS behaviour "warts and all". I'm not sure exactly how we approach other platforms.

I also don't like listing specific behaviours, because they vary more often than you'd expect. Your "for example" is fine, but let's not try to make it an exhaustive list.

It probably also belongs in the os module docs, along with the other platform-specific notes near the top. (It's unfortunate that os.path also accesses the filesystem and so suffers from OS-specific behaviours, but properly the relevant module is os.)