ntpath.relpath behaves differently on Windows with trailing spaces

[jaraco]

BPO	41509
Nosy	@pfmoore, @jaraco, @tjguk, @zware, @eryksun, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-08-08.17:34:37.931>
labels = ['3.8', 'OS-windows']
title = 'ntpath.relpath behaves differently on Windows with trailing spaces'
updated_at = <Date 2020-08-08.19:15:05.636>
user = 'https://github.com/jaraco'

bugs.python.org fields:

activity = <Date 2020-08-08.19:15:05.636>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows']
creation = <Date 2020-08-08.17:34:37.931>
creator = 'jaraco'
dependencies = []
files = []
hgrepos = []
issue_num = 41509
keywords = []
message_count = 2.0
messages = ['375053', '375058']
nosy_count = 6.0
nosy_names = ['paul.moore', 'jaraco', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue41509'
versions = ['Python 3.8']

[jaraco]

On Windows:

Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ntpath 
>>> ntpath.relpath('foo ', 'foo')
'.'

On macOS:

Python 3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ntpath
>>> ntpath.relpath('foo ', 'foo')
'..\\foo '

I stumbled into this issue when troubleshooting an issue in a Setuptools PR.

I suspect the Windows version is using some API that strips whitespace from the filename before performing a relative path. However, when using relpath to detect characters after a common path, stripping the whitespace can cause problems.

I wouldn't expect Windows to be performing normalization of paths in relpath, but it seems it does. If this behavior is by design and has a good reason, that behavior should be mirrored in the non-Windows implementation.

[eryksun]

I suspect the Windows version is using some API that strips whitespace
from the filename before performing a relative path.

When normalizing a path, the Windows API strips trailing dots and spaces from the final component. Apparently it also strips a single trailing dot from other components if the dot is preceded by any character other than a dot. For example:

    >>> nt._getfullpathname('foo../bar./bam. . .')
    'C:\\foo..\\bar\\bam'

The way to create or open a file or directory with a name that ends with a dot or space is to prefix the path with exactly "\\\\?\\". However, it is strongly advised to never create a file or directory with such an abnormal name. Note that this prefix only affects using a path in an open/create context. It doesn't prevent an explicit normalization, such as the following:

    >>> nt._getfullpathname('\\\\?\\C:/foo../bar./bam. . .')
    '\\\\?\\C:\\foo..\\bar\\bam'

I wouldn't expect Windows to be performing normalization of paths in
relpath, but it seems it does.

ntpath.relpath calls ntpath.abspath, which calls nt._getfullpathname in Windows. On POSIX systems, ntpath.abspath is ntpath._abspath_fallback. The latter and ntpath.normpath need to be improved to better implement Windows path normalization.

normpath should not exclude paths that start with '\\\\.\\' and '\\\\?\\', and it should trim trailing spaces and dots from component names to match how Windows normalizes a path.

_abspath_fallback should default to the root directory for drive relative paths (e.g. "C:foo" -> "C:\\foo"). It should also substitute a device path if the final component in a relative or drive-letter path (but not a UNC path or device path) matches a known DOS device name. Matching a DOS device name ignores everything after a dot or colon that's preceded by zero or more spaces (e.g. "con .anything" -> "\\\\.\\con" and "nul:anything" -> "\\\\.\\nul").