Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pathlib.Path.resolve() returns path with double slash when resolving a relative path in root directory #77841

Closed
QbLearningPython mannequin opened this issue May 27, 2018 · 10 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@QbLearningPython
Copy link
Mannequin

QbLearningPython mannequin commented May 27, 2018

BPO 33660
Nosy @pitrou, @ambv, @serhiy-storchaka, @timofurrer, @corona10, @miss-islington, @killerrex, @mbarkhau
PRs
  • bpo-33660: Fix PosixPath to resolve a relative path on root #7666
  • bpo-33660: Fix corner-case in PosixPath.resolve resulting in "//path" #21971
  • bpo-33660: Fix PosixPath to resolve a relative path on root #21972
  • [3.9] bpo-33660: Fix PosixPath to resolve a relative path on root #21974
  • [3.8] bpo-33660: Fix PosixPath to resolve a relative path on root #21975
  • [3.8] bpo-33660: Fix PosixPath to resolve a relative path on root #23750
  • [3.9] bpo-33660: Fix PosixPath to resolve a relative path on root #23751
  • Files
  • plbug.py: Test script of the pathlib behaviour
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2020-08-27.00:52:52.993>
    created_at = <Date 2018-05-27.10:58:32.972>
    labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
    title = 'pathlib.Path.resolve() returns path with double slash when resolving a relative path in root directory'
    updated_at = <Date 2020-12-13.00:59:01.487>
    user = 'https://bugs.python.org/QbLearningPython'

    bugs.python.org fields:

    activity = <Date 2020-12-13.00:59:01.487>
    actor = 'miss-islington'
    assignee = 'none'
    closed = True
    closed_date = <Date 2020-08-27.00:52:52.993>
    closer = 'lukasz.langa'
    components = ['Library (Lib)']
    creation = <Date 2018-05-27.10:58:32.972>
    creator = 'QbLearningPython'
    dependencies = []
    files = ['47630']
    hgrepos = []
    issue_num = 33660
    keywords = ['patch']
    message_count = 10.0
    messages = ['317790', '318452', '319395', '320837', '375968', '375969', '375971', '375973', '375974', '375978']
    nosy_count = 9.0
    nosy_names = ['pitrou', 'lukasz.langa', 'serhiy.storchaka', 'tuxtimo', 'corona10', 'QbLearningPython', 'miss-islington', 'killerrex', 'mbarkhau']
    pr_nums = ['7666', '21971', '21972', '21974', '21975', '23750', '23751']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue33660'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @QbLearningPython
    Copy link
    Mannequin Author

    QbLearningPython mannequin commented May 27, 2018

    I have recently found a weird behaviour while trying to resolve a relative path located on the root directory on a macOs.

    I tried to resolve a Path('spam') and the interpreter answered PosixPath('//spam') —double slash for root— instead of (my) expected PosixPath('/spam').

    I think that this is a bug.

    I ran the interpreter from root directory (cd /; python). Once running the interpreter, this is what I did:

    >>> import pathlib
    >>> pathlib.Path.cwd()
    PosixPath('/')
    # since the interpreter has been launched from root
    >>> p = pathlib.Path('spam')
    >>> p
    PosixPath('spam')
    # just for checking
    >>> p.resolve()
    PosixPath('//spam')
    # beware of double slash instead of single slash

    I also checked the behaviour of Path.resolve() in a non-root directory (in my case launching the interpreter from /Applications).

    >>> import pathlib
    >>> pathlib.Path.cwd()
    PosixPath('/Applications')
    >>> p = pathlib.Path('eggs')
    >>> p
    PosixPath('eggs')
    >>> p.resolve()
    PosixPath('/Applications/eggs')
    # just one slash as root in this case (as should be)

    So it seems that double slashes just appear while resolving relative paths in the root directory.

    More examples are:
    	
    >>> pathlib.Path('spam/egg').resolve()
    PosixPath('//spam/egg')
    >>> pathlib.Path('./spam').resolve()
    PosixPath('//spam')
    >>> pathlib.Path('./spam/egg').resolve()
    PosixPath('//spam/egg')

    but

    >>> pathlib.Path('').resolve()
    PosixPath('/')
    >>> pathlib.Path('.').resolve()
    PosixPath('/')

    Intriguingly,

    >>> pathlib.Path('spam').resolve().resolve()
    PosixPath('/spam')
    # 'spam'.resolve = '//spam'
    # '//spam'.resolve = '/spam'!!!
    >>> pathlib.Path('//spam').resolve()
    PosixPath('/spam')

    I have found the same behaviour in several Python versions:

    Python 3.6.5 (default, May 15 2018, 08:20:57)
    [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin

    Python 3.4.8 (default, Mar 29 2018, 16:18:25)
    [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin

    Python 3.5.5 (default, Mar 29 2018, 16:22:58)
    [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin

    Python 3.7.0b4 (default, May 4 2018, 22:01:49)
    [Clang 9.1.0 (clang-902.0.39.1)] on darwin

    All running on: macOs High Sierra 10.13.4 (17E202)

    There is also confirmation of same issue on Ubuntu 16.04 (Python 3.5.2) and Opensuse tumbleweed (Python 3.6.5)

    I have searched for some information on this issue but I did not found anything useful.

    Python docs (https://docs.python.org/3/library/pathlib.html) talks about "UNC shares" but this is not the case (in using a macOs HFS+ filesystem).

    PEP-428 (https://www.python.org/dev/peps/pep-0428/) says:

    Multiple leading slashes are treated differently depending on the path flavour. They are always retained on Windows paths (because of the UNC notation):
    
        >>> PureWindowsPath('//some/path')
        PureWindowsPath('//some/path/')
    On POSIX, they are collapsed except if there are exactly two leading slashes, which is a special case in the POSIX specification on pathname resolution [8] (this is also necessary for Cygwin compatibility):
    
        >>> PurePosixPath('///some/path')
        PurePosixPath('/some/path')
        >>> PurePosixPath('//some/path')
        PurePosixPath('//some/path')

    I do not think that this is related to the aforementioned issue.

    However, I also checked the POSIX specification link (http://pubs.opengroup.org/onlinepubs/009...#tag_04_11) and found:

    A pathname that begins with two successive slashes may be interpreted in an implementation-defined manner, although more than two leading slashes shall be treated as a single slash.
    

    I do not really think that this can cause a double slashes while resolving a relative path on macOs.

    So, I think that this issue could be a real bug in pathlib.Path.resolve() method. Specifically on POSIX flavour.

    A user of Python Forum (killerrex) and I have traced the bugs to Lib/pathlib.py:319 in the Python 3.6 repository https://github.com/python/cpython/blob/3...pathlib.py.

    Specifically, in line 319:

    newpath = path + sep + name
    

    For pathlib.Path('spam').resolve() in the root directory, newpath is '//spam' since:

    path is '/'
    sep is '/'
    name is 'spam'
    

    killerrex has suggested two solutions:

    1. from line 345

      base = '' if path.is_absolute() else os.getcwd()
      if base == sep:
      base = ''
      return _resolve(base, str(path)) or sep

    2. from line 319:

        if path.endswith(sep):
            newpath = path + name
        else:
            newpath = path + sep + name

    Thank you.

    @QbLearningPython QbLearningPython mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels May 27, 2018
    @terryjreedy terryjreedy added the 3.8 only security fixes label Jun 1, 2018
    @killerrex
    Copy link
    Mannequin

    killerrex mannequin commented Jun 1, 2018

    Script to reproduce:

    import os
    import pathlib
    
    # Change to the Root directory
    os.chdir('/')
    
    # Create a relative path object.
    p = pathlib.Path('spam')
    print(p.resolve())

    Expected output:
    /span

    Incorrect output
    //span

    @corona10
    Copy link
    Member

    For reviewers,

    I submitted a patch PR 7666.
    Please take a look

    @corona10
    Copy link
    Member

    corona10 commented Jul 1, 2018

    Please review my pull request.
    This PR is pending for 19 days.

    @mbarkhau
    Copy link
    Mannequin

    mbarkhau mannequin commented Aug 26, 2020

    This issue cropped up recently in the black project: psf/black#1631

    It appears to me that PR 7666 was closed without being merged.

    I created #21971 before I had found this issue.

    @mbarkhau mbarkhau mannequin added 3.9 only security fixes 3.10 only security fixes labels Aug 26, 2020
    @ambv
    Copy link
    Contributor

    ambv commented Aug 27, 2020

    New changeset 94ad6c6 by Łukasz Langa (Dong-hee Na) in branch 'master':
    bpo-33660: Fix PosixPath to resolve a relative path on root
    94ad6c6

    @ambv
    Copy link
    Contributor

    ambv commented Aug 27, 2020

    New changeset 7475aa2 by Miss Islington (bot) in branch '3.8':
    bpo-33660: Fix PosixPath to resolve a relative path on root (GH-21975)
    7475aa2

    @ambv
    Copy link
    Contributor

    ambv commented Aug 27, 2020

    New changeset 211e4c6 by Miss Islington (bot) in branch '3.9':
    bpo-33660: Fix PosixPath to resolve a relative path on root (bpo-21974)
    211e4c6

    @ambv
    Copy link
    Contributor

    ambv commented Aug 27, 2020

    Sorry it took so long, this will get released in Python 3.8.6 and newer.

    @ambv ambv removed the 3.7 (EOL) end of life label Aug 27, 2020
    @ambv ambv closed this as completed Aug 27, 2020
    @corona10
    Copy link
    Member

    Thanks Łukasz for finanlizing this work :)

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants