symlinks incorrectly resolved on POSIX platforms

[swarecki]

BPO	6975
Nosy	@birkenfeld, @pitrou, @larryhastings, @benjaminp, @ezio-melotti, @bitdancer, @hynek, @serhiy-storchaka, @o11c
Files	issue6975.patch: Patch provided for issue 6975 issue6975_py3.patch: Patch ported to python3 posix_realpath.patch posix_realpath_2.patch pathlib_resolve_test.py: Test pathlib for recursive symlinks

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2013-02-18.11:09:31.077>
created_at = <Date 2009-09-23.05:55:12.224>
labels = ['type-bug', 'library', 'release-blocker']
title = 'symlinks incorrectly resolved on POSIX platforms'
updated_at = <Date 2013-02-20.20:33:41.977>
user = 'https://bugs.python.org/swarecki'

bugs.python.org fields:

activity = <Date 2013-02-20.20:33:41.977>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2013-02-18.11:09:31.077>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2009-09-23.05:55:12.224>
creator = 'swarecki'
dependencies = []
files = ['15150', '27668', '27698', '28672', '28720']
hgrepos = []
issue_num = 6975
keywords = ['patch']
message_count = 29.0
messages = ['93027', '93029', '94149', '94179', '94199', '94209', '111082', '164111', '173590', '173696', '178613', '179467', '179469', '179472', '179479', '179573', '179886', '179920', '179974', '181782', '182296', '182297', '182309', '182311', '182315', '182427', '182537', '182540', '182549']
nosy_count = 14.0
nosy_names = ['georg.brandl', 'pitrou', 'larry', 'benjamin.peterson', 'swarecki', 'ezio.melotti', 'Arfrever', 'r.david.murray', 'BreamoreBoy', 'python-dev', 'hynek', 'serhiy.storchaka', 'o11c', 'tzimmo']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue6975'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']

[swarecki]

Hi

The behavior of os.realpath function on Linux is different from the one
presented by the system.
Although the path (pointing to the linked dir) is correct
and is recognized by os.path.exists once it is resolved
by the realpath() it does not exist - at the end of the
resolved path you will see the additional parent directory name.
you will see
..... /two/two at the end of the path.

Below is the code that shows the issue.

def linkGen():
  import os
  print os.getcwd()
  test = "./test"
  os.system( "rm -rf " + test  )  
  #os.mkdir( test )
  os.makedirs( test + "/one" )
  os.makedirs( test + "/two" )
  os.symlink( "../two", test + "/two/this_dir" )
  os.symlink( "../two/this_dir/this_dir/this_dir/this_dir/", test + 
"/one/that_dir" )
  print test + "/one/that_dir", "EXISTS?",   
  print os.path.exists( test + "/one/that_dir" )
  print os.path.realpath( test + "/one/that_dir" ), "EXISTS?", 
  print os.path.exists( os.path.realpath( test + "/one/that_dir" ) )
  os.system( "ls -l " + test + "/one/that_dir" )
  # the line above will show that system can see the directory just fine

linkGen()

[swarecki]

Hi
I meant of course
os.path.realpath()
Sylwester

[markon]

I've provided a patch.

I've also added a new test, and it passes.

[markon]

I've just found a similar bug:
http://bugs.python.org/issue1646838

However, does os.path.realpath's behavior have to be as the one
specified by the POSIX standard (
http://www.kernel.org/doc/man-pages/online/pages/man3/realpath.3.html )?
If we wanted to follow the standard, we should break the
retro-compatibility, since we should raise an exception in the case the
path passed as argument doesn't exist.

[swarecki]

Marco,

Thanks for looking deeper into it.
I would like to check the solution ASAP.
We have really crazy dir structure which can catch a lot of
the unexpected problems with paths, links, circular links etc.

Should I expect the new version to generate the exception
as you suggested?

Best Regards,
Sywlester Warecki

Marco Buccini wrote:

Marco Buccini <marcusbu@gmail.com> added the comment:

I've just found a similar bug:
http://bugs.python.org/issue1646838

However, does os.path.realpath's behavior have to be as the one
specified by the POSIX standard (
http://www.kernel.org/doc/man-pages/online/pages/man3/realpath.3.html )?
If we wanted to follow the standard, we should break the
retro-compatibility, since we should raise an exception in the case the
path passed as argument doesn't exist.

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue6975\>

---

[markon]

Thanks for looking deeper into it.
I would like to check the solution ASAP.
We have really crazy dir structure which can catch a lot of
the unexpected problems with paths, links, circular links etc.

Oh well, so you can see if the patch works correctly on all paths :)

Should I expect the new version to generate the exception
as you suggested?

Unfortunately I'm not God, so I cannot change the "code" of the current
implementation of os.path.realpath (raising an OSError exception)
without breaking any existing code.
So, consider using the current implementation of os.path.realpath ;)
If you experience other strange behaviors, then open a new bug report.

Bye!

[BreamoreBoy]

swarecki or markon, are either of you interested in following up this issue?

[o11c]

After filing a duplicate, bpo-15196, I analyzed this:

What happens:
test/one/that_dir
test/one/../two/this_dir/this_dir/this_dir/this_dir
test/two/this_dir/this_dir/this_dir/this_dir
test/two/this_dir/this_dir/this_dir/../two
test/two/this_dir/this_dir/two

What should happen:
test/one/that_dir
test/two/this_dir/this_dir/this_dir/this_dir
test/two/../two/this_dir/this_dir/this_dir
test/two/this_dir/this_dir/this_dir
test/two/../two/this_dir/this_dir
test/two/this_dir/this_dir
test/two/../two/this_dir
test/two/this_dir
test/two/../two
test/two

[tzimmo]

Tried it out and it seemed to fix the issue.
Ported the patch to python3.

[serhiy-storchaka]

Previous patches contain redundant code, are not protected from all symlink loops (link -> link/x; dir/link -> ../dir/link) and when a loop is detected the returned result differs from the current one.

The proposed patch solves this issue and some others. Added many new tests.

The test for this issue was simplified:

dir/self -> ../dir
dir/link -> self/self/self
resolve dir/link