8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win) #15525

bplb · 2023-08-31T22:45:49Z

Windows implementation of integrated pull request #15397. The test java/nio/file/Path/ToRealPath.java is also removed from the problem list.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win) (Bug - P4)

Reviewers

Daniel Jeliński (@djelinski - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/15525/head:pull/15525
$ git checkout pull/15525

Update a local copy of the PR:
$ git checkout pull/15525
$ git pull https://git.openjdk.org/jdk.git pull/15525/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 15525

View PR using the GUI difftool:
$ git pr show -t 15525

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/15525.diff

Webrev

Link to Webrev Comment

…../../" follows a link (win)

bridgekeeper · 2023-08-31T22:46:59Z

👋 Welcome back bpb! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2023-08-31T22:48:30Z

@bplb The following label will be automatically applied to this pull request:

nio

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2023-08-31T22:52:49Z

Webrevs

bridgekeeper · 2023-09-29T01:00:29Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2023-09-29T02:15:11Z

continue;

bridgekeeper · 2023-10-27T09:03:49Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bridgekeeper · 2023-12-12T21:26:41Z

@bplb This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.

bplb · 2023-12-12T21:39:17Z

/open

openjdk · 2023-12-12T21:39:48Z

@bplb This pull request is now open

bridgekeeper · 2024-01-10T00:37:32Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-01-10T01:59:57Z

continue;

bridgekeeper · 2024-02-07T03:08:56Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-02-07T17:02:11Z

continue;

openjdk · 2024-03-13T20:19:57Z

@bplb This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win)

Reviewed-by: djelinski

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 29 new commits pushed to the master branch:

5459518: 8339627: Cleanup Unsafe.setMemory intrinsic code
a4eb9a0: 8339918: Remove checks for outdated -t -tm -Xfuture -checksource -cs -noasyncgc options from the launcher
6be15c3: 8340012: [C2] assert(KlassEncodingMetaspaceMax > pd) failed: change encoding max if new encoding after 8338526
4b79063: 8339842: Open source several AWT focus tests - series 2
dc00eb8: 8338912: CDS: Segmented roots array
74add0e: 8340105: Expose BitMap::print_on in release builds
0e0f10f: 8340102: Move assert-only loop in OopMapSort::sort under debug macro
a0794e0: 8339639: Opensource few AWT PopupMenu tests
a8f143c: 8306679: com/sun/jdi/InterruptHangTest.java asserts with -Xcomp -Dmain.wrapper=Virtual options
c91fa27: 8339725: Concurrent GC crashed due to GetMethodDeclaringClass
... and 19 more: https://git.openjdk.org/jdk/compare/81ff91ef27a6a856ae2c453a9a9b8333b91da3ab...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

bridgekeeper · 2024-04-11T01:58:44Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-04-11T16:14:07Z

continue;

bridgekeeper · 2024-05-10T01:31:57Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-05-10T16:18:14Z

continue;

bridgekeeper · 2024-06-07T19:10:42Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-06-07T20:08:21Z

continue;

bridgekeeper · 2024-07-05T23:19:10Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-07-16T17:50:02Z

continue;

bridgekeeper · 2024-08-13T19:10:50Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-08-13T19:16:10Z

continue;

bridgekeeper · 2024-09-10T21:52:30Z

@bplb This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

bplb · 2024-09-10T21:53:30Z

continue;

djelinski · 2024-09-11T11:20:43Z

I don't think we want to do this.
On Windows, symbolic links to directories are almost identical to directories: you can have a symbolic link on the path to the working directory, and entering and exiting a symbolic link takes you back to the directory you were originally in. This behavior is observed in many Windows applications. If we change the handling of .. in paths containing symlinks, Java's interpretation of these paths will be inconsistent with other applications.

I think we should relax the ToRealPath test; instead of making assumptions on what toRealPath should return, the test could verify that a file created using a path containing symlink\..\..\file can be opened using the path returned by toRealPath. As long as toRealPath points to the same file, the exact path shouldn't matter.

bplb · 2024-09-11T23:49:15Z

Consider this case:

CWD = C:\Users\bpb\dev\bugs:

1. Create these directories, file, and link in the CWD:

file
dir\subdir
link -> dir\subdir
path = link\..\..\file

2. Invoke path.toRealPath:
2.1 mainline results

follow links     C:\Users\bpb\dev\file
no follow links  C:\Users\bpb\dev\file

2.2 PR results

follow links      C:\Users\bpb\dev\bugs\file
no follow links   C:\Users\bpb\dev\bugs\link\..\..\file

The results of the current mainline are incorrect as they locate a non-existent file. The results of the PR version both correctly locate the file.

djelinski · 2024-09-12T07:58:21Z

Well let me show you a counterexample:
Setup:

C:\>mkdir tmp\cwd\dir\subdir

C:\>cd tmp\cwd

C:\tmp\cwd>mklink /d link dir\subdir
symbolic link created for link <<===>> dir\subdir

C:\tmp\cwd>jshell.exe

Before your changes:

jshell> Path.of("link/../../file")
$1 ==> link\..\..\file

jshell> Files.createFile($1)
$2 ==> link\..\..\file

jshell> Files.exists($1)
$3 ==> true

jshell> Files.exists($1.toRealPath())
$4 ==> true

After your changes:

jshell> Path.of("link/../../file")
$1 ==> link\..\..\file

jshell> Files.createFile($1)
$2 ==> link\..\..\file

jshell> Files.exists($1)
$3 ==> true

jshell> Files.exists($1.toRealPath())
|  Exception java.nio.file.NoSuchFileException: C:\tmp\cwd\file
|        at WindowsException.translateToIOException (WindowsException.java:85)
|        at WindowsException.rethrowAsIOException (WindowsException.java:103)
|        at WindowsException.rethrowAsIOException (WindowsException.java:108)
|        at WindowsLinkSupport.collapseDots (WindowsLinkSupport.java:238)
|        at WindowsLinkSupport.getRealPath (WindowsLinkSupport.java:285)
|        at WindowsPath.toRealPath (WindowsPath.java:944)
|        at WindowsPath.toRealPath (WindowsPath.java:42)
|        at (#4:1)

path and path.toRealPath() are supposed to point to the same file, but as you can see, this does not happen here.

I guess you could adjust all the methods in Files class to be consistent with the new toRealPath implementation, but this would break consistency with other Windows apps, see for example:

C:\tmp\cwd>type file
The system cannot find the file specified.

C:\tmp\cwd>type link\..\..\file

C:\tmp\cwd>

Based on the above, I think the current toRealPath implementation on Windows is doing just fine, only the ToRealPath test needs to be fixed.

As far as I could tell, Path.toRealPath does not make any promises about what the resulting path should be. The only requirement is that the path should refer to the same file, so the current implementation does not violate the spec.

bplb · 2024-09-12T20:40:09Z

This does not address your example above with respect to toRealPath, but the behavior it shows seems inconsistent.

The setup is the same:

/tmp/cwd/dir/subdir or C:\tmp\cwd\dir\subdir where link is a symbolic link in cwd with the relatve target dir/subdir.

Then running jshell in cwd, the call

Files.createFile(Path.of("link/../../file"))

creates /tmp/cwd/file on macOS whereas on Windows it creates C:\tmp\file.

bplb · 2024-09-12T21:49:24Z

Based on the above, I think the current toRealPath implementation on Windows is doing just fine, only the ToRealPath test needs to be fixed.

Based on these comments in WindowsPath

        // Long paths need to have "." and ".." removed and be prefixed with
        // "\\?\". Note that it is okay to remove ".." even when it follows
        // a link - for example, it is okay for foo/link/../bar to be changed
        // to foo/bar. The reason is that Win32 APIs to access foo/link/../bar
        // will access foo/bar anyway (which differs to Unix systems)

it seems safe to say that the current toRealPath() implementation is, as you say, all right as is. If these comments are correct, then it appears that on Unix the link in foo/link/../bar would first be resolved and then the .. would be removed along with the element preceding it, but on Windows, link is just treated as a name element without resolution and is summarily removed along with ... This is consistent with the Files.createFile(Path.of("link/../../file")) example I posted above.

bplb · 2024-09-12T21:50:21Z

[...] only the ToRealPath test needs to be fixed.

As far as I could tell, Path.toRealPath does not make any promises about what the resulting path should be. The only requirement is that the path should refer to the same file, so the current implementation does not violate the spec.

At this point, I tend to agree.

… a small type in Files

bplb · 2024-09-16T23:49:22Z

In C:\Users\bpb\dev\bugs, create dir, dir\subdir, and link -> dir\subdir. Then running a simple native test using CreateFileW to create a new file C:\Users\bpb\dev\bugs\link\..\file.txt actually creates C:\Users\bpb\dev\bugs\file.txt, which corroborates the conclusion of this PR. The initial version of this PR incorrectly expected that file.txt would be C:\Users\bpb\dev\bugs\dir\file.txt.

bplb · 2024-09-17T15:49:23Z

/integrate

openjdk · 2024-09-17T15:50:18Z

Going to push as commit f877016.
Since your change was applied there have been 50 commits pushed to the master branch:

b39e6a8: 8329816: Add SLEEF version 3.6.1
80db6e7: 8339648: ZGC: Division by zero in rule_major_allocation_rate
269cd38: 8338566: Lazy creation of exception instances is not thread safe
8b6e277: 8340273: Remove CounterHalfLifeTime
c6721a0: 8340009: Improve the output from assert_different_registers
7834662: 8340119: Remove oopDesc::size_might_change()
10050a7: 8332442: C2: refactor Mod cases in Compile::final_graph_reshaping_main_switch()
7849f25: 8340184: Bug in CompressedKlassPointers::is_in_encodable_range
a4cf191: 8339793: Fix incorrect APX feature enabling with -XX:-UseAPX
3e03e66: 8340176: Replace usage of -noclassgc with -Xnoclassgc in test/jdk/java/lang/management/MemoryMXBean/LowMemoryTest2.java
... and 40 more: https://git.openjdk.org/jdk/compare/81ff91ef27a6a856ae2c453a9a9b8333b91da3ab...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-09-17T15:50:25Z

@bplb Pushed as commit f877016.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "…

a83f0e9

…../../" follows a link (win)

openjdk bot added the rfr Pull request is ready for review label Aug 31, 2023

openjdk bot added the nio nio-dev@openjdk.org label Aug 31, 2023

8315273: Add bug ID to test

a632bd8

bplb added 2 commits October 27, 2023 11:29

Merge

dd3ce6d

Merge

e124841

bridgekeeper bot closed this Dec 12, 2023

openjdk bot reopened this Dec 12, 2023

bplb added 3 commits March 1, 2024 13:04

8315273: Revert ProblemList

81253ee

Merge

0f270a0

8315273: Re-remove ToRealPath test

466aa69

bplb added 2 commits September 12, 2024 15:18

Merge

b97d0a7

8315273: Revert source changes; update test to match reality; correct…

7e1b469

… a small type in Files

djelinski approved these changes Sep 16, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Sep 16, 2024

openjdk bot added the integrated Pull request has been integrated label Sep 17, 2024

openjdk bot closed this Sep 17, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Sep 17, 2024

bplb deleted the Path-toRealPath-win-8315273 branch September 17, 2024 17:08

8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win) #15525

8315273: (fs) Path.toRealPath(LinkOption.NOFOLLOW_LINKS) fails when "../../" follows a link (win) #15525

Uh oh!

Conversation

bplb commented Aug 31, 2023 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Webrev

Uh oh!

bridgekeeper bot commented Aug 31, 2023

Uh oh!

openjdk bot commented Aug 31, 2023

Uh oh!

mlbridge bot commented Aug 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

bridgekeeper bot commented Sep 29, 2023

Uh oh!

bplb commented Sep 29, 2023

Uh oh!

bridgekeeper bot commented Oct 27, 2023

Uh oh!

bridgekeeper bot commented Dec 12, 2023

Uh oh!

bplb commented Dec 12, 2023

Uh oh!

openjdk bot commented Dec 12, 2023

Uh oh!

bridgekeeper bot commented Jan 10, 2024

Uh oh!

bplb commented Jan 10, 2024

Uh oh!

bridgekeeper bot commented Feb 7, 2024

Uh oh!

bplb commented Feb 7, 2024

Uh oh!

openjdk bot commented Mar 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bridgekeeper bot commented Apr 11, 2024

Uh oh!

bplb commented Apr 11, 2024

Uh oh!

bridgekeeper bot commented May 10, 2024

Uh oh!

bplb commented May 10, 2024

Uh oh!

bridgekeeper bot commented Jun 7, 2024

Uh oh!

bplb commented Jun 7, 2024

Uh oh!

bridgekeeper bot commented Jul 5, 2024

Uh oh!

bplb commented Jul 16, 2024

Uh oh!

bridgekeeper bot commented Aug 13, 2024

Uh oh!

bplb commented Aug 13, 2024

Uh oh!

bridgekeeper bot commented Sep 10, 2024

Uh oh!

bplb commented Sep 10, 2024

Uh oh!

djelinski commented Sep 11, 2024

Uh oh!

bplb commented Sep 11, 2024

Uh oh!

djelinski commented Sep 12, 2024

Uh oh!

bplb commented Sep 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bplb commented Sep 12, 2024

Uh oh!

bplb commented Sep 12, 2024

Uh oh!

bplb commented Aug 31, 2023 •

edited by openjdk bot

Loading

mlbridge bot commented Aug 31, 2023 •

edited

Loading

openjdk bot commented Mar 13, 2024 •

edited

Loading

bplb commented Sep 12, 2024 •

edited

Loading