New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

path/filepath: TestIssue29372 fails on windows-arm builder #29746

Open
alexbrainman opened this Issue Jan 15, 2019 · 8 comments

Comments

Projects
None yet
5 participants
@alexbrainman
Copy link
Member

alexbrainman commented Jan 15, 2019

Since 44cf595 windows-arm builder fails with

https://build.golang.org/log/c00262fb40118e6917815b61ce7e1643b702bdc3

--- FAIL: TestIssue29372 (0.02s)
    path_test.go:1398: test#0: want "The system cannot find the path specified.", got %!q(<nil>)
FAIL
FAIL	path/filepath	26.728s

I don't see how TestIssue29372 can fail this way. @jordanrh1 please help to debug this.

We can skip the test on windows-arm, but I would like to understand what is wrong here first.

Thank you.

Alex

@jordanrh1

This comment has been minimized.

Copy link
Contributor

jordanrh1 commented Jan 22, 2019

  • On ARM, the temporary file gets created at C:\Data\Users\administrator\AppData\Local\Temp\issue29372940456735.
  • C:\Data is actually a volume mount point.
  • When walkSymlinks() tests to see if C:\Data is a symlink, it finds that it is, and Readlink returns Volume{ae420040-0000-0000-0000-008200000000}.
  • walkSymlinks then tries to Lstat this path, but Lstat fails because the path name is missing the required \\?\ prefix. The path must look like \\?\Volume{ae420040-0000-0000-0000-008200000000}.
  • walkSymlinks then returns an error, which causes evalSymlinks to call evalSymlinksUsingGetFinalPathNameByHandle, which succeeds.
  • Since the test is expecting EvalSymlinks to fail with ENOTDIR, but EvalSymlinks succeeded, the test fails.

I'm looking into Readlink to see if the returned value should be prefixed with \\?\.

@jordanrh1

This comment has been minimized.

Copy link
Contributor

jordanrh1 commented Jan 22, 2019

@alexbrainman Do you know why walkSymlinks is used? Why is it not sufficient to use evalSymlinksUsingGetFinalPathNameByHandle ?

After fixing Readlink, walkSymlinks is now succeeding in cases where it didn't before, which causes evalSymlinks not to fall back to evalSymlinksUsingGetFinalPathNameByHandle. evalSymlinksUsingGetFinalPathNameByHandle returns paths with a drive letter when possible, but walkSymlinks returns volume-style names. There is a lot of code that relies on EvalSymlinks to return drive letter style names.

@jordanrh1

This comment has been minimized.

Copy link
Contributor

jordanrh1 commented Jan 22, 2019

It is worth noting that none of this is ARM-specific. It is showing up on the ARM builder because %tmp% is a path that contains a volume mount point.

@alexbrainman

This comment has been minimized.

Copy link
Member Author

alexbrainman commented Jan 24, 2019

  • C:\Data is actually a volume mount point. ...

Nice debugging. So %TMP% directory path has symlink in it. And it is symlink into nt namespace. I can reproduce it here to debug this myself. Thank you very much.

Do you know why walkSymlinks is used? Why is it not sufficient to use evalSymlinksUsingGetFinalPathNameByHandle ?

From what I remember, we had walkSymlinks first. And that worked for awhile. But then we discovered symlinks that points to unusual places, like symlink into nt namespace. Just like you have discovered.

We could have adjusted os.Readlink to return these paths as is. But I don't think you could use nt namespace path in all normal Windows APIs. Could you?

So we came up with current approach to use CreateFile+GetFinalPathNameByHandle to implement path/filepath.EvalSymlinks. And that worked, but it works differently in some corner cases. For example, see path/filepath.TestIssue29372. I suspect many others. So, for backward compatibility, we decided to use walkSymlinks first, and, if that fails, use evalSymlinksUsingGetFinalPathNameByHandle. That covers most current scenarios.

After fixing Readlink, walkSymlinks is now succeeding in cases where it didn't before, which causes evalSymlinks not to fall back to evalSymlinksUsingGetFinalPathNameByHandle. evalSymlinksUsingGetFinalPathNameByHandle returns paths with a drive letter when possible, but walkSymlinks returns volume-style names. There is a lot of code that relies on EvalSymlinks to return drive letter style names.

I don't think we could return nt namespace paths. These will not work properly with current os and path/filepath package implementations. But I could be wrong about that. You are welcome to corect me.

It is worth noting that none of this is ARM-specific. It is showing up on the ARM builder because %tmp% is a path that contains a volume mount point.

Yes, I agree. This is not windows-arm specific. Let me think about this for a couple of days. If I won't come up with anything, I will skip the test for go1.13.

Alex

@jordanrh1

This comment has been minimized.

Copy link
Contributor

jordanrh1 commented Jan 24, 2019

I don't think we could return nt namespace paths

Are you sure GetFinalPathNameByHandle will return nt namespace paths? It looks like there is an option to request DOS-style paths (VOLUME_NAME_DOS).

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jan 25, 2019

Change https://golang.org/cl/159578 mentions this issue: path/filepath: skip TestIssue29372 on windows, if /tmp has symilinks

@alexbrainman

This comment has been minimized.

Copy link
Member Author

alexbrainman commented Jan 25, 2019

Are you sure GetFinalPathNameByHandle will return nt namespace paths? It looks like there is an option to request DOS-style paths (VOLUME_NAME_DOS).

I was not talking not about GetFinalPathNameByHandle when I spoke about nt namespace paths. I was talking about os.Readlink implementation.

If you look at syscal.Readlink, you will see that it uses DeviceIoControl(..., FSCTL_GET_REPARSE_POINT ... to find symlink contents. When I use mklink /j tmplink \\?\Volume{....} command to create symlink, and then use DeviceIoControl to fetch that symlink contents, I get \??\Volume{...}\. But syscal.Readlink cannot return \??\Volume{...}\ as is. If syscal.Readlink would return \??\Volume{...}\ as is, then that path might get passed to other Windows API functions. I don't think all Windows API functions we use will work with path like \??\Volume{...}\.

Anyway I will make TestIssue29372 pass for the moment (see https://golang.org/cl/159578). We would have to deal with this later. I don't see any quick solutions here.

Alex

@alexbrainman

This comment has been minimized.

Copy link
Member Author

alexbrainman commented Jan 25, 2019

Anyway I will make TestIssue29372 pass for the moment (see https://golang.org/cl/159578).

@jordanrh1 I forgotten to ask, if you could verify that https://golang.org/cl/159578 makes TestIssue29372 pass.

Thank you.

Alex

gopherbot pushed a commit that referenced this issue Jan 27, 2019

path/filepath: skip TestIssue29372 on windows, if /tmp has symilinks
TestIssue29372 is broken on windows when temporary directory has
symlink in its path.

Adjust the test to use filepath.EvalSymlinks of temporary directory,
instead of temporary directory on windows. This change is not a
proper fix, but at least it makes TestIssue29372 pass on windows-arm.

See issue for details.

Updates #29746

Change-Id: I2af8ebb89da7cb9daf027a5e49e32ee22dbd0e3d
Reviewed-on: https://go-review.googlesource.com/c/159578
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

@bcmills bcmills added this to the Go1.12 milestone Jan 29, 2019

@bcmills bcmills added the Testing label Jan 29, 2019

@andybons andybons modified the milestones: Go1.12, Go1.13 Feb 12, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment