Symlink reparse points not resolved above the mount point #454

mikelatcs · 2022-10-11T06:30:40Z

Bug Report

I think there is something loose in the code that solves symlinks.

Relative paths crossing the mount point fail to work for both files and directories.
Directory symlinks with relative paths pointing to the mount point (not only above it) will fail to be traversed by applications.

When testing with directories:

cd from cmd: The name of the file cannot be resolved by the system.
Windows Explorer shows a similar error when trying to open the symlink.
WinFSP returns 0xC0000280 (STATUS_REPARSE_POINT_NOT_RESOLVED).

Absolute paths work OK.
The symlinks that trigger this behavior work OK when moved down the file hierarchy.

How to Reproduce

Case 1 - Link to dot

C:\SOMEWHERE\MOUNT_POINT> mkdir dir
C:\SOMEWHERE\MOUNT_POINT> cd dir
C:\SOMEWHERE\MOUNT_POINT\dir> mklink /D link .
C:\SOMEWHERE\MOUNT_POINT\dir> cd link # You can also try to open it in Windows Explorer.
C:\SOMEWHERE\MOUNT_POINT\dir\link>
# This is OK. New path is ends with "link". Actually, we are in the same directory.

C:\SOMEWHERE\MOUNT_POINT\dir\link> cd ..
C:\SOMEWHERE\MOUNT_POINT\dir> move link ..
C:\SOMEWHERE\MOUNT_POINT\dir> cd ..
C:\SOMEWHERE\MOUNT_POINT> cd link # You can also try to open it in Windows Explorer.
The name of the file cannot be resolved by the system.
# [[ This is KO! ]]

Case 2 - Link to dot-dot

C:\SOMEWHERE\MOUNT_POINT\dir> mklink /D link ..\.. # Target is "C:\SOMEWHERE"
C:\SOMEWHERE\MOUNT_POINT\dir> cd link # You can also try to open it in Windows Explorer.
The name of the file cannot be resolved by the system.
# [[ This is KO! ]]

Behaviors

It would be expected to be able to access both, the mount point directory and locations out of it by using relative symlinks (already possible with absolute paths).

Environment

OS version and build: 10.0.19044.2006
WinFsp version and build: WinFsp 2022.2 Beta 1 (C# version of the library).

The text was updated successfully, but these errors were encountered:

billziss-gh · 2022-10-18T14:13:49Z

If I understand your scenario correctly, you are trying to create a relative symbolic link that points outside the file system.

I believe that this is not allowed on Windows, because a relative symbolic link must point inside the file system.

mikelatcs · 2022-10-18T15:35:09Z

If I understand your scenario correctly, you are trying to create a relative symbolic link that points outside the file system.

Note that I'm trying to point at the mount point also, which lies inside the file system!

I believe that this is not allowed on Windows, because a relative symbolic link must point inside the file system.

I think that both, junction mount points and symlinks allow to cross file system boundaries on Windows by using relative target paths. Look (I used one VHD and WinFPS, which involves three file systems!):

C:\Environment\VHD>mklink /D symlink ..\..\Environment\MOUNT_POINT
symbolic link created for symlink <<===>> ..\..\Environment\MOUNT_POINT

C:\Environment\VHD>mklink /J junction ..\..\Environment\MOUNT_POINT
Junction created for junction <<===>> ..\..\Environment\MOUNT_POINT

>cd symlink
>cd ..
>cd junction

billziss-gh · 2022-10-18T19:18:18Z

Can you try fsutil reparsepoint query to see what is inside these reparse points?

mikelatcs · 2022-10-19T07:31:20Z

Sure! On top of it, I can show you the fields already processed with an additional tool.

Anecdotically, as we don't have junction support, I'll only include a junction example for the non-virtual file system at the end. Note I was wrong for these! I.e.: for junctions, an absolute path is stored no matter what.

My humble guess is that the additional "Flags" field in the reparse point "header" makes all the difference for symlinks, which support relative paths as a feature as an improvement over junctions...

Tested in:

C:\Environment
C:\Environment\MOUNT_POINT

Output is identical for both directories (I checked just in case we had a bug). Here, I only included one copy of it.

1. SYMLINK - SAME FILE SYSTEM

> mklink /D dot .

C:\Environment>fsutil reparsepoint query dot
Reparse Tag Value : 0xa000000c
Tag value: Microsoft, Name Surrogate, Symbolic Link

Reparse Data Length: 0x10
Reparse Data:
0000:  02 00 02 00 00 00 02 00  01 00 00 00 2e 00 2e 00  ................

C:\Environment>linktool read dot
File attributes: 1040 (410h)

Payload:
.ReparseTag: A000000C
.ReparseDataLength: 16
.Reserved: 0
.SymbolicLinkReparseBuffer
    .SubstituteNameOffset: 2
    .SubstituteNameLength: 2
    .PrintNameOffset: 0
    .PrintNameLength: 2
    .Flags: 1 (SYMLINK_FLAG_RELATIVE)
    .PathBuffer:
    Substitute name: .
    Print name: .

{
  0x0C, 0x00, 0x00, 0xA0, 0x10, 0x00, 0x00, 0x00, 0x02, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x2E, 0x00, 0x2E, 0x00,
}

2. SYMLINK - CROSSING FILE SYSTEMS

> mklink /D symlink ..\..\Environment\MOUNT_POINT

C:\Environment>fsutil reparsepoint query symlink
Reparse Tag Value : 0xa000000c
Tag value: Microsoft, Name Surrogate, Symbolic Link

Reparse Data Length: 0x80
Reparse Data:
0000:  3a 00 3a 00 00 00 3a 00  01 00 00 00 2e 00 2e 00  :.:...:.........
0010:  5c 00 2e 00 2e 00 5c 00  45 00 6e 00 76 00 69 00  \.....\.E.n.v.i.
0020:  72 00 6f 00 6e 00 6d 00  65 00 6e 00 74 00 5c 00  r.o.n.m.e.n.t.\.
0030:  4d 00 4f 00 55 00 4e 00  54 00 5f 00 50 00 4f 00  M.O.U.N.T._.P.O.
0040:  49 00 4e 00 54 00 2e 00  2e 00 5c 00 2e 00 2e 00  I.N.T.....\.....
0050:  5c 00 45 00 6e 00 76 00  69 00 72 00 6f 00 6e 00  \.E.n.v.i.r.o.n.
0060:  6d 00 65 00 6e 00 74 00  5c 00 4d 00 4f 00 55 00  m.e.n.t.\.M.O.U.
0070:  4e 00 54 00 5f 00 50 00  4f 00 49 00 4e 00 54 00  N.T._.P.O.I.N.T.

C:\Environment>linktool read symlink
File attributes: 1040 (410h)

Payload:
.ReparseTag: A000000C
.ReparseDataLength: 128
.Reserved: 0
.SymbolicLinkReparseBuffer
    .SubstituteNameOffset: 58
    .SubstituteNameLength: 58
    .PrintNameOffset: 0
    .PrintNameLength: 58
    .Flags: 1 (SYMLINK_FLAG_RELATIVE)
    .PathBuffer:
    Substitute name: ..\..\Environment\MOUNT_POINT
    Print name: ..\..\Environment\MOUNT_POINT

{
  0x0C, 0x00, 0x00, 0xA0, 0x80, 0x00, 0x00, 0x00, 0x3A, 0x00, 0x3A, 0x00, 0x00, 0x00, 0x3A, 0x00,
  0x01, 0x00, 0x00, 0x00, 0x2E, 0x00, 0x2E, 0x00, 0x5C, 0x00, 0x2E, 0x00, 0x2E, 0x00, 0x5C, 0x00,
  0x45, 0x00, 0x6E, 0x00, 0x76, 0x00, 0x69, 0x00, 0x72, 0x00, 0x6F, 0x00, 0x6E, 0x00, 0x6D, 0x00,
  0x65, 0x00, 0x6E, 0x00, 0x74, 0x00, 0x5C, 0x00, 0x4D, 0x00, 0x4F, 0x00, 0x55, 0x00, 0x4E, 0x00,
  0x54, 0x00, 0x5F, 0x00, 0x50, 0x00, 0x4F, 0x00, 0x49, 0x00, 0x4E, 0x00, 0x54, 0x00, 0x2E, 0x00,
  0x2E, 0x00, 0x5C, 0x00, 0x2E, 0x00, 0x2E, 0x00, 0x5C, 0x00, 0x45, 0x00, 0x6E, 0x00, 0x76, 0x00,
  0x69, 0x00, 0x72, 0x00, 0x6F, 0x00, 0x6E, 0x00, 0x6D, 0x00, 0x65, 0x00, 0x6E, 0x00, 0x74, 0x00,
  0x5C, 0x00, 0x4D, 0x00, 0x4F, 0x00, 0x55, 0x00, 0x4E, 0x00, 0x54, 0x00, 0x5F, 0x00, 0x50, 0x00,
  0x4F, 0x00, 0x49, 0x00, 0x4E, 0x00, 0x54, 0x00,
}

3. JUNCTION - CROSSING FILE SYSTEMS

Only tested in C:\Environment\MOUNT_POINT

> mklink /J junction ..\..\Environment\MOUNT_POINT

C:\Environment>fsutil reparsepoint query junction
Reparse Tag Value : 0xa0000003
Tag value: Microsoft, Name Surrogate, Mount Point
Substitue Name offset: 0
Substitue Name length: 60
Print Name offset:     62
Print Name Length:     52
Substitute Name:       \??\C:\Environment\MOUNT_POINT
Print Name:            C:\Environment\MOUNT_POINT

Reparse Data Length: 0x7c
Reparse Data:
0000:  00 00 3c 00 3e 00 34 00  5c 00 3f 00 3f 00 5c 00  ..<.>.4.\.?.?.\.
0010:  43 00 3a 00 5c 00 45 00  6e 00 76 00 69 00 72 00  C.:.\.E.n.v.i.r.
0020:  6f 00 6e 00 6d 00 65 00  6e 00 74 00 5c 00 4d 00  o.n.m.e.n.t.\.M.
0030:  4f 00 55 00 4e 00 54 00  5f 00 50 00 4f 00 49 00  O.U.N.T._.P.O.I.
0040:  4e 00 54 00 00 00 43 00  3a 00 5c 00 45 00 6e 00  N.T...C.:.\.E.n.
0050:  76 00 69 00 72 00 6f 00  6e 00 6d 00 65 00 6e 00  v.i.r.o.n.m.e.n.
0060:  74 00 5c 00 4d 00 4f 00  55 00 4e 00 54 00 5f 00  t.\.M.O.U.N.T._.
0070:  50 00 4f 00 49 00 4e 00  54 00 00 00              P.O.I.N.T...

C:\Environment>linktool read junction
File attributes: 1040 (410h)

Payload:
.ReparseTag: A0000003
.ReparseDataLength: 124
.Reserved: 0
.MountPointReparseBuffer
    .SubstituteNameOffset: 0
    .SubstituteNameLength: 60
    .PrintNameOffset: 62
    .PrintNameLength: 52
    .PathBuffer:
    Substitute name: \??\C:\Environment\MOUNT_POINT
    Print name: C:\Environment\MOUNT_POINT

{
  0x03, 0x00, 0x00, 0xA0, 0x7C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x3C, 0x00, 0x3E, 0x00, 0x34, 0x00,
  0x5C, 0x00, 0x3F, 0x00, 0x3F, 0x00, 0x5C, 0x00, 0x43, 0x00, 0x3A, 0x00, 0x5C, 0x00, 0x45, 0x00,
  0x6E, 0x00, 0x76, 0x00, 0x69, 0x00, 0x72, 0x00, 0x6F, 0x00, 0x6E, 0x00, 0x6D, 0x00, 0x65, 0x00,
  0x6E, 0x00, 0x74, 0x00, 0x5C, 0x00, 0x4D, 0x00, 0x4F, 0x00, 0x55, 0x00, 0x4E, 0x00, 0x54, 0x00,
  0x5F, 0x00, 0x50, 0x00, 0x4F, 0x00, 0x49, 0x00, 0x4E, 0x00, 0x54, 0x00, 0x00, 0x00, 0x43, 0x00,
  0x3A, 0x00, 0x5C, 0x00, 0x45, 0x00, 0x6E, 0x00, 0x76, 0x00, 0x69, 0x00, 0x72, 0x00, 0x6F, 0x00,
  0x6E, 0x00, 0x6D, 0x00, 0x65, 0x00, 0x6E, 0x00, 0x74, 0x00, 0x5C, 0x00, 0x4D, 0x00, 0x4F, 0x00,
  0x55, 0x00, 0x4E, 0x00, 0x54, 0x00, 0x5F, 0x00, 0x50, 0x00, 0x4F, 0x00, 0x49, 0x00, 0x4E, 0x00,
  0x54, 0x00, 0x00, 0x00,
}

billziss-gh · 2022-10-19T11:11:52Z

Anecdotically, as we don't have junction support, I'll only include a junction example for the non-virtual file system at the end. Note I was wrong for these! I.e.: for junctions, an absolute path is stored no matter what.

Yes, junctions have some limitations:

They only support absolute paths.
They can only point to directories.
They cannot point to network file systems.

C:\Environment\VHD>mklink /D symlink ..\..\Environment\MOUNT_POINT
symbolic link created for symlink <<===>> ..\..\Environment\MOUNT_POINT

Earlier you posted this relative symbolic link across a VHD boundary. (I am assuming that the VHD was formatted with NTFS and mounted on a directory.) Can you also post the fsutil / linktool output for that symlink?

My humble guess is that the additional "Flags" field in the reparse point "header" makes all the difference for symlinks, which support relative paths as a feature as an improvement over junctions...

Indeed symbolic links have the SYMLINK_FLAG_RELATIVE flag which makes them "relative". It is/was my impression that this "relative" works differently from UNIX and that relative symlinks are confined to the device (file system) they are on. It's been a while that I worked with reparse points in WinFsp and there may be some subtlety that I do not remember. Or the inability to cross file system boundaries may be an outright bug in WinFsp.

BTW, you can have a relative symbolic link that points to \ in Windows and it points to the root of the file system that it is in. WinFsp uses this trick in the FUSE symlink implementation.

mikelatcs · 2022-10-20T14:56:46Z

Earlier you posted this relative symbolic link across a VHD boundary. (I am assuming that the VHD was formatted with NTFS and mounted on a directory.) Can you also post the fsutil / linktool output for that symlink?

Yes, sure! Mind you, no big surprises here. Also, it is just as your said: I formated the drive in NTFS and mounted it as directory.

It is/was my impression that this "relative" works differently from UNIX and that relative symlinks are confined to the device (file system) they are on.

After these checks, I'm pretty sure they are not confined to file system, and I think it should work for all those cases.

Besides, the solver is failing to find the root of the virtual file system, so at least there is something off there. Not super sure if my code is doing something wrong though.

Could you check the same "mklink /D dot ." test on a reference implementation, just in case? That would at least narrow down what is going on, whether it is in my side on in WinFSP.

BTW, you can have a relative symbolic link that points to \ in Windows and it points to the root of the file system that it is in. WinFsp uses this trick in the FUSE symlink implementation.

Clever! Indeed, I wasn't trying that in the example, I just added an extra level out of a whim, but makes a lot of sense.

C:\Environment\VHD>mklink /D symlink ..\..\Environment\MOUNT_POINT
symbolic link created for symlink <<===>> ..\..\Environment\MOUNT_POINT

C:\Environment\VHD>fsutil reparsepoint query symlink
Reparse Tag Value : 0xa000000c
Tag value: Microsoft, Name Surrogate, Symbolic Link

Reparse Data Length: 0x80
Reparse Data:
0000:  3a 00 3a 00 00 00 3a 00  01 00 00 00 2e 00 2e 00  :.:...:.........
0010:  5c 00 2e 00 2e 00 5c 00  45 00 6e 00 76 00 69 00  \.....\.E.n.v.i.
0020:  72 00 6f 00 6e 00 6d 00  65 00 6e 00 74 00 5c 00  r.o.n.m.e.n.t.\.
0030:  4d 00 4f 00 55 00 4e 00  54 00 5f 00 50 00 4f 00  M.O.U.N.T._.P.O.
0040:  49 00 4e 00 54 00 2e 00  2e 00 5c 00 2e 00 2e 00  I.N.T.....\.....
0050:  5c 00 45 00 6e 00 76 00  69 00 72 00 6f 00 6e 00  \.E.n.v.i.r.o.n.
0060:  6d 00 65 00 6e 00 74 00  5c 00 4d 00 4f 00 55 00  m.e.n.t.\.M.O.U.
0070:  4e 00 54 00 5f 00 50 00  4f 00 49 00 4e 00 54 00  N.T._.P.O.I.N.T.

C:\Environment\VHD>linktool read symlink
File attributes: 1040 (410h)

Payload:
.ReparseTag: A000000C
.ReparseDataLength: 128
.Reserved: 0
.SymbolicLinkReparseBuffer
    .SubstituteNameOffset: 58
    .SubstituteNameLength: 58
    .PrintNameOffset: 0
    .PrintNameLength: 58
    .Flags: 1 (SYMLINK_FLAG_RELATIVE)
    .PathBuffer:
    Substitute name: ..\..\Environment\MOUNT_POINT
    Print name: ..\..\Environment\MOUNT_POINT

{
 0x0C, 0x00, 0x00, 0xA0, 0x80, 0x00, 0x00, 0x00, 0x3A, 0x00, 0x3A, 0x00, 0x00, 0x00, 0x3A, 0x00,
 0x01, 0x00, 0x00, 0x00, 0x2E, 0x00, 0x2E, 0x00, 0x5C, 0x00, 0x2E, 0x00, 0x2E, 0x00, 0x5C, 0x00,
 0x45, 0x00, 0x6E, 0x00, 0x76, 0x00, 0x69, 0x00, 0x72, 0x00, 0x6F, 0x00, 0x6E, 0x00, 0x6D, 0x00,
 0x65, 0x00, 0x6E, 0x00, 0x74, 0x00, 0x5C, 0x00, 0x4D, 0x00, 0x4F, 0x00, 0x55, 0x00, 0x4E, 0x00,
 0x54, 0x00, 0x5F, 0x00, 0x50, 0x00, 0x4F, 0x00, 0x49, 0x00, 0x4E, 0x00, 0x54, 0x00, 0x2E, 0x00,
 0x2E, 0x00, 0x5C, 0x00, 0x2E, 0x00, 0x2E, 0x00, 0x5C, 0x00, 0x45, 0x00, 0x6E, 0x00, 0x76, 0x00,
 0x69, 0x00, 0x72, 0x00, 0x6F, 0x00, 0x6E, 0x00, 0x6D, 0x00, 0x65, 0x00, 0x6E, 0x00, 0x74, 0x00,
 0x5C, 0x00, 0x4D, 0x00, 0x4F, 0x00, 0x55, 0x00, 0x4E, 0x00, 0x54, 0x00, 0x5F, 0x00, 0x50, 0x00,
 0x4F, 0x00, 0x49, 0x00, 0x4E, 0x00, 0x54, 0x00,
}

billziss-gh · 2022-10-20T16:31:20Z

Thank you for the new output, which seems to confirm that relative symlinks are supposed to work across file system boundaries.

If we were to change this behavior in WinFsp and allow relative symlinks to cross file system boundaries we would have to consider complications such as backwards compatibility and security. For example, existing file systems that support reparse points may fully expect the current WinFsp behavior and we should not change that behavior without explicit consent. I would likely do this by adding a new field (e.g. AllowRelSymlinksAcrossFileSystem) in FSP_FSCTL_VOLUME_PARAMS. File systems that want to opt into the new behavior will have to set this field.

Reparse point resolution happens in user mode at FspFileSystemResolveReparsePointsInternal. Dot handling happens at this location. The kernel mode FSD also checks any returned symlink for validity at this location. These locations and likely others would have to be changed. All existing tests should pass (unless they test for the opposite behavior!) and we should likely devise new tests for the new behavior under AllowRelSymlinksAcrossFileSystem.

I currently have no time to work on this. Perhaps you might consider taking a stab at it on your own?

mhx · 2023-11-07T20:15:43Z

I just ran into this limitation as well when looking for a potential solution to a problem of one of my project's users. This solution (a relative symlink crossing the boundary from a read-only file system to a writeable volume) works fine on Linux, but not using WinFsp.

Unfortunately I don't currently have the time to dig into the WinFsp code myself, especially given my lack of Windows development experience.

eryksun · 2024-05-07T22:12:56Z

BTW, you can have a relative symbolic link that points to \ in Windows and it points to the root of the file system that it is in. WinFsp uses this trick in the FUSE symlink implementation.

Actually, the I/O manager reparses "\" targets on the device of the opened path, not the root of the filesystem. For example, if the opened path is "\Device\HarddiskVolume2\Mounts\MountPoint\Symlink", and "Symlink" targets "\", then it resolves to "\Device\HarddiskVolume2\".

There's a lot of careful code that handles system mount points (i.e. IO_REPARSE_TAG_MOUNT_POINT, AKA "junctions") differently from symlinks (i.e. IO_REPARSE_TAG_SYMLINK). The implementation tries to match the behavior of mount points and symlinks on POSIX, as close as possible since every device on Windows has its own root path in its own namespace (i.e. relative symlinks aren't allowed to traverse into the object namespace). The I/O manager maintains an opened path across any number of traversed junctions. In constrast, resolving a symlink always replaces the opened path in subsequent path parsing, just as it does on POSIX.

This design also supports junction mount points and symlinks in remote paths. The server side is always responsible for resolving junctions, which must target a local path on the server. The client side is always responsible for resolving symlinks (and may disallow the target type -- L2L, L2R, R2L, R2R). For reparsing a symlink, the server sends a message back to the client with the opened path on the share and the symlink target path. The opened path will include any number of traversed junctions. They remain in the opened path like regular directories as far as the client is concerned.

Note that relative symlinks can resolve differently depending on the mount point that's used. They may even fail to resolve when accessed from a particular mount point. For example, if "\Device\HarddiskVolume3\Symlink" targets "..", path resolution will fail as invalid reparse data if the system tries to traverse "Symlink" via the mount point "\Device\HarddiskVolume3\". On the other hand, if the opened path is "\Device\HarddiskVolume2\Mounts\MountPoint\Symlink", for which "\Device\HarddiskVolume2\Mounts\MountPoint" is a junction that targets "\Device\HarddiskVolume3\", then the path will be successfully resolved as "\Device\HarddiskVolume2\Mounts".

This was referenced Nov 7, 2023

Command line option (for windows) to set a folder where writable files go [] mhx/dwarfs#176

Closed

[FUSE API] drive letters in symlinks are not handled correctly #530

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Symlink reparse points not resolved above the mount point #454

Symlink reparse points not resolved above the mount point #454

mikelatcs commented Oct 11, 2022 •

edited

Loading

billziss-gh commented Oct 18, 2022

mikelatcs commented Oct 18, 2022 •

edited

Loading

billziss-gh commented Oct 18, 2022

mikelatcs commented Oct 19, 2022 •

edited

Loading

billziss-gh commented Oct 19, 2022

mikelatcs commented Oct 20, 2022

billziss-gh commented Oct 20, 2022

mhx commented Nov 7, 2023

eryksun commented May 7, 2024

Symlink reparse points not resolved above the mount point #454

Symlink reparse points not resolved above the mount point #454

Comments

mikelatcs commented Oct 11, 2022 • edited Loading

Bug Report

How to Reproduce

Behaviors

Environment

billziss-gh commented Oct 18, 2022

mikelatcs commented Oct 18, 2022 • edited Loading

billziss-gh commented Oct 18, 2022

mikelatcs commented Oct 19, 2022 • edited Loading

1. SYMLINK - SAME FILE SYSTEM

2. SYMLINK - CROSSING FILE SYSTEMS

3. JUNCTION - CROSSING FILE SYSTEMS

billziss-gh commented Oct 19, 2022

mikelatcs commented Oct 20, 2022

billziss-gh commented Oct 20, 2022

mhx commented Nov 7, 2023

eryksun commented May 7, 2024

mikelatcs commented Oct 11, 2022 •

edited

Loading

mikelatcs commented Oct 18, 2022 •

edited

Loading

mikelatcs commented Oct 19, 2022 •

edited

Loading