Problem resolving NFS4 symlinks during "Update library" operations

[dhwalker]

Bug report

Describe the bug

Here is a clear and concise description of what the problem is:

Kodi does not create a new entry in the video library for a symlink that points to another symlink that is outside the scope
of a video source's NFS4 namespace, even if that second symlink points back into the NFS4 namespace (a situation that the NFS4 server handles). This is true, even though the video can be played by browsing Kodi's source for the video. This happens on both Linux and Android TV.

Here's some background...

My NFS4 server (rpi4a) is mounted on the Kodi client system (rpi4c) as /mnt/DWNfs:

dhwalker@rpi4c:~> ls -l /mnt/DWNfs
total 0
dr-xr-xr-x 9 dhwalker dhwalker 114 Jul  4  2022 home
dr-xr-xr-x 1 root     root     108 Sep  8  2021 mnt
dhwalker@rpi4c:~> ls -l /mnt/DWNfs/mnt
total 0
drwxrwxr-x 5 1001 dhwalker  67 Jan 23 11:33 DWMedia7
drwxrwxr-x 8 1001 dhwalker 126 Mar  5 15:48 DWMedia9
drwxrwxr-x 7 1001 dhwalker  94 Apr 12 12:20 DWMyth

DWMedia7, DWMedia9, and DWMyth are physical devices that are exported on the server as follows:

dhwalker@rpi4a> more /etc/exports
/home	*(sec=sys,rw,no_root_squash,async,no_subtree_check,insecure)
/mnt/DWMyth	*(sec=sys,rw,no_root_squash,async,no_subtree_check,insecure)
/mnt/DWMedia7	*(sec=sys,rw,no_root_squash,async,no_subtree_check,insecure)
/mnt/DWMedia9	*(sec=sys,rw,no_root_squash,async,no_subtree_check,insecure)

On the client machine, Kodi is configured to use NFS4, and the sources for Movies and TV shows are /mnt/DWNfs/mnt/DWMyth/VideosMetadata/Movies and /mnt/DWNfs/mnt/DWMyth/VideosMetadata/TV. To save storage space the video files (as opposed to the .nfo, etc. files) are symlinks to the "real" video files stored on DWMedia7 and DWMedia9. For example, the movie "2 Lava 2 Lantula" is on the client machine as:

dhwalker@rpi4c:~> ls -l /mnt/DWNfs/mnt/DWMyth/VideosMetadata/Movies/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg
lrwxrwxrwx 1 dhwalker dhwalker 58 May 23 00:38 /mnt/DWNfs/mnt/DWMyth/VideosMetadata/Movies/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg -> /DWMedia/Videos/DVD5/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg

/DWMedia/Videos/DVD5/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg is not part of the NFS4 namespace, of course, however on the server:

dhwalker@rpi4a> realpath /DWMedia/Videos/DVD5/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg
/mnt/DWMedia9/DVD5/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg

which is in the NFS4 namespace. In fact realpath also finds the "real" video file on the client machine:

dhwalker@rpi4c:~> realpath /mnt/DWNfs/mnt/DWMyth/VideosMetadata/Movies/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg
/mnt/DWNfs/mnt/DWMedia9/DVD5/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg

The upshot of all this is that the video is not added to the library, but Kodi can play the video by browsing the video source.

Expected Behavior

Here is a clear and concise description of what was expected to happen:

Updating the library should add these videos to the library.

Actual Behavior

The videos are not added to the library, and the following is displayed in the debug log.

2023-05-24 14:54:45.095 T:11826   debug <general>: NFS: version: 4
2023-05-24 14:54:45.120 T:11826   debug <general>: NFS: Connected to server 192.168.1.5 and export / in tmpContext
2023-05-24 14:54:45.120 T:11826   error <general>: NFS: Failed to stat(/DWMedia/Videos/DVD5/2_Lava_2_Lantula/2_Lava_2_Lantula.mpg) on link resolve stat call failed with "NFS4: (path /mnt/DWMyth/VideosMetadata/Movies/2_Lava_2_Lantula/.nomedia) failed with NFS4ERR_NOENT(-2)"
2023-05-24 14:54:45.120 T:11826   debug <general>: VideoInfoScanner: Rescanning dir 'nfs://dwnfs.service.walkerstreet.info/mnt/DWMyth/VideosMetadata/Movies/2_Lava_2_Lantula/' due to change (cf26ce8be4fb5f0c880effbd13918045 != 7d874d9413f6a0bb63bf950b8bd29337)
2023-05-24 14:54:45.120 T:11826   debug <general>: VideoInfoScanner: No (new) information was found in dir nfs://dwnfs.service.walkerstreet.info/mnt/DWMyth/VideosMetadata/Movies/2_Lava_2_Lantula/

Possible Fix

From the debug log, it appears that Kodi is tracing each step in the sequence of symlinks itself. Using realpath or some other mechanism that is supported by NFS4 to locate the "real" file at the end of the sequence should resolve the problem.

To Reproduce

Steps to reproduce the behavior:

1. On an NFS4 server that is used as a video source for Kodi, create a new video that is a symlink that points to a second symlink, outside the scope of the NFS4 server, that finally points to a "real" video file.
2. Use Kodi to browse the video source to that new video. You can play it.
3. Use Kodi to update your video library. The new video will not appear in the library.

Debuglog

The debuglog can be found here:
Part 1: https://paste.kodi.tv/wazapujuga.kodi
Part 2: https://paste.kodi.tv/noqaciqaya.kodi
Part 3: https://paste.kodi.tv/ayelubuzud
Part 4: https://paste.kodi.tv/xibicalofi.kodi

Screenshots

Here are some links or screenshots to help explain the problem:

Additional context or screenshots (if appropriate)

Here is some additional context or explanation that might help:

Your Environment

Used Operating system:

• Android

• iOS

• tvOS

• Linux

• macOS

• Windows

• Windows UWP

• Operating system version/name: openSUSE Tumbleweed 20230509

• Kodi version: 20.1 (20.1.0)

note: Once the issue is made we require you to update it with new information or Kodi versions should that be required.
Team Kodi will consider your problem report however, we will not make any promises the problem will be solved.

[pduveau]

The KODI code uses nfs_stat64 to stat files/directories. This method is expected to act the same way with links as opening the file. So Kodi does the right call to the nfs library. With the option "no_subtree_check" in your export, you indicate to your nfs server to accept that not all links in the chain has to be exported. It seems that this option is efficient while opening but not while stating the same chain. So it seems that the issue is more on the nfs server side than in Kodi.

[dhwalker]

Thanks for getting back to me. I just tried removing "no_subtree_check" from the exports, but it didn't help. Nevertheless, I think your analysis of it being on the NFS server side is right. Since I don't have a clean way to demonstrate the problem to NFS /kernel supporters, I think I'll go back to the workaround I used under NFS3 (separate Kodi-friendly directory hierarchies for each of DWMedia7 and DWMedia9). If you'd like me to try anything else, though, let me know.

[pduveau]

Exporting the folder "/DWMedia" in a read-only mode for kodi host/ip (rw is not required in your case), should solve the problem in V4.

[neo1973]

Could you test if removing this part fixes the issue for you?

xbmc/xbmc/filesystem/NFSDirectory.cpp

Lines 254 to 264 in e007eb0

	if(tmpDirent.type == NF3LNK)
	{
	CURL linkUrl;
	//resolve symlink changes tmpDirent and strName
	if(!ResolveSymlink(strDirName,&tmpDirent,linkUrl))
	{
	continue;
	}
	path = linkUrl.Get();
	}

[dhwalker]

Sorry, I haven't been keeping up with this issue...

• Regarding pduveau's comment of May 31, 2023, I tried mounting everything read-only; it didn't help.
• Regarding neo1973's comment of January 20, 2024, was that directed to me? I'm not really set up to build Kodi, but I can if it will help.