Skip to content

System.IO.Directory.GetFiles() API usage with recursive symlinks on Unix #20195

@am11

Description

@am11

When the recursive symlink is encountered by GetFiles(path, pattern, SearchOption.AllDirectories) methos, .NET Core throws PathTooLongException (whereas Mono, in this case, throws IndexOutOfRangeException).

  • Is it recommended for the consumers to handle the recursion cases in our code by keeping track of the directory graph node visits?

  • To achieve this, would it be possible to provide an overload which accepts a predicate to determine when to skip / bail out of entering certain directories or symlinks during the traversal? Something like:

    public static string[] GetFiles(string path, string searchPattern, SearchOption searchOption);
    + public static string[] GetFiles(string path, string searchPattern, SearchOption searchOption,
    +                                 Func<string, bool> predicate);

I was looking for the best approach to handle the exceptions during the FS scan and found a similar issue with GetDirectories API in CLI repo: dotnet/cli#5578 which pointed me to the man page about /proc/[pid]/root symlink being recursive by design (added that info in code comment below).

Code:

public class Program
{
  public static void Main(string[] args)
  {
    var files = System.IO.Directory.GetFiles(args[0], "*.*",
                                             System.IO.SearchOption.AllDirectories);
    System.Console.WriteLine($"Files count: {files.Length}");
  }
}

Steps to repro:

# Setup and invocation:

# Platform:
# Ubuntu Trusty 14.04 (Bash on Unbuntu on Windows 10; tested dist info with `cat /etc/*release`)

sudo sh -c 'echo "deb [arch=amd64] https://apt-mo.trafficmanager.net/repos/dotnet-release/ trusty main" > /etc/apt/sources.list.d/dotnetdev.list'
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 417A0893
sudo apt-get update
sudo apt-get install dotnet-dev-1.0.0-rc4-004771

mkdir ~/test && cd $_
dotnet new console

cat <<EOT > Program.cs
public class Program
{
  public static void Main(string[] args)
  {
    var files = System.IO.Directory.GetFiles(args[0], "*.*",
                                             System.IO.SearchOption.AllDirectories);
    System.Console.WriteLine($"Files count: {files.Length}");
  }
}
EOT

sudo dotnet restore && sudo dotnet run -- /proc/self

# Note that /proc/self encounters a symlink at /proc/self/root and proc
# man page (https://linux.die.net/man/5/proc) says:
#
# /proc/[pid]/root
# UNIX and Linux support the idea of a per-process root of the file system, set by the chroot(2)
# system call. This file is a symbolic link that points to the process's root directory, and behaves
# as exe, fd/*, etc. do. 
# In a multithreaded process, the contents of this symbolic link are not available if the main
# thread has already terminated (typically by calling pthread_exit(3)). 
Exception:
Unhandled Exception: System.IO.PathTooLongException: The specified file name or path is too long, or a component of the specified path is too long.
   at System.IO.UnixFileSystem.FileSystemEnumerable`1.OpenDirectory(String fullPath)
   at System.IO.UnixFileSystem.FileSystemEnumerable`1.<Enumerate>d__11.MoveNext()
   at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source, Int32& length)
   at System.IO.Directory.InternalGetFileDirectoryNames(String path, String userPathOriginal, String searchPattern, Boolean includeFiles, Boolean includeDirs, SearchOption searchOption)
   at System.IO.Directory.GetFiles(String path, String searchPattern, SearchOption searchOption)
   at Program.Main(String[] args)

Metadata

Metadata

Assignees

No one assigned

    Labels

    api-needs-workAPI needs work before it is approved, it is NOT ready for implementationarea-System.IOos-linuxLinux OS (any supported distro)

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions