`Directory.EnumerateFiles` does not respect per-directory case-sensitivity setting

[ericwj]

See also #34235. Area System.IO.

Description

Summary

After creating file.txt and FILE.txt in a directory configured to be case-sensitive, running Directory.EnumerateFiles against it with a pattern of f* returns two elements instead of one, where FindFirstFileEx returns just the one.

Test result

  Message: 
    Assert.Equal() Failure
    Expected: String[] ["file.txt"]
    Actual:   String[] ["FILE.txt", "file.txt"]

Enable NtfsEnableDirCaseSensitivity globally (optional, recommended)

Have to elevate Visual Studio or review/enable/disable/reset opting into case-sensitivity (global setting) from elevated PowerShell before running the test:

Get-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem -Name NtfsEnableDirCaseSensitivity
/// # this will divide by zero => have to pick enabled (1) resp. disabled (0)
# Set-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem -Name NtfsEnableDirCaseSensitivity -Type DWord -Value (1/0)
# Remove-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem -Name NtfsEnableDirCaseSensitivity

Manually set the current directory to be case-sensitive

Not required, for demonstration purposes:

PS> fsutil file SetCaseSensitiveInfo $PWD enable
PS> fsutil file SetCaseSensitiveInfo $PWD disable

Useful links

• FindFirstFileExW function (fileapi.h) - Win32 apps | Microsoft Docs
• How to enable NTFS support to treat folders as case sensitive on Windows 10
• NTFS Case Sensitivity on Windows a very detailed blog

Perhaps these:

• NtQueryInformationFile function (ntifs.h) - Windows drivers | Microsoft Docs
• _FILE_CASE_SENSITIVE_INFORMATION (ntifs.h) - Windows drivers | Microsoft Docs

XUnit test

Create project:

dotnet new xunit

Add references:

<ItemGroup>
  <PackageReference Include="Vanara.PInvoke.Kernel32" Version="3.2.12" />
  <PackageReference Include="Microsoft.Win32.Registry" Version="4.7.0" />
</ItemGroup>

Test:

private const string NtfsEnableDirCaseSensitivity = nameof(NtfsEnableDirCaseSensitivity);
/// <summary>Modifying the registry key will throw if not elevated. 
/// This doesn't change anything if it finds bit 1 set in
/// <c>SYSTEM\CurrentControlSet\Control\FileSystem DWORD NtfsEnableDirCaseSensitivity </c>.</summary>
private static bool TrySetCaseSensitivityGlobal(bool reset, ref int? oldValue) {
    const string fspath = @"SYSTEM\CurrentControlSet\Control\FileSystem";
    var fs = Microsoft.Win32.Registry.LocalMachine.OpenSubKey(fspath);
    var o = fs.GetValue(NtfsEnableDirCaseSensitivity, null);
    var value = o is null ? (int?)null : (int)o;
    if (reset) {
        if (oldValue == null)
            fs.DeleteValue(NtfsEnableDirCaseSensitivity);
        else if (value != oldValue)
            fs.SetValue(NtfsEnableDirCaseSensitivity, oldValue.Value);
        return true;
    } else {
        oldValue = value;
        if (value.HasValue && (1 & value.Value) == 1) return true;
        if (oldValue == null)
            fs.SetValue(NtfsEnableDirCaseSensitivity, 1);
        else
            fs.SetValue(NtfsEnableDirCaseSensitivity, 1 | oldValue.Value);
        return true;
    }
}
private static bool TrySetDirectoryCaseSensitivity(bool enable, string path) {
    var status = enable ? "enable" : "disable";
    var fsutilpsi = new ProcessStartInfo("fsutil",
        $"file SetCaseSensitiveInfo \"{path}\" {status}") {
        CreateNoWindow = true,
        LoadUserProfile = false,
        RedirectStandardError = true,
        RedirectStandardOutput = true,
        UseShellExecute = false,
    };
    var fsutil = Process.Start(fsutilpsi);
    var @out = fsutil.StandardOutput.ReadToEnd();
    var err = fsutil.StandardError.ReadToEnd();
    fsutil.WaitForExit();
    if (!string.IsNullOrEmpty(@out)) Debug.WriteLine(@out);
    if (!string.IsNullOrEmpty(err)) Debug.WriteLine($"ERROR: {err}");
    return fsutil.ExitCode == 0 && string.IsNullOrEmpty(err);
}
[Fact]
public void EnumerateFilesCaseSensitiveDir() {
    var cwd = Environment.CurrentDirectory;
    var path = Environment.GetEnvironmentVariable("TEMP");
    path = Path.Combine(path, nameof(EnumerateFilesCaseSensitiveDir));
    path += $"-{new Random().Next():x8}";
    int? NtfsEnableDirCaseSensitivity = null;

    Directory.CreateDirectory(path);
    try {
        Environment.CurrentDirectory = path;
        try {
            Assert.True(TrySetCaseSensitivityGlobal(reset: false, ref NtfsEnableDirCaseSensitivity));
            try {
                Assert.True(TrySetDirectoryCaseSensitivity(enable: true, path));
                File.WriteAllText("file.txt", "lowercase");
                File.WriteAllText("FILE.txt", "uppercase");
                string[] expected = null;
                using (var searchHandle = Kernel32.FindFirstFileEx(@$"{path}\f*",
                    Kernel32.FINDEX_INFO_LEVELS.FindExInfoBasic,
                    out var ffdata,
                    Kernel32.FINDEX_SEARCH_OPS.FindExSearchNameMatch,
                    lpSearchFilter: default,
                    Kernel32.FIND_FIRST.FIND_FIRST_EX_CASE_SENSITIVE)) {

                    var results = new List<string>();
                    while (!searchHandle.IsInvalid) {
                        results.Add(Path.GetRelativePath(path, ffdata.cFileName));
                        if (!Kernel32.FindNextFile(searchHandle, out ffdata)) break;
                    }
                    expected = results.ToArray();
                } // FindClose; searchHandle.handle = default;

                var actual = Directory
                    .EnumerateFiles(path, "f*.*", SearchOption.TopDirectoryOnly)
                    .Select(file => Path.GetRelativePath(path, file))
                    .ToArray();
                Assert.Equal(expected, actual);
            } finally {
                Assert.True(TrySetCaseSensitivityGlobal(reset: true, ref NtfsEnableDirCaseSensitivity));
            }
        } finally {
            Environment.CurrentDirectory = cwd;
        }
    } finally {
        Directory.Delete(path, recursive: true);
    }
}

Configuration

PS> dotnet --list-sdks
3.1.301 [C:\Program Files\dotnet\sdk]
3.1.400-preview-015178 [C:\Program Files\dotnet\sdk]
5.0.100-preview.5.20279.10 [C:\Program Files\dotnet\sdk]

Regression?

No. Haven't checked, but no.

Opinion

There is an overload of Directory.EnumerateFiles which takes EnumerationOptions that can have its MatchCasing property set to MatchCasing.CaseSensitive, but it might be more convenient, useful and correct to auto-detect case-sensitivity on behalf of the caller on a directory-by-directory basis through the use of perhaps NtQueryInformationFile once per directory during enumeration. Such behavior can today not be specified in any way.

It is slightly disappointing to see the complexity of the imlementation of EnumerateFiles and similar, doing pattern patching and calling into NtXxx instead of more regular Win32 API surface, since that complicates implementing any feature like this.

[carlossanlop]

Thanks for the details.

Changing the default behavior of Directory.EnumerateFiles to be case sensitive would be a breaking change, so the correct way to capture sensitivity is, as you pointed out, by using EnumerationOptions.MatchCasing.

The related issue you shared #34235 has the API proposal to detect sensitivity for the folder. Feel free to also propose adding an API that sets the directory sensitivity at the global level, as you did in your code by modifying the registry.

If there isn't anything else, I can close this issue as a duplicate of #34235 and we can continue the conversation there.

[ericwj]

I don't want Directory.EnumerateFiles to be case-sensitive, but context-sensitive. I don't see how that could be a breaking change since the situation where it matters could only have been encountered very recently with the introduction of that global switch which is off on most systems I presume - such that the context-sensitivity will always have it remain case-insensitive.

[dotnet-policy-service]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.

[dotnet-policy-service]

This issue will now be closed since it had been marked no-recent-activity but received no further activity in the past 14 days. It is still possible to reopen or comment on the issue, but please note that the issue will be locked if it remains inactive for another 30 days.