cl.exe can't access file in mixed case sub-directory of parent created through wsl

[rkwwgit]

This is new in the latest Insider build:

Microsoft Windows [Version 10.0.17093.1000]
(c) 2017 Microsoft Corporation. All rights reserved.

-- reproduction steps:

In wsl shell:

$ mkdir -p /mnt/c/Users/roger/first/mixedCase
$ touch /mnt/c/Users/roger/first/mixedCase/test.cpp

in Developer Command Prompt for VS2015:

cd \Users\roger\first\mixedCase
cl.exe test.cpp

Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x86
Copyright (C) Microsoft Corporation. All rights reserved.

test.cpp
c1xx: fatal error C1003: Cannot open source file: 'test.cpp': No such file or directory

--

'dir' shows the file exists and has zero length (which is not significant here.)

--

It seems to be necessary to have a two-level directory - /mnt/c/Users/roger/mixedCase/test.cpp works fine, as does /mnt/c/Users/roger/first/second/test.cpp

[benhillis]

@SvenGroot - Could you take a look?

[rkwwgit]

The same issue occurs if the mixed case subdirectory is created in Windows (but it's okay if the parent 'first' is created in Windows).

[rkwwgit]

Can work around by re-mounting /mnt/c with -o case=off

[crhilton]

I ran into this too, and I believe I've worked around it semi-permanently with this /etc/wsl.conf:
[automount]
options=case=off

All of the following was with options=case=dir:

I created a new c++ console project in visual studio 2015 names "SimpleTest". Changed nothing in it, just built it. It builds fine. Closed Visual Studio.

Then I create a directory (not under lxss, under my windows user directory) from the WSL prompt, and I moved SimpleTest into that directory using explorer. I reopened SimpleTest, using explorer, and built. Intellisense tells me it can't find headers to import, the compiler says it can't find files. I can open files up and edit them in visual studio, but I can't compile.

If I move this back it fixes it. If I use fsutil file setcasesensitiveinfo to turn sensitivity off for the parent folder, then it works.

From the outside it feels like the bug is in cl.exe. Everything else, explorer.exe, notepad.exe, linux utilities, the editing and project organization part of visual studio are behaving as I expect.

[archer0001]

I noticed in vs 2013 and vs 2017 that certain features of the IDE also failed. e.g. I was unable to search through code for strings, leading me to believe it's not just cl.exe that's having issues with this.

[onomatopellan]

Something is really wrong with case sensitive folders in build 17134 and 17666 too that could explain this issue and #3163 .

I think I have a repo that always shows the bug. These would be the steps:

• Use WSL to create a case sensitive folder in C:
  mkdir /mnt/c/sensible
• Copy any executable in that folder
  cp /mnt/c/Windows/System32/calc.exe /mnt/c/sensible
• Open the sensible folder in Windows Explorer and create a new folder with CTRL+SHIFT+N and press ENTER
• Create another new folder and rename it to anything
• Copy the calc.exe file inside every new created folder

You will see Windows Explorer can't show the Icon of the executable inside the not renamed folder. Process Monitor says PATH NOT FOUND when trying to access the executable.

Right now I recommend using options=case=off and avoid enabling case sensitive folders until this problem is fixed.

[therealkenc]

General pro-tip since this issue is chirping crickets since February. Make your working directory C:\....\source\repos or something along those lines if you can. The -o case=off workaround appears to work for some folks, but at least for the time being you'll save heartache by just avoiding mixed case paths in the first place.

[kromain]

Just chiming in with a few more tips as I struggled to overcome this issue in a mixed wsl-vcvars build environment:

For me adding the case=off option to etc/wsl.conf as recommended above eventually did the trick, but only after the following extra steps:

1. It needed a full reboot, simply closing all my ubuntu shells wasn't enough for me. One simple way to make sure the option has been taken into account is to type mount in a bash shell, you should see the option listed at the end: C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000,case=off). I didn't see it until I did a full machine restart.
2. I needed to re-checkout the parent folder from WSL once mounted with the case=off option. Just deleting the mixed case folder itself and checking it out again with git didn't solve the problem. It's as if the case information was stored in the parent folder, but it is consistent with what's been reported in one of the other comments above.

What's really confusing to me is why cl.exe (or its c1xx frontend) would care about file casing in the first place, and how the parent folder comes into play. Given that the bug manifests itself completely outside of WSL, I'm guessing it's more of a cl.exe bug than a WSL bug. Maybe the compiler is using a obsolete/undocumented Win32 API call to access the input file that's not correctly handling the new metadata added for the per-directory file-casing support added by WSL?

[onomatopellan]

@kromain Here Sven explains how they will treat this and similar case sensitive issues. https://blogs.msdn.microsoft.com/commandline/2018/06/14/improved-per-directory-case-sensitivity-support-in-wsl/

One explanation of why this issue happens is applications like explorer.exe and cl.exe could transform filenames to use all upper or lower case.

[kromain]

@onomatopellan thanks for pointing me to the Sven's blog, somehow I missed it (though I've read the previous ones). Glad to hear there's a fix coming in soon!

Keep it up guys, I'm loving WSL, haven't been that excited about an OS feature in a long time :)

[Brian-Perkins]

The default mount option has been changed to case=off as of Insider Build 17704

[therealkenc]

The default mount option has been changed to case=off

Sad face. Is this a temporary work-around for the Fall release, or is this "the plan"? The blog states:

While in the long term the preferred solution is to fix these applications to correctly handle case sensitive directories

That does not work because you can't update all of them. Say I were highly motivated and hacked up libuv, electron, and VS Code tonight to support the per-directory case sensitivity flag. Great. But I can't stop someone from doing a mkdir from the CMD prompt or creating the folder in Explorer. I can't stop them from opening the project in Sublime or Webstorm. The inheritance helps, but at base doesn't work because I don't know if they are using the latest-greatest per-directory case sensitivity version Windows Git to check out the project (and remembered to check Windows Git's new "use case sensitivity" flag). Neither do they probably. If the flag was not set, okay I leave it alone in libuv. But then they breathe on the directory in WSL with case=dir and the behaviour changes without warning. Not tenable.

[SvenGroot]

Yes, unfortunately, using "case=off" is the plan for the foreseeable future. The fact that DrvFs always created case sensitive directories, everywhere on your system, regardless of the fact that many applications aren't prepared to deal with that. The scenario you describe is exactly one of the reasons why inheritance alone isn't enough with case=dir, and we believe using case=off with more fine-grained control is a better option.

We hope that by switching the default to case=off, providing an easy way to change it in WSL itself (the extended attribute), and turning on inheritance, we've made it easier for you to control where case sensitive directories are used, leading to less random "app X can suddenly no longer access these files" type of problems. We are open to any and all feedback about how we could handle this better.

Note that changing the default to "case=off" only affects newly created directories. Previously created directories that were marked case sensitive are still case sensitive, even after this change.

In conjunction with this change, we've also made it so that the information printed in /proc/self/mounts now always includes the case option, even if you didn't explicitly specify it when mounting. This way, you don't have to guess what the default is based on what build you're running, but can now always see what mode you're running in.

[therealkenc]

We are open to any and all feedback about how we could handle this better.

Equivalent of mount --bind with a case-sensitivity flag on the Windows side. Which has been Cygwin's solution (/cygdrive/c) for about a quarter century.

Someone accessing their 20 year old Visual Studio codebase that has their header file #include directives all case sloppy never operate on that mount. They get the tried-and-loved case insensitivity behaviour. Users of MSVS that open a project with Foo.h and foo.h in the same directory on a case-insensitive mount (classic C:) eat cake, which is no different than it has always been. And they don't care, because they don't name files like that. They just want their "Windows.h" and "windows.h" header includes to work.

Users of VS Code built with a new-fangled libuv/nodejs who operate on the case sensitive mount get all the case sensitive and metadata goodness.

Note that changing the default to "case=off" only affects newly created directories. Previously created directories that were marked case sensitive are still case sensitive, even after this change.

Exactly. And they don't know, because Linux ls or Gnome Nautilus and CMD dir and Windows Explorer don't tell them. They have to remember if they touched some directory three months ago with case=dir.

To sum up: Case sensitivity behaviour isn't sticky. The directory case-sensitivity bit isn't going to be added to the tar file format or git database (though for $7.5b you could, I guess). It is a mount property.

[rkwwgit]

I think it's correct (in the context of this bug) that the default be case=off so the /mnt/c filesystem remains case-preserving but not case sensitive.

The only time case=dir is required is when two filenames in the same directory differ only by (the filesystem's view of) case. That's normally easy to work around - rename a file - and if it isn't, you
can simply mark the directory as case sensitive. (Although that could be hard to maintain.)

It's unfortunate that cl.exe changes filenames internally (you can see what it's doing if you use /showIncludes) because (even with case=off) you can't use the showIncludes output to auto-generate wsl-side 'make' dependencies because 'make' is case sensitive. You can however map the names back to their actual on-disk name with wslpath (not sure how that would work with case=dir where the reverse mapping could be ambiguous).

(A symlink /Users -> /mnt/c/Users also helps with name mapping.)

[rkwwgit]

Outside the context of this bug, case sensitive / case insensitive are not the only options.

Filenames are by their nature human-readable labels for files; the underlying implementation should treat a name as a raw byte array; only the UI needs to convert to and from glyphs (or audio...) and there it should try and avoid ambiguity. It it's open to debate how you police non-UI filename generation and potential spoofing. (See e.g. RFC 3490).

Files should have multiple identities; their 'raw bytes' canonical name (always) and (e.g.) their 'english label' and their 'chinese label'. If, in (e.g.) 'english label view' two label look the same (i.e. the names differ only by case), a visual alert would be appropriate.

The problem here is that wsl assumes raw bytes <=> english label, while Windows doesn't allow two labels that look the same.

[therealkenc]

Mercy "discussion ran course". Arguably by-design per:

...using "case=off" is the plan for the foreseeable future.

[MartyLake]

Struggled a lot to pinpoint the issue, from now on, I'm just going to avoid using git with WSL, and use git-sdk instead. (I'll keep using WSL for everything else though).