Support long file names on Windows #645

Closed
JeremyKuhne opened this Issue Feb 5, 2015 · 52 comments

Comments

Projects
None yet
@JeremyKuhne
Member

JeremyKuhne commented Feb 5, 2015

Now would seem to be the time to finally tackle the MAX_PATH nightmare. With the Win32FileSystem and other abstractions we can convert incoming paths to their long path equivalent if needed before handing off to the actual Win32 APIs. It appears that all of the APIs currently in use support the extended syntax (?), with the exception of Get/SetDirectory.

@paulcbetts

This comment has been minimized.

Show comment
Hide comment
@paulcbetts

paulcbetts Feb 7, 2015

Now would seem to be the time to finally tackle the MAX_PATH nightmare

This will never happen, and should never happen. Please do not add implicit long path support to .NET, it will be a nightmare for users, who now have files that they have created in one app are inexplicably unopenable in other apps.

MAX_PATH being a #define in Win32 means that it is hard-coded into tens of thousands of applications, in a way that is almost impossible to change after-the-fact (i.e. via AppCompat shims, etc). You can't even set this in new versions of the SDK because a bunch of apps have made wire / IPC protocols which would include MAX_PATH in the definition, and now they disagree and you've broken things. Using extended syntax also bypasses all of the path correction that Win32 does and means that path semantics are now different between apps, depending on their .NET version. Attempting to "fix" MAX_PATH is simply not tenable in any sane way, especially from a non-OS component like the CLR.

Now would seem to be the time to finally tackle the MAX_PATH nightmare

This will never happen, and should never happen. Please do not add implicit long path support to .NET, it will be a nightmare for users, who now have files that they have created in one app are inexplicably unopenable in other apps.

MAX_PATH being a #define in Win32 means that it is hard-coded into tens of thousands of applications, in a way that is almost impossible to change after-the-fact (i.e. via AppCompat shims, etc). You can't even set this in new versions of the SDK because a bunch of apps have made wire / IPC protocols which would include MAX_PATH in the definition, and now they disagree and you've broken things. Using extended syntax also bypasses all of the path correction that Win32 does and means that path semantics are now different between apps, depending on their .NET version. Attempting to "fix" MAX_PATH is simply not tenable in any sane way, especially from a non-OS component like the CLR.

@smbecker

This comment has been minimized.

Show comment
Hide comment
@smbecker

smbecker Feb 7, 2015

The reality is that you already have tons if files that are unopenable by .Net applications because the framework doesn't let you bypass MAX_PATH. Or you have to resort to rewriting System.IO classes using P/Invoke like in the case of AlphaFS. This should be possible to enable long path support from within .Net. Should it be implicit? I could be convinced either way. However, it is possible in Windows and should at least be possible in .Net.

smbecker commented Feb 7, 2015

The reality is that you already have tons if files that are unopenable by .Net applications because the framework doesn't let you bypass MAX_PATH. Or you have to resort to rewriting System.IO classes using P/Invoke like in the case of AlphaFS. This should be possible to enable long path support from within .Net. Should it be implicit? I could be convinced either way. However, it is possible in Windows and should at least be possible in .Net.

@Alexx999

This comment has been minimized.

Show comment
Hide comment
@Alexx999

Alexx999 Feb 7, 2015

Collaborator

@paulcbetts I believe that this change is really needed because backwards-compatibility with some ancient software is actually not a goal. Even worse, if this kind of backwards compatibility is enforced in such ways.
.NET should not enforce MAX_PATH if this is not required. It's legacy platform-specific stuff that shouldn't be present in modern platform.
If what you say is important - this should be removed from WinAPI so no one ever will create such file by accident.
This is leftover from FAT times, nothing more. And I've encountered situations where files in some directory could be read using anything (well, of tools I used at a time) but my .NET program, which was frustrating at least. (that was part of build system, and there was lots of subprojects... so paths were long).

And that posts by BCL team state that they understood that enforcing MAX_PATH is not really long-term option even back in 2007.
Now, we don't have to support Windows XP (and even no Vista support), so we can just ignore issues that are present there and look if its doable using WIn7+ APIs

Collaborator

Alexx999 commented Feb 7, 2015

@paulcbetts I believe that this change is really needed because backwards-compatibility with some ancient software is actually not a goal. Even worse, if this kind of backwards compatibility is enforced in such ways.
.NET should not enforce MAX_PATH if this is not required. It's legacy platform-specific stuff that shouldn't be present in modern platform.
If what you say is important - this should be removed from WinAPI so no one ever will create such file by accident.
This is leftover from FAT times, nothing more. And I've encountered situations where files in some directory could be read using anything (well, of tools I used at a time) but my .NET program, which was frustrating at least. (that was part of build system, and there was lots of subprojects... so paths were long).

And that posts by BCL team state that they understood that enforcing MAX_PATH is not really long-term option even back in 2007.
Now, we don't have to support Windows XP (and even no Vista support), so we can just ignore issues that are present there and look if its doable using WIn7+ APIs

@paulcbetts

This comment has been minimized.

Show comment
Hide comment
@paulcbetts

paulcbetts Feb 7, 2015

I believe that this change is really needed because backwards-compatibility with some ancient software is actually not a goal.

This is leftover from FAT times, nothing more.

The Windows 8.1 SDK defines MAX_PATH to be 260 characters. That's brand new software, written today, that has all of the issues that I described. The situation between 15 years ago when Win2k came out, and software written 10 minutes ago, is 100% identical - trying to fix this won't break ancient software, it'll break software written today too

I believe that this change is really needed because backwards-compatibility with some ancient software is actually not a goal.

This is leftover from FAT times, nothing more.

The Windows 8.1 SDK defines MAX_PATH to be 260 characters. That's brand new software, written today, that has all of the issues that I described. The situation between 15 years ago when Win2k came out, and software written 10 minutes ago, is 100% identical - trying to fix this won't break ancient software, it'll break software written today too

@Alexx999

This comment has been minimized.

Show comment
Hide comment
@Alexx999

Alexx999 Feb 7, 2015

Collaborator

Yes, Windows 8.1 SDK has MAX_PATH defined to 260.
But it doesn't make it any less legacy - .NET BCL still contains non-generic collections for the very same reason. That doesn't mean that anyone will use them (actually, I hope that no-one uses them this days).

And this doesn't mean that we shouldn't support something new only because old applications will not be able to open resulting files. Using this logic we might as well revert back to DOS and FAT16, because some old applications don't work well with Windows and non-8.3 names.

Collaborator

Alexx999 commented Feb 7, 2015

Yes, Windows 8.1 SDK has MAX_PATH defined to 260.
But it doesn't make it any less legacy - .NET BCL still contains non-generic collections for the very same reason. That doesn't mean that anyone will use them (actually, I hope that no-one uses them this days).

And this doesn't mean that we shouldn't support something new only because old applications will not be able to open resulting files. Using this logic we might as well revert back to DOS and FAT16, because some old applications don't work well with Windows and non-8.3 names.

@Yomodo

This comment has been minimized.

Show comment
Hide comment
@Yomodo

Yomodo Feb 8, 2015

Whatever you choose, we're right here:
Your GitHub neighbor: https://github.com/alphaleonis/AlphaFS

Yomodo commented Feb 8, 2015

Whatever you choose, we're right here:
Your GitHub neighbor: https://github.com/alphaleonis/AlphaFS

@smbecker

This comment has been minimized.

Show comment
Hide comment
@smbecker

smbecker Feb 8, 2015

Yes Windows defines MAX_PATH at 260 but it at least gives you a means to go beyond that if you need to and has for a number of years. Today, core .Net gives you no options but to bypass System.IO almost entirely and drop down to P/Invoke. It is pointless to debate whether or not you should be able to go beyond MAX_PATH or whether that will break backwards compatibility because Windows already decided that you can. The question then becomes "how can you access those files from within .Net?" Today you can't from the BCL. Your only option is to use something like AlphaFS, which is a great library but it solves a problem that frankly shouldn't exist in .Net.

smbecker commented Feb 8, 2015

Yes Windows defines MAX_PATH at 260 but it at least gives you a means to go beyond that if you need to and has for a number of years. Today, core .Net gives you no options but to bypass System.IO almost entirely and drop down to P/Invoke. It is pointless to debate whether or not you should be able to go beyond MAX_PATH or whether that will break backwards compatibility because Windows already decided that you can. The question then becomes "how can you access those files from within .Net?" Today you can't from the BCL. Your only option is to use something like AlphaFS, which is a great library but it solves a problem that frankly shouldn't exist in .Net.

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Feb 8, 2015

Member

To make sure I'm super clear- I'm not suggesting that .NET return ?\ paths. Just that it convert internally when talking to Win32 APIs that require it and strip the prefix off again.

I struggled with all of this years ago when I was writing the project system for Expression Blend. It was super common to get relative paths that were longer than 260 characters when combined with their base source location (notably where the project file is). While you could set the working directory, it isn't wise as any other thread can also do so (consistent bug for an inconsistent one isn't the best of trades).

I discovered that Win32's GetFullPathName could care less about the path length (at least on XP and later) and made my own call to attempt to get the path name below 260 before letting the System.IO.Path.GetFullPath method get it. (System.IO does a little extra normalizing, primarily expanding all path segments that are expandable from 8.3, even if the full path doesn't exist.)

The ability of GetFullPathName to take long strings is a part of the answer to the normalization issue mentioned above. But even if this didn't work, the normalization that gets skipped by ?\ isn't that terribly hard to replicate.

I'm happy to contribute towards snuffing out the MAX_PATH and other Path.IO related problems.
I actually spoke at some length with Kim a year or so before she wrote those posts regarding my IO struggles. It has been a long time, but I'm still anxious to see this get better. :)

Member

JeremyKuhne commented Feb 8, 2015

To make sure I'm super clear- I'm not suggesting that .NET return ?\ paths. Just that it convert internally when talking to Win32 APIs that require it and strip the prefix off again.

I struggled with all of this years ago when I was writing the project system for Expression Blend. It was super common to get relative paths that were longer than 260 characters when combined with their base source location (notably where the project file is). While you could set the working directory, it isn't wise as any other thread can also do so (consistent bug for an inconsistent one isn't the best of trades).

I discovered that Win32's GetFullPathName could care less about the path length (at least on XP and later) and made my own call to attempt to get the path name below 260 before letting the System.IO.Path.GetFullPath method get it. (System.IO does a little extra normalizing, primarily expanding all path segments that are expandable from 8.3, even if the full path doesn't exist.)

The ability of GetFullPathName to take long strings is a part of the answer to the normalization issue mentioned above. But even if this didn't work, the normalization that gets skipped by ?\ isn't that terribly hard to replicate.

I'm happy to contribute towards snuffing out the MAX_PATH and other Path.IO related problems.
I actually spoke at some length with Kim a year or so before she wrote those posts regarding my IO struggles. It has been a long time, but I'm still anxious to see this get better. :)

@ellismg ellismg added enhancement and removed suggestion labels Feb 8, 2015

@poizan42

This comment has been minimized.

Show comment
Hide comment
@poizan42

poizan42 Feb 8, 2015

Collaborator

System.IO is in mscorlib. Shouldn't this issue be tracked at CoreCLR instead?

Collaborator

poizan42 commented Feb 8, 2015

System.IO is in mscorlib. Shouldn't this issue be tracked at CoreCLR instead?

@Alexx999

This comment has been minimized.

Show comment
Hide comment
@Alexx999

Alexx999 Feb 8, 2015

Collaborator

@poizan42 see dev guide for coreclr - it states that API review should happen in corefx, and this item definitely requires review

Collaborator

Alexx999 commented Feb 8, 2015

@poizan42 see dev guide for coreclr - it states that API review should happen in corefx, and this item definitely requires review

@poizan42

This comment has been minimized.

Show comment
Hide comment
@poizan42

poizan42 Feb 8, 2015

Collaborator

@Alexx999 well I think it says that cases which requires changes to mscorlib will be addressed on a case-by-case basis.

Collaborator

poizan42 commented Feb 8, 2015

@Alexx999 well I think it says that cases which requires changes to mscorlib will be addressed on a case-by-case basis.

@mikedn

This comment has been minimized.

Show comment
Hide comment
@mikedn

mikedn Feb 8, 2015

Collaborator

The relevant IO stuff is no longer in mscorlib for .NET Core, it's in System.IO.FileSystem and that's already part of this repository:

https://github.com/dotnet/corefx/tree/master/src/System.IO.FileSystem/src/System/IO

Collaborator

mikedn commented Feb 8, 2015

The relevant IO stuff is no longer in mscorlib for .NET Core, it's in System.IO.FileSystem and that's already part of this repository:

https://github.com/dotnet/corefx/tree/master/src/System.IO.FileSystem/src/System/IO

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Feb 8, 2015

Member

What exactly is the deal with mscorlib System.IO vs. System.IO.FileSystem? Looking at the implementation in FileSystem is what caused me to kick off this thread here. System.IO.FileSystem seems mostly isolated from mscorlib's System.IO and implements most of the functionality (GetFullPath being one of the few calls I see going to System.IO.Path).

Member

JeremyKuhne commented Feb 8, 2015

What exactly is the deal with mscorlib System.IO vs. System.IO.FileSystem? Looking at the implementation in FileSystem is what caused me to kick off this thread here. System.IO.FileSystem seems mostly isolated from mscorlib's System.IO and implements most of the functionality (GetFullPath being one of the few calls I see going to System.IO.Path).

@ericstj

This comment has been minimized.

Show comment
Hide comment
@ericstj

ericstj Feb 8, 2015

Member

MSCorlib contains the old desktop & phone implementations. We are trying to move as much as possible out of mscorlib that doesn't need to be there. Getting things out of mscorlib lets us ship them independently and enables more pay-for-play eventually when we can slim down mscorlib.

The long path work will need to touch the Path class primarily, which is still in mscorlib, but we do have plans to pull it out. I'd prefer that we wait for that to happen so that we can do this potentially destabilizing work in isolation without risking the phone & desktop platforms which ship the mscorlib implementation inbox.

This is all good discussion. I agree that we need to support long paths. It will be done and must be done in a way that doesn't break existing apps. We'll likely have an implementation that lets you put an app in "max path mode" that would put up guard rails needed if you are working with legacy components that aren't long path aware. We'd need to identify the trigger for such a mitigation (opt-in-api vs opt-out-api vs quirk) and the right scope (app, context, or instance). I'd be curious to hear folks opinion on this.

Member

ericstj commented Feb 8, 2015

MSCorlib contains the old desktop & phone implementations. We are trying to move as much as possible out of mscorlib that doesn't need to be there. Getting things out of mscorlib lets us ship them independently and enables more pay-for-play eventually when we can slim down mscorlib.

The long path work will need to touch the Path class primarily, which is still in mscorlib, but we do have plans to pull it out. I'd prefer that we wait for that to happen so that we can do this potentially destabilizing work in isolation without risking the phone & desktop platforms which ship the mscorlib implementation inbox.

This is all good discussion. I agree that we need to support long paths. It will be done and must be done in a way that doesn't break existing apps. We'll likely have an implementation that lets you put an app in "max path mode" that would put up guard rails needed if you are working with legacy components that aren't long path aware. We'd need to identify the trigger for such a mitigation (opt-in-api vs opt-out-api vs quirk) and the right scope (app, context, or instance). I'd be curious to hear folks opinion on this.

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Feb 9, 2015

Member

I'm not sure how helpful the Framework can be for interacting with components that don't handle >MAX_PATH. Ultimately there is nothing preventing you from sending a really long path to some random component. While I can imagine that IO.FileSystem throwing PathToLong might help in some cases- that seems more likely to be accidental than intentional. If callers do robust error handling on System.IO, why wouldn't they do it on other, presumably more sensitive, components?

I guess one argument would be around mitigating buffer overruns in unmanaged components. If the framework throws you're probably less likely to get to the unmanaged component. That is, of course, presuming code usually hits some System.IO API beforehand.

One thought is that having iterators (GetFiles) never return long paths might be a reasonable compat option to have? Outside of that Path.Combine and Path.GetFullPath are the most likely choke points.

Member

JeremyKuhne commented Feb 9, 2015

I'm not sure how helpful the Framework can be for interacting with components that don't handle >MAX_PATH. Ultimately there is nothing preventing you from sending a really long path to some random component. While I can imagine that IO.FileSystem throwing PathToLong might help in some cases- that seems more likely to be accidental than intentional. If callers do robust error handling on System.IO, why wouldn't they do it on other, presumably more sensitive, components?

I guess one argument would be around mitigating buffer overruns in unmanaged components. If the framework throws you're probably less likely to get to the unmanaged component. That is, of course, presuming code usually hits some System.IO API beforehand.

One thought is that having iterators (GetFiles) never return long paths might be a reasonable compat option to have? Outside of that Path.Combine and Path.GetFullPath are the most likely choke points.

@Alexx999

This comment has been minimized.

Show comment
Hide comment
@Alexx999

Alexx999 Feb 9, 2015

Collaborator

@JeremyKuhne I don't think that having restrictions on select APIs will really help - I don't see real difference between long path from iterator vs long path from some user input. It may go to some native code or be written to some location where it will cause problems anyway.
Moreover, in many cases having iterators with long path is required - for example some resource compilation when you go through directory tree and process what you find.

Collaborator

Alexx999 commented Feb 9, 2015

@JeremyKuhne I don't think that having restrictions on select APIs will really help - I don't see real difference between long path from iterator vs long path from some user input. It may go to some native code or be written to some location where it will cause problems anyway.
Moreover, in many cases having iterators with long path is required - for example some resource compilation when you go through directory tree and process what you find.

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Feb 9, 2015

Member

@Alexx999 That is sort of my main point. I don't really think restrictions would be that useful as there are no guarantees people will call them on all paths before passing them on. And, to be clear, I would not recommend any of these APIs do anything but support long filenames by default. Just brainstorming possible opt-in behaviors.

Along the lines of mitigation- Kim pointed out that the Windows shell collapses paths to short equivalents to get them to fit under 260. While that is interesting, that behavior breaks the desire for paths to be normalized. In addition, it seems that this feature can be disabled and isn't on by default for all drives. (Anyone know of any links that describes the expected behavior? It seems like only the system drive gets this turned on in Win8- or perhaps there is a drive size gate?)

I think it would be useful to provide an API that tries to create these compacted paths for legacy component compat. In addition, potentially provide APIs that help you create hard links (with shorter paths)?

Member

JeremyKuhne commented Feb 9, 2015

@Alexx999 That is sort of my main point. I don't really think restrictions would be that useful as there are no guarantees people will call them on all paths before passing them on. And, to be clear, I would not recommend any of these APIs do anything but support long filenames by default. Just brainstorming possible opt-in behaviors.

Along the lines of mitigation- Kim pointed out that the Windows shell collapses paths to short equivalents to get them to fit under 260. While that is interesting, that behavior breaks the desire for paths to be normalized. In addition, it seems that this feature can be disabled and isn't on by default for all drives. (Anyone know of any links that describes the expected behavior? It seems like only the system drive gets this turned on in Win8- or perhaps there is a drive size gate?)

I think it would be useful to provide an API that tries to create these compacted paths for legacy component compat. In addition, potentially provide APIs that help you create hard links (with shorter paths)?

@mikedn

This comment has been minimized.

Show comment
Hide comment
@mikedn

mikedn Feb 9, 2015

Collaborator

I think the compatibility concerns may be a bit exaggerated. It's already possible to create files with long paths. Granted, the mechanism that can achieve this isn't common but I done this once or twice by accident and end up with files that cannot be opened and that's simply illogical.

var longDirPath = @"D:\" + new string('x', 200);
var shortDirPath = @"D:\" + new string('x', 40);
var fileName = "filesdjhaskjdhkashdkjshdkjahsdkahskdhakjsdhkd.txt";

Directory.CreateDirectory(shortDirPath);
File.WriteAllText(Path.Combine(shortDirPath, fileName), "");

Directory.CreateDirectory(longDirPath);

// create a file with a long path...
Directory.Move(shortDirPath, Path.Combine(longDirPath, Path.GetFileName(shortDirPath)));

// ...enumeration works fine...
foreach (var filePath in Directory.EnumerateFiles(longDirPath, "*", SearchOption.AllDirectories)) {
    // .. but the file cannot be read because FileStream can't open files with long paths
    Console.WriteLine(File.ReadAllText(filePath));
}
Collaborator

mikedn commented Feb 9, 2015

I think the compatibility concerns may be a bit exaggerated. It's already possible to create files with long paths. Granted, the mechanism that can achieve this isn't common but I done this once or twice by accident and end up with files that cannot be opened and that's simply illogical.

var longDirPath = @"D:\" + new string('x', 200);
var shortDirPath = @"D:\" + new string('x', 40);
var fileName = "filesdjhaskjdhkashdkjshdkjahsdkahskdhakjsdhkd.txt";

Directory.CreateDirectory(shortDirPath);
File.WriteAllText(Path.Combine(shortDirPath, fileName), "");

Directory.CreateDirectory(longDirPath);

// create a file with a long path...
Directory.Move(shortDirPath, Path.Combine(longDirPath, Path.GetFileName(shortDirPath)));

// ...enumeration works fine...
foreach (var filePath in Directory.EnumerateFiles(longDirPath, "*", SearchOption.AllDirectories)) {
    // .. but the file cannot be read because FileStream can't open files with long paths
    Console.WriteLine(File.ReadAllText(filePath));
}
@gulbanana

This comment has been minimized.

Show comment
Hide comment
@gulbanana

gulbanana Mar 5, 2015

it's worth mentioning that the need to deal with long paths is becoming more common. for example, consider visual studio kproj (asp.net 5) projects: they have built-in npm package installation. npm uses recursive paths for dependencies, such that installing even a single package can tip you over 260 - grunt-browserify, last I checked, included long enough paths that not even storing your solution at c:\ would save you. and that's a common package.

the implications of this right now are that your build tooling cannot be written in .net if using npm packages, and the git provider for VS doesn't correctly handle the directory structure. it's still a niche use case, but it's growing, and .net will have to provide a way to do this sooner or later.

it's worth mentioning that the need to deal with long paths is becoming more common. for example, consider visual studio kproj (asp.net 5) projects: they have built-in npm package installation. npm uses recursive paths for dependencies, such that installing even a single package can tip you over 260 - grunt-browserify, last I checked, included long enough paths that not even storing your solution at c:\ would save you. and that's a common package.

the implications of this right now are that your build tooling cannot be written in .net if using npm packages, and the git provider for VS doesn't correctly handle the directory structure. it's still a niche use case, but it's growing, and .net will have to provide a way to do this sooner or later.

@nvivo

This comment has been minimized.

Show comment
Hide comment
@nvivo

nvivo Mar 7, 2015

+1 to supporting long file paths. Lots of issues dealing with node_modules folders that go well beyond 260 chars.

nvivo commented Mar 7, 2015

+1 to supporting long file paths. Lots of issues dealing with node_modules folders that go well beyond 260 chars.

@glen-84

This comment has been minimized.

Show comment
Hide comment
@glen-84

glen-84 May 7, 2015

Has any work been done on this yet?

I'm not able to open my project in VS 2015 RC:

Error 1
(note the grammatical errors as well)

Error 2

Log:

=====================
2015/05/07 14:25:45
Crippling
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: The item metadata "%(FullPath)" cannot be applied to the path "wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\". D:\Programming\Projects\ASP.NET\Apex\src\Apex.Web\wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInvalidOperation(String resourceName, Object[] args)
   at Microsoft.Build.Shared.ErrorUtilities.VerifyThrowInvalidOperation(Boolean condition, String resourceName, Object arg0, Object arg1, Object arg2)
   at Microsoft.Build.Shared.FileUtilities.ItemSpecModifiers.GetItemSpecModifier(String currentDirectory, String itemSpec, String definingProjectEscaped, String modifier, String& fullPath)
   at Microsoft.Build.Evaluation.BuiltInMetadata.GetMetadataValueEscaped(String currentDirectory, String evaluatedIncludeBeforeWildcardExpansionEscaped, String evaluatedIncludeEscaped, String definingProjectEscaped, String name, String& fullPath)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetBuiltInMetadataEscaped(String name)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadataEscaped(String metadataName)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadata(String metadataName)
   at Microsoft.VisualStudio.ProjectSystem.ProjectInstanceItemProperties.GetEvaluatedPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.PropertyPages.PageRuleBase.GetPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.GetItemMetadata(IImmutableDictionary`2 basis, IRule rule)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.BuildItemsSnapshot(IRule rule, IProjectCatalogSnapshot projectCatalogSnapshot, ImmutableDictionary`2 previousVersion, ImmutableList`1 recentItemRenames, ImmutableHashSet`1& addedItems, ImmutableHashSet`1& removedItems, ImmutableHashSet`1& changedItems, ImmutableDictionary`2& renamedItems)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.UpdateRuleSnapshotCore(IProjectVersionedValue`1 source, String ruleName, IRule rule, ImmutableDictionary`2 items, ImmutableDictionary`2 properties, ImmutableList`1 recentItemRenames, IProjectChangeDiff& diff)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectRuleSubscriptionServiceBase`1.UpdateSnapshotCoreAsync(TSource source, IImmutableSet`1 subscribedElements, IProjectVersionedValue`1 previousResult)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<UpdateSnapshotAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<>c__DisplayClass27_0.<<Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__72.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask`1.<JoinAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<<Initialize>b__27_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Utilities.DataflowExtensions.<>c__DisplayClass22_0`2.<<CreateSelfFilteringTransformBlock>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Package.UnconfiguredProjectHostBridge`3.<ApplyAsync>d__18.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: The item metadata "%(FullPath)" cannot be applied to the path "wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\". D:\Programming\Projects\ASP.NET\Apex\src\Apex.Web\wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInvalidOperation(String resourceName, Object[] args)
   at Microsoft.Build.Shared.ErrorUtilities.VerifyThrowInvalidOperation(Boolean condition, String resourceName, Object arg0, Object arg1, Object arg2)
   at Microsoft.Build.Shared.FileUtilities.ItemSpecModifiers.GetItemSpecModifier(String currentDirectory, String itemSpec, String definingProjectEscaped, String modifier, String& fullPath)
   at Microsoft.Build.Evaluation.BuiltInMetadata.GetMetadataValueEscaped(String currentDirectory, String evaluatedIncludeBeforeWildcardExpansionEscaped, String evaluatedIncludeEscaped, String definingProjectEscaped, String name, String& fullPath)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetBuiltInMetadataEscaped(String name)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadataEscaped(String metadataName)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadata(String metadataName)
   at Microsoft.VisualStudio.ProjectSystem.ProjectInstanceItemProperties.GetEvaluatedPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.PropertyPages.PageRuleBase.GetPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.GetItemMetadata(IImmutableDictionary`2 basis, IRule rule)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.BuildItemsSnapshot(IRule rule, IProjectCatalogSnapshot projectCatalogSnapshot, ImmutableDictionary`2 previousVersion, ImmutableList`1 recentItemRenames, ImmutableHashSet`1& addedItems, ImmutableHashSet`1& removedItems, ImmutableHashSet`1& changedItems, ImmutableDictionary`2& renamedItems)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.UpdateRuleSnapshotCore(IProjectVersionedValue`1 source, String ruleName, IRule rule, ImmutableDictionary`2 items, ImmutableDictionary`2 properties, ImmutableList`1 recentItemRenames, IProjectChangeDiff& diff)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectRuleSubscriptionServiceBase`1.UpdateSnapshotCoreAsync(TSource source, IImmutableSet`1 subscribedElements, IProjectVersionedValue`1 previousResult)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<UpdateSnapshotAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<>c__DisplayClass27_0.<<Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__72.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask`1.<JoinAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<<Initialize>b__27_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Utilities.DataflowExtensions.<>c__DisplayClass22_0`2.<<CreateSelfFilteringTransformBlock>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Package.UnconfiguredProjectHostBridge`3.<ApplyAsync>d__18.MoveNext()<---

===================

Can this be made a higher priority?

glen-84 commented May 7, 2015

Has any work been done on this yet?

I'm not able to open my project in VS 2015 RC:

Error 1
(note the grammatical errors as well)

Error 2

Log:

=====================
2015/05/07 14:25:45
Crippling
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: The item metadata "%(FullPath)" cannot be applied to the path "wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\". D:\Programming\Projects\ASP.NET\Apex\src\Apex.Web\wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInvalidOperation(String resourceName, Object[] args)
   at Microsoft.Build.Shared.ErrorUtilities.VerifyThrowInvalidOperation(Boolean condition, String resourceName, Object arg0, Object arg1, Object arg2)
   at Microsoft.Build.Shared.FileUtilities.ItemSpecModifiers.GetItemSpecModifier(String currentDirectory, String itemSpec, String definingProjectEscaped, String modifier, String& fullPath)
   at Microsoft.Build.Evaluation.BuiltInMetadata.GetMetadataValueEscaped(String currentDirectory, String evaluatedIncludeBeforeWildcardExpansionEscaped, String evaluatedIncludeEscaped, String definingProjectEscaped, String name, String& fullPath)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetBuiltInMetadataEscaped(String name)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadataEscaped(String metadataName)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadata(String metadataName)
   at Microsoft.VisualStudio.ProjectSystem.ProjectInstanceItemProperties.GetEvaluatedPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.PropertyPages.PageRuleBase.GetPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.GetItemMetadata(IImmutableDictionary`2 basis, IRule rule)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.BuildItemsSnapshot(IRule rule, IProjectCatalogSnapshot projectCatalogSnapshot, ImmutableDictionary`2 previousVersion, ImmutableList`1 recentItemRenames, ImmutableHashSet`1& addedItems, ImmutableHashSet`1& removedItems, ImmutableHashSet`1& changedItems, ImmutableDictionary`2& renamedItems)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.UpdateRuleSnapshotCore(IProjectVersionedValue`1 source, String ruleName, IRule rule, ImmutableDictionary`2 items, ImmutableDictionary`2 properties, ImmutableList`1 recentItemRenames, IProjectChangeDiff& diff)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectRuleSubscriptionServiceBase`1.UpdateSnapshotCoreAsync(TSource source, IImmutableSet`1 subscribedElements, IProjectVersionedValue`1 previousResult)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<UpdateSnapshotAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<>c__DisplayClass27_0.<<Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__72.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask`1.<JoinAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<<Initialize>b__27_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Utilities.DataflowExtensions.<>c__DisplayClass22_0`2.<<CreateSelfFilteringTransformBlock>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Package.UnconfiguredProjectHostBridge`3.<ApplyAsync>d__18.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: The item metadata "%(FullPath)" cannot be applied to the path "wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\". D:\Programming\Projects\ASP.NET\Apex\src\Apex.Web\wwwroot\jspm_packages\github\glen-84\apex-sui-theme@2.0.0\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInvalidOperation(String resourceName, Object[] args)
   at Microsoft.Build.Shared.ErrorUtilities.VerifyThrowInvalidOperation(Boolean condition, String resourceName, Object arg0, Object arg1, Object arg2)
   at Microsoft.Build.Shared.FileUtilities.ItemSpecModifiers.GetItemSpecModifier(String currentDirectory, String itemSpec, String definingProjectEscaped, String modifier, String& fullPath)
   at Microsoft.Build.Evaluation.BuiltInMetadata.GetMetadataValueEscaped(String currentDirectory, String evaluatedIncludeBeforeWildcardExpansionEscaped, String evaluatedIncludeEscaped, String definingProjectEscaped, String name, String& fullPath)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetBuiltInMetadataEscaped(String name)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadataEscaped(String metadataName)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadata(String metadataName)
   at Microsoft.VisualStudio.ProjectSystem.ProjectInstanceItemProperties.GetEvaluatedPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.PropertyPages.PageRuleBase.GetPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.GetItemMetadata(IImmutableDictionary`2 basis, IRule rule)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.BuildItemsSnapshot(IRule rule, IProjectCatalogSnapshot projectCatalogSnapshot, ImmutableDictionary`2 previousVersion, ImmutableList`1 recentItemRenames, ImmutableHashSet`1& addedItems, ImmutableHashSet`1& removedItems, ImmutableHashSet`1& changedItems, ImmutableDictionary`2& renamedItems)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.UpdateRuleSnapshotCore(IProjectVersionedValue`1 source, String ruleName, IRule rule, ImmutableDictionary`2 items, ImmutableDictionary`2 properties, ImmutableList`1 recentItemRenames, IProjectChangeDiff& diff)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectRuleSubscriptionServiceBase`1.UpdateSnapshotCoreAsync(TSource source, IImmutableSet`1 subscribedElements, IProjectVersionedValue`1 previousResult)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<UpdateSnapshotAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<>c__DisplayClass27_0.<<Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__72.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask`1.<JoinAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<<Initialize>b__27_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Utilities.DataflowExtensions.<>c__DisplayClass22_0`2.<<CreateSelfFilteringTransformBlock>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Package.UnconfiguredProjectHostBridge`3.<ApplyAsync>d__18.MoveNext()<---

===================

Can this be made a higher priority?

@glen-84 glen-84 referenced this issue in Microsoft/nodejstools May 22, 2015

Closed

Node/npm MAX_PATH issues #69

@JeremyKuhne JeremyKuhne assigned JeremyKuhne and unassigned ericstj Jul 17, 2015

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Jul 21, 2015

Member

I'm starting on implementing this support. My initial plan of attack is roughly the following:

1.Allow extended syntax (\?) for common file operations (file info/navigation first, then stream handling) [CoreFX]
2.Allow longer than MAX_PATH for common file operations (will also unblock Unix) [CoreFX]
3.Unblock MAX_PATH constraints with assembly and resource loading (will also unblock Unix) [CoreCLR]
4.Allow using "normal" long paths (no extended syntax needed) to avoid the need to write custom code to use long paths on Windows

I'll be opening a variety of subtasks on this over the next day or so.

Member

JeremyKuhne commented Jul 21, 2015

I'm starting on implementing this support. My initial plan of attack is roughly the following:

1.Allow extended syntax (\?) for common file operations (file info/navigation first, then stream handling) [CoreFX]
2.Allow longer than MAX_PATH for common file operations (will also unblock Unix) [CoreFX]
3.Unblock MAX_PATH constraints with assembly and resource loading (will also unblock Unix) [CoreCLR]
4.Allow using "normal" long paths (no extended syntax needed) to avoid the need to write custom code to use long paths on Windows

I'll be opening a variety of subtasks on this over the next day or so.

@be5invis

This comment has been minimized.

Show comment
Hide comment
@be5invis

be5invis Aug 4, 2015

npm/npm#3697
I am waiting powershell to support UNC and long paths natively, since the IO is done by CLR.

be5invis commented Aug 4, 2015

npm/npm#3697
I am waiting powershell to support UNC and long paths natively, since the IO is done by CLR.

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Aug 4, 2015

👍 thank (deity), finally!

whoisj commented Aug 4, 2015

👍 thank (deity), finally!

@mlidbom

This comment has been minimized.

Show comment
Hide comment
@mlidbom

mlidbom Aug 5, 2015

What will this mean for existing .net applications? Will they just start working correctly with long paths once this is implemented?

Of particular interest: What about visual studio and msbuild?

mlidbom commented Aug 5, 2015

What will this mean for existing .net applications? Will they just start working correctly with long paths once this is implemented?

Of particular interest: What about visual studio and msbuild?

@OlegKi

This comment has been minimized.

Show comment
Hide comment
@OlegKi

OlegKi Aug 5, 2015

+1 to supporting long file paths.

I want just add very simple realistic scenario, which shows how files with long path names could be created. Let us you have the following structure of directories:

C:\Users\User1
C:\Users\User2
...

and both C:\Users and C:\Users\User1 have shares. The user User1 could have his home directory connected as H: in the login script. So if we saves document with the long name under H:\ (H:\longFilename.doc for example), then the document will be saved in C:\Users\User1 which have more long file name (C:\Users\User1\longFilename.doc). If one access the same file on C:\ or if one used U: connected to the shared directory C:\Users then one can have path with is longer as MAX_PATH characters.

So one have to support paths, which are longer as MAX_PATH. In any way one have to implement such support in all server based applications, in all software which makes backup or synchronization of data, all virus scanner and so on. It would be very helpful if .Net would support \\?\" prefixes in the filenames to have 32K limit existing in the file system.

OlegKi commented Aug 5, 2015

+1 to supporting long file paths.

I want just add very simple realistic scenario, which shows how files with long path names could be created. Let us you have the following structure of directories:

C:\Users\User1
C:\Users\User2
...

and both C:\Users and C:\Users\User1 have shares. The user User1 could have his home directory connected as H: in the login script. So if we saves document with the long name under H:\ (H:\longFilename.doc for example), then the document will be saved in C:\Users\User1 which have more long file name (C:\Users\User1\longFilename.doc). If one access the same file on C:\ or if one used U: connected to the shared directory C:\Users then one can have path with is longer as MAX_PATH characters.

So one have to support paths, which are longer as MAX_PATH. In any way one have to implement such support in all server based applications, in all software which makes backup or synchronization of data, all virus scanner and so on. It would be very helpful if .Net would support \\?\" prefixes in the filenames to have 32K limit existing in the file system.

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Aug 5, 2015

Member

@mlidbom For existing .NET Desktop apps the hope is to make it "just work" for most apps. First priority, however, is getting implementation in place for .NET Core. We'll then look at exactly how and when we pull support back to the desktop.

For VS and MSBuild there may (will probably) be some work that has to be done for existing unmanaged code and P/Invokes.

Member

JeremyKuhne commented Aug 5, 2015

@mlidbom For existing .NET Desktop apps the hope is to make it "just work" for most apps. First priority, however, is getting implementation in place for .NET Core. We'll then look at exactly how and when we pull support back to the desktop.

For VS and MSBuild there may (will probably) be some work that has to be done for existing unmanaged code and P/Invokes.

@JeremyKuhne JeremyKuhne added this to the 1.0.0-rc1 milestone Aug 10, 2015

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Aug 10, 2015

Member

Extended syntax should now work throughout CoreFx (\?). 🎉 (as of #2700)

Member

JeremyKuhne commented Aug 10, 2015

Extended syntax should now work throughout CoreFx (\?). 🎉 (as of #2700)

@whoisj

This comment has been minimized.

Show comment
Hide comment

whoisj commented Aug 12, 2015

👍

@gulbanana

This comment has been minimized.

Show comment
Hide comment
@gulbanana

gulbanana Aug 18, 2015

that's great news!

that's great news!

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Aug 29, 2015

Member

Now you can use long paths without extended syntax! (as of #3001 & #3010)

Member

JeremyKuhne commented Aug 29, 2015

Now you can use long paths without extended syntax! (as of #3001 & #3010)

@be5invis

This comment has been minimized.

Show comment
Hide comment

🎉

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Aug 31, 2015

🎉 🏆

whoisj commented Aug 31, 2015

🎉 🏆

@be5invis

This comment has been minimized.

Show comment
Hide comment
@be5invis

be5invis Oct 26, 2015

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Oct 26, 2015

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

These improvements will be shipped with the next version of the .NET Framework. This has little, to nothing to do with Windows 10. Long path names have been supported on Windows since, at least, Vista -- maybe XP.

whoisj commented Oct 26, 2015

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

These improvements will be shipped with the next version of the .NET Framework. This has little, to nothing to do with Windows 10. Long path names have been supported on Windows since, at least, Vista -- maybe XP.

@poizan42

This comment has been minimized.

Show comment
Hide comment
@poizan42

poizan42 Oct 26, 2015

Collaborator

@whoisj: Long before XP... Here is an excerpt from the NT4.0 SDK:

lpFileName
Points to a null-terminated string that specifies the name of the object (file, pipe, mailslot, communications resource, disk device, console, or directory) to create or open.
If *lpFileName is a path, there is a default string size limit of MAX_PATH characters. This limit is related to how the CreateFile function parses paths.
Windows NT: You can use paths longer than MAX_PATH characters by calling the wide (W) version of CreateFile and prepending “\?\” to the path. The “\?\” tells the function to turn off path parsing. This lets you use paths that are nearly 32,000 Unicode characters long. You must use fully-qualified paths with this technique. This also works with UNC names. The “\?\” is ignored as part of the path. For example, “\?\C:\myworld\private” is seen as “C:\myworld\private”, and “\?\UNC\tom_1\hotstuff\coolapps” is seen as “\tom_1\hotstuff\coolapps”.

The documentation for CreateFile in the NT3.1 SDK does not mention any size restrictions at all, though that's probably an oversight. Nt/ZwCreateFile isn't documented in the NT3.1 WDK, so not hint there about whether the underlying system supported long paths.

Collaborator

poizan42 commented Oct 26, 2015

@whoisj: Long before XP... Here is an excerpt from the NT4.0 SDK:

lpFileName
Points to a null-terminated string that specifies the name of the object (file, pipe, mailslot, communications resource, disk device, console, or directory) to create or open.
If *lpFileName is a path, there is a default string size limit of MAX_PATH characters. This limit is related to how the CreateFile function parses paths.
Windows NT: You can use paths longer than MAX_PATH characters by calling the wide (W) version of CreateFile and prepending “\?\” to the path. The “\?\” tells the function to turn off path parsing. This lets you use paths that are nearly 32,000 Unicode characters long. You must use fully-qualified paths with this technique. This also works with UNC names. The “\?\” is ignored as part of the path. For example, “\?\C:\myworld\private” is seen as “C:\myworld\private”, and “\?\UNC\tom_1\hotstuff\coolapps” is seen as “\tom_1\hotstuff\coolapps”.

The documentation for CreateFile in the NT3.1 SDK does not mention any size restrictions at all, though that's probably an oversight. Nt/ZwCreateFile isn't documented in the NT3.1 WDK, so not hint there about whether the underlying system supported long paths.

@JeremyKuhne JeremyKuhne modified the milestones: 1.0.0-rc2, 1.0.0-rc1 Oct 26, 2015

@smbecker

This comment has been minimized.

Show comment
Hide comment
@smbecker

smbecker Oct 30, 2015

Will this be included in .Net 4.6.1?

Will this be included in .Net 4.6.1?

@weshaggard

This comment has been minimized.

Show comment
Hide comment
@weshaggard

weshaggard Oct 30, 2015

Member

@smbecker No it will not be in .NET 4.6.1 however we do plan to get it added to a future release of full .NET framework, not sure which one yet.

Member

weshaggard commented Oct 30, 2015

@smbecker No it will not be in .NET 4.6.1 however we do plan to get it added to a future release of full .NET framework, not sure which one yet.

@Ashtonian

This comment has been minimized.

Show comment
Hide comment

👍

@onovotny onovotny referenced this issue in NuGet/Home Jan 6, 2016

Closed

SemVer 2.0 #1359

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Jan 21, 2016

Member

dotnet/coreclr#1776 tracks the work for the runtime. CoreFx is fully implemented.

Member

JeremyKuhne commented Jan 21, 2016

dotnet/coreclr#1776 tracks the work for the runtime. CoreFx is fully implemented.

@kumarharsh

This comment has been minimized.

Show comment
Hide comment
@kumarharsh

kumarharsh Feb 25, 2016

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

These improvements will be shipped with the next version of the .NET Framework. This has little, to nothing to do with Windows 10. Long path names have been supported on Windows since, at least, Vista -- maybe XP.

@whoisj what people generally mean by "support for Windows 10" is that the Explorer and Powershell/Command-Prompt be able to open/delete these files or folders.

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

These improvements will be shipped with the next version of the .NET Framework. This has little, to nothing to do with Windows 10. Long path names have been supported on Windows since, at least, Vista -- maybe XP.

@whoisj what people generally mean by "support for Windows 10" is that the Explorer and Powershell/Command-Prompt be able to open/delete these files or folders.

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Feb 29, 2016

@kumarharsh, while I appreciate what you're saying. if people are asking about the Windows Explorer Shell, then they should be looking for avenues to query the owners of the Windows Explorer Shell. The CRL/NetFX developers have little information and influence in that arena.

As an aside, if CoreFX supports long file names then it will not be long before PowerShell does as well.

whoisj commented Feb 29, 2016

@kumarharsh, while I appreciate what you're saying. if people are asking about the Windows Explorer Shell, then they should be looking for avenues to query the owners of the Windows Explorer Shell. The CRL/NetFX developers have little information and influence in that arena.

As an aside, if CoreFX supports long file names then it will not be long before PowerShell does as well.

@be5invis

This comment has been minimized.

Show comment
Hide comment
@be5invis

be5invis Mar 23, 2016

@whoisj I am care about the time: Since CoreFX now supports long path, when will the change published to end users, to make at least PowerShell work? The key question is the “when”.

@whoisj I am care about the time: Since CoreFX now supports long path, when will the change published to end users, to make at least PowerShell work? The key question is the “when”.

@Jaykul

This comment has been minimized.

Show comment
Hide comment
@Jaykul

Jaykul Mar 23, 2016

As I understand it, the process for that is so complicated, that you're
never going to get an answer on a timeline.

First of all, because right now we have this awkward dual-mode .Net where
there's legacy .Net and then there's .Net Core, and products (like Asp.NET)
are *picking one *to ship against. PowerShell ships against legacy .Net,
but CoreFX is ... well, currently it's never been shipped in legacy.

So first they have to actually decide (how) to ship the new CoreFX code in
an upgrade to legacy .Net

Then they have to decide if it's a minor point release upgrade (like 4.5.1
or 4.5.2 were, versus the way 4.5 or 4.0 were)

Then they have to ship that version of .Net in Windows ... but in the
client world, that's a complete unknown: will they ever force a new
version of .Net as a Windows 10 update? That's never been done outside of a
"R2" release, as far as I know?

Then, if they decided it was a major release of .Net, you have to get a new
version of PowerShell that's compiled to target that version of .Net ...

Of course, in the meantime, on Nano Server they're shipping a Core
PowerShell which is compiled against .Net Core and might get that long path
fix incidentally, so it could just show up on Nano Server ...

Joel "Jaykul" Bennett
http://HuddledMasses.org
PowerShell MVP, Software Dev., Speaker
585-563-9632 <//1-585-563-9632> | Jaykul @ GitHub
https://github.com/jaykul | Twitter https://twitter.com/Jaykul | Linked
In http://www.linkedin.com/in/jaykul/ http://PowerShellGroup.org

On Wed, Mar 23, 2016 at 4:37 AM, Belleve Invis notifications@github.com
wrote:

@whoisj https://github.com/whoisj I am care about the time: Since
CoreFX now supports long path, when will the change published to end
users, to make at least PowerShell work
? The key question is the “when”.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#645 (comment)

Jaykul commented Mar 23, 2016

As I understand it, the process for that is so complicated, that you're
never going to get an answer on a timeline.

First of all, because right now we have this awkward dual-mode .Net where
there's legacy .Net and then there's .Net Core, and products (like Asp.NET)
are *picking one *to ship against. PowerShell ships against legacy .Net,
but CoreFX is ... well, currently it's never been shipped in legacy.

So first they have to actually decide (how) to ship the new CoreFX code in
an upgrade to legacy .Net

Then they have to decide if it's a minor point release upgrade (like 4.5.1
or 4.5.2 were, versus the way 4.5 or 4.0 were)

Then they have to ship that version of .Net in Windows ... but in the
client world, that's a complete unknown: will they ever force a new
version of .Net as a Windows 10 update? That's never been done outside of a
"R2" release, as far as I know?

Then, if they decided it was a major release of .Net, you have to get a new
version of PowerShell that's compiled to target that version of .Net ...

Of course, in the meantime, on Nano Server they're shipping a Core
PowerShell which is compiled against .Net Core and might get that long path
fix incidentally, so it could just show up on Nano Server ...

Joel "Jaykul" Bennett
http://HuddledMasses.org
PowerShell MVP, Software Dev., Speaker
585-563-9632 <//1-585-563-9632> | Jaykul @ GitHub
https://github.com/jaykul | Twitter https://twitter.com/Jaykul | Linked
In http://www.linkedin.com/in/jaykul/ http://PowerShellGroup.org

On Wed, Mar 23, 2016 at 4:37 AM, Belleve Invis notifications@github.com
wrote:

@whoisj https://github.com/whoisj I am care about the time: Since
CoreFX now supports long path, when will the change published to end
users, to make at least PowerShell work
? The key question is the “when”.


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#645 (comment)

@bugith

This comment has been minimized.

Show comment
Hide comment
@bugith

bugith Jan 6, 2017

So the key question turns in “Why so many skilled-guys work so hard wasted-efforts for... They?” Do they hope a sweet?

bugith commented Jan 6, 2017

So the key question turns in “Why so many skilled-guys work so hard wasted-efforts for... They?” Do they hope a sweet?

@ghost ghost referenced this issue in PowerShell/PowerShell Feb 2, 2017

Closed

PowerShell 6.0 Plan #3046

75 of 116 tasks complete
@fostersanders

This comment has been minimized.

Show comment
Hide comment
@fostersanders

fostersanders Feb 13, 2017

Hello,

I had the same problem. I didn´t know what to do so I searched on the internet for some solutions.

And I read about [b]Long Path Tool[/b], which is a great tool in these type of cases. :o

It worked really well. Hope it works for you too :p

Hello,

I had the same problem. I didn´t know what to do so I searched on the internet for some solutions.

And I read about [b]Long Path Tool[/b], which is a great tool in these type of cases. :o

It worked really well. Hope it works for you too :p

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Feb 13, 2017

Member

@whoisj, @be5invis, @Jaykul The anniversary update (RS1) of Windows 10 has PowerShell support for long paths. 4.6.2 is part of that release and PowerShell turns on the .NET functionality. The only caveat is that Windows long path policy needs enabled as (for now) it is off by default. I go over these desktop details in one of my blog posts.

Anything that relies on versions of CoreFX will support long paths without any policy changes or other updates to the consuming code as we do the work to make the paths work with (\\?\). We didn't do that in 4.6.2 (yet) due to constraints around Code Access Security (canonicalizing paths when you allow device paths is difficult and costly due to the need to allow for paths that don't fully exist).

Member

JeremyKuhne commented Feb 13, 2017

@whoisj, @be5invis, @Jaykul The anniversary update (RS1) of Windows 10 has PowerShell support for long paths. 4.6.2 is part of that release and PowerShell turns on the .NET functionality. The only caveat is that Windows long path policy needs enabled as (for now) it is off by default. I go over these desktop details in one of my blog posts.

Anything that relies on versions of CoreFX will support long paths without any policy changes or other updates to the consuming code as we do the work to make the paths work with (\\?\). We didn't do that in 4.6.2 (yet) due to constraints around Code Access Security (canonicalizing paths when you allow device paths is difficult and costly due to the need to allow for paths that don't fully exist).

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Feb 13, 2017

@JeremyKuhne interesting that you were looking at this issue, I was just looking too.

Have you ever look at this nice write up on long-path support and the canonicalization nightmare?

https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html

whoisj commented Feb 13, 2017

@JeremyKuhne interesting that you were looking at this issue, I was just looking too.

Have you ever look at this nice write up on long-path support and the canonicalization nightmare?

https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html

@JeremyKuhne

This comment has been minimized.

Show comment
Hide comment
@JeremyKuhne

JeremyKuhne Feb 13, 2017

Member

@whoisj No, I hadn't seen that write up- it is quite good. Thanks for sharing the link.

FYI, I did a dump of my own around that time as well:

Path Format Overview
Path Normalization
DOS to NT: A Path’s Journey

I've made the various .NET APIs much more friendly to the range of path formats. Mostly by not trying to second-guess what a "bad" path was. :)

Member

JeremyKuhne commented Feb 13, 2017

@whoisj No, I hadn't seen that write up- it is quite good. Thanks for sharing the link.

FYI, I did a dump of my own around that time as well:

Path Format Overview
Path Normalization
DOS to NT: A Path’s Journey

I've made the various .NET APIs much more friendly to the range of path formats. Mostly by not trying to second-guess what a "bad" path was. :)

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Mar 27, 2017

@JeremyKuhne I shared your with some folks looking to improve Msys, they seem to think it was rather valuable - so thanks for that!

whoisj commented Mar 27, 2017

@JeremyKuhne I shared your with some folks looking to improve Msys, they seem to think it was rather valuable - so thanks for that!

Olafski pushed a commit to Olafski/corefx that referenced this issue Jun 15, 2017

Merge pull request #645 from leecow/master
add vs4mac and vscode sections
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment