New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Troubles with GetContentFiles on Linux and MacOS (NuGet2) #4509

Closed
AndreyAkinshin opened this Issue Feb 5, 2017 · 5 comments

Comments

Projects
None yet
3 participants
@AndreyAkinshin

AndreyAkinshin commented Feb 5, 2017

In the NuGet2 source code, there are many places where Path.DirectorySeparatorChar is used. Examples:

public static class PackageExtensions
{
    public static IEnumerable<IPackageFile> GetFiles(this IPackage package, string directory)
    {
        string folderPrefix = directory + Path.DirectorySeparatorChar; // !!!
        return package.GetFiles().Where(file => file.Path.StartsWith(folderPrefix, StringComparison.OrdinalIgnoreCase));
    }  
}
public static FrameworkName ParseFrameworkNameFromFilePath(string filePath, out string effectivePath)
{
    var knownFolders = new string[]
    {
        Constants.ContentDirectory,
        Constants.LibDirectory,
        Constants.ToolsDirectory,
        Constants.BuildDirectory
    };
    for (int i = 0; i < knownFolders.Length; i++)
    {
        string folderPrefix = knownFolders[i] + System.IO.Path.DirectorySeparatorChar; // !!!
        if (filePath.Length > folderPrefix.Length &&
            filePath.StartsWith(folderPrefix, StringComparison.OrdinalIgnoreCase))
        {
            string frameworkPart = filePath.Substring(folderPrefix.Length);
            try
            {
                return VersionUtility.ParseFrameworkFolderName(
                    frameworkPart,
                    strictParsing: knownFolders[i] == Constants.LibDirectory,
                    effectivePath: out effectivePath);
            }
            catch (ArgumentException)
            {
                // if the parsing fails, we treat it as if this file
                // doesn't have target framework.
                effectivePath = frameworkPart;
                return null;
            }
        }
    }
    effectivePath = filePath;
    return null;
}

However, the actual nuspec file paths often use Windows path separator \. So, you can't use Path.DirectorySeparatorChar in the file.Path.StartsWith expressions, it doesn't work on Linux and MacOS.

I found 53 usages of Path.DirectorySeparatorChar in the source code:

Search target
    System.IO.Path.DirectorySeparatorChar:char
Found usages  (53 usages found)
    <tests>  (6 usages found)
        <Core.Test>  (2 usages found)
            PathUtilityTest.cs  (2 usages found)
                NuGet.Test  (2 usages found)
                    PathUtilityTest  (2 usages found)
                        EnsureTrailingSlashAppendsSlashIfPathDoesNotTerminateInSlash()  (2 usages found)
                            (57: 39) Assert.Equal(path1 + Path.DirectorySeparatorChar, output1);
                            (58: 39) Assert.Equal(path2 + Path.DirectorySeparatorChar, output2);
        <Test.Integration>  (3 usages found)
            NuGetCommandLine  (3 usages found)
                NuGetDeleteCommandTest.cs  (1 usage found)
                    NuGet.Test.Integration.NuGetCommandLine  (1 usage found)
                        NuGetDeleteCommandTest  (1 usage found)
                            DeleteCommand_DeleteFromFileSystemSourceUnixStyle()  (1 usage found)
                                (51: 42) source = source.Replace(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar);
                NuGetPushCommandTest.cs  (2 usages found)
                    NuGet.Test.Integration.NuGetCommandLine  (2 usages found)
                        NuGetPushCommandTest  (2 usages found)
                            PushCommand_PushToFileSystemSourceUncStyle()  (1 usage found)
                                (89: 22) if (Path.DirectorySeparatorChar == '/')
                            PushCommand_PushToFileSystemSourceUnixStyle()  (1 usage found)
                                (59: 42) source = source.Replace(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar);
        <Test.Utility>  (1 usage found)
            PathFixUtility.cs  (1 usage found)
                NuGet.Test.Utility  (1 usage found)
                    PathFixUtility  (1 usage found)
                        FixPath(string)  (1 usage found)
                            (24: 85) return String.IsNullOrWhiteSpace(path) ? path : path.Replace('\\', Path.DirectorySeparatorChar);
    <CommandLine>  (7 usages found)
        Commands  (7 usages found)
            InstallCommand.cs  (1 usage found)
                NuGet.Commands  (1 usage found)
                    InstallCommand  (1 usage found)
                        ResolveInstallPath()  (1 usage found)
                            (103: 88) var solutionSettingsFile = Path.Combine(SolutionDirectory.TrimEnd(Path.DirectorySeparatorChar), NuGetConstants.NuGetSolutionSettingsFolder);
            PackCommand.cs  (3 usages found)
                NuGet.Commands  (3 usages found)
                    PackCommand  (3 usages found)
                        ResolvePath(IPackageFile)  (3 usages found)
                            (247: 60) int index = path.IndexOf(BasePath.TrimEnd(Path.DirectorySeparatorChar) + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase);
                            (247: 91) int index = path.IndexOf(BasePath.TrimEnd(Path.DirectorySeparatorChar) + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase);
                            (252: 79) path = path.Substring(index + BasePath.Length).TrimStart(Path.DirectorySeparatorChar);
            ProjectFactory.cs  (2 usages found)
                NuGet.Commands  (2 usages found)
                    ProjectFactory  (2 usages found)
                        AddSolutionDir()  (1 usage found)
                            (712: 33) solutionDir += Path.DirectorySeparatorChar;
                        Normalize(string)  (1 usage found)
                            (970: 89) return fullPath.Substring(_project.DirectoryPath.Length).TrimStart(Path.DirectorySeparatorChar);
            PushCommand.cs  (1 usage found)
                NuGet.Commands  (1 usage found)
                    PushCommand  (1 usage found)
                        EnsurePackageExtension(string)  (1 usage found)
                            (171: 54) packagePath = packagePath + Path.DirectorySeparatorChar + '*';
    <Core>  (30 usages found)
        Analysis  (6 usages found)
            Rules  (6 usages found)
                InvalidFrameworkFolderRule.cs  (1 usage found)
                    NuGet.Analysis.Rules  (1 usage found)
                        InvalidFrameworkFolderRule  (1 usage found)
                            Validate(IPackage)  (1 usage found)
                                (19: 50) string[] parts = path.Split(Path.DirectorySeparatorChar);
                MisplacedAssemblyRule.cs  (2 usages found)
                    NuGet.Analysis.Rules  (2 usages found)
                        MisplacedAssemblyRule  (2 usages found)
                            Validate(IPackage)  (2 usages found)
                                (27: 78) else if (!directory.StartsWith(Constants.LibDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase)
                                (28: 82) && !directory.StartsWith(Constants.AnalyzersDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase))
                MisplacedScriptFileRule.cs  (1 usage found)
                    NuGet.Analysis.Rules  (1 usage found)
                        MisplacedScriptFileRule  (1 usage found)
                            Validate(IPackage)  (1 usage found)
                                (23: 70) if (!path.StartsWith(Constants.ToolsDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase))
                MisplacedTransformFileRule.cs  (2 usages found)
                    NuGet.Analysis.Rules  (2 usages found)
                        MisplacedTransformFileRule  (2 usages found)
                            Validate(IPackage)  (2 usages found)
                                (29: 72) if (!path.StartsWith(Constants.ContentDirectory + Path.DirectorySeparatorChar,
                                (31: 80) && !path.StartsWith(Constants.ContentFilesDirectory + Path.DirectorySeparatorChar,
        Authoring  (10 usages found)
            PackageBuilder.cs  (5 usages found)
                NuGet  (5 usages found)
                    PackageBuilder  (5 usages found)
                        HasContentFilesV2(ICollection<IPackageFile>)  (1 usage found)
                            (345: 77) file.Path.StartsWith(Constants.ContentFilesDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase));
                        HasXdtTransformFile(ICollection<IPackageFile>)  (1 usage found)
                            (359: 72) file.Path.StartsWith(Constants.ContentDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase) &&
                        RequiresV4TargetFrameworkSchema(ICollection<IPackageFile>)  (3 usages found)
                            (324: 75) (f.Path.StartsWith(Constants.ContentDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase) ||
                            (325: 73) f.Path.StartsWith(Constants.ToolsDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase)));
                            (335: 70) f.Path.StartsWith(Constants.LibDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase) &&
            PathResolver.cs  (5 usages found)
                NuGet  (5 usages found)
                    PathResolver  (5 usages found)
                        GetPathToEnumerateFrom(string, string)  (1 usage found)
                            (170: 76) int directorySeparatoryIndex = searchPath.LastIndexOf(Path.DirectorySeparatorChar, wildcardIndex);
                        IsDirectoryPath(string)  (1 usage found)
                            (247: 48) (path[path.Length - 1] == Path.DirectorySeparatorChar ||
                        PerformWildcardSearchInternal(string, string, bool, out string)  (1 usage found)
                            (108: 55) searchPath = searchPath + "**" + Path.DirectorySeparatorChar + "*";
                        ResolvePackagePath(string, string, string, string)  (1 usage found)
                            (203: 89) packagePath = fullPath.Substring(searchDirectory.Length).TrimStart(Path.DirectorySeparatorChar);
                        WildcardToRegex(string)  (1 usage found)
                            (51: 22) if (Path.DirectorySeparatorChar == '/')
        Configuration  (2 usages found)
            Settings.cs  (2 usages found)
                NuGet  (2 usages found)
                    Settings  (2 usages found)
                        LoadMachineWideSettings(IFileSystem, params string[])  (1 usage found)
                            (224: 59) int index = combinedPath.LastIndexOf(Path.DirectorySeparatorChar);
                        ResolvePath(string, string)  (1 usage found)
                            (278: 70) if (root != null && root.Length == 1 && (root[0] == Path.DirectorySeparatorChar || value[0] == Path.AltDirectorySeparatorChar))
        Extensions  (4 usages found)
            FileSystemExtensions.cs  (1 usage found)
                NuGet  (1 usage found)
                    FileSystemExtensions  (1 usage found)
                        GetDirectories(string)  (1 usage found)
                            (201: 57) foreach (var index in IndexOfAll(path, Path.DirectorySeparatorChar))
            PackageExtensions.cs  (2 usages found)
                NuGet  (2 usages found)
                    PackageExtensions  (2 usages found)
                        GetFiles(this IPackage, string)  (1 usage found)
                            (66: 52) string folderPrefix = directory + Path.DirectorySeparatorChar;
                        GetSatelliteFiles(this IPackage)  (1 usage found)
                            (119: 100) return package.GetLibFiles().Where(file => Path.GetDirectoryName(file.Path).Split(Path.DirectorySeparatorChar)
            SettingsExtensions.cs  (1 usage found)
                NuGet  (1 usage found)
                    SettingsExtensions  (1 usage found)
                        GetRepositoryPath(this ISettings)  (1 usage found)
                            (14: 57) path = path.Replace('/', System.IO.Path.DirectorySeparatorChar);
        Packages  (1 usage found)
            LocalPackage.cs  (1 usage found)
                NuGet  (1 usage found)
                    LocalPackage  (1 usage found)
                        IsAssemblyReference(string)  (1 usage found)
                            (254: 68) if (!filePath.StartsWith(Constants.LibDirectory + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase))
        ProjectSystem  (1 usage found)
            PhysicalFileSystem.cs  (1 usage found)
                NuGet  (1 usage found)
                    PhysicalFileSystem  (1 usage found)
                        MakeRelativePath(string)  (1 usage found)
                            (292: 67) return fullPath.Substring(Root.Length).TrimStart(Path.DirectorySeparatorChar);
        Utility  (6 usages found)
            PathUtility.cs  (2 usages found)
                NuGet  (2 usages found)
                    PathUtility  (2 usages found)
                        EnsureTrailingSlash(string)  (1 usage found)
                            (24: 55) return EnsureTrailingCharacter(path, Path.DirectorySeparatorChar);
                        IsSubdirectory(string, string)  (1 usage found)
                            (18: 46) basePath = basePath.TrimEnd(Path.DirectorySeparatorChar);
            UriUtility.cs  (2 usages found)
                NuGet  (2 usages found)
                    UriUtility  (2 usages found)
                        CreatePartUri(string)  (1 usage found)
                            (29: 58) var segments = path.Split( new[] { '/', Path.DirectorySeparatorChar }, StringSplitOptions.None)
                        GetPath(Uri)  (1 usage found)
                            (23: 66) return Uri.UnescapeDataString(path.Replace('/', Path.DirectorySeparatorChar));
            VersionUtility.cs  (2 usages found)
                NuGet  (2 usages found)
                    VersionUtility  (2 usages found)
                        ParseFrameworkFolderName(string, bool, out string)  (1 usage found)
                            (837: 83) string targetFrameworkString = Path.GetDirectoryName(path).Split(Path.DirectorySeparatorChar).First();
                        ParseFrameworkNameFromFilePath(string, out string)  (1 usage found)
                            (785: 72) string folderPrefix = knownFolders[i] + System.IO.Path.DirectorySeparatorChar;
    <VisualStudio10>  (1 usage found)
        VCProjectHelper.cs  (1 usage found)
            NuGet.VisualStudio10  (1 usage found)
                VCProjectHelper  (1 usage found)
                    GetFilter(VCProject, string, bool)  (1 usage found)
                        (172: 52) string[] paths = folderPath.Split(Path.DirectorySeparatorChar);
    <VisualStudio>  (8 usages found)
        ProjectSystems  (1 usage found)
            WebSiteProjectSystem.cs  (1 usage found)
                NuGet.VisualStudio  (1 usage found)
                    WebSiteProjectSystem  (1 usage found)
                        IsUnderAppCode(string)  (1 usage found)
                            (204: 90) return PathUtility.EnsureTrailingSlash(path).StartsWith(AppCodeFolder + Path.DirectorySeparatorChar, StringComparison.OrdinalIgnoreCase);
        Utility  (3 usages found)
            PathHelper.cs  (3 usages found)
                NuGet.VisualStudio  (3 usages found)
                    PathHelper  (3 usages found)
                        SmartTruncate(string, int)  (3 usages found)
                            (30: 53) string folder = path.Split(new[] { Path.DirectorySeparatorChar }, StringSplitOptions.RemoveEmptyEntries).LastOrDefault() ?? String.Empty;
                            (32: 27) folder = Path.DirectorySeparatorChar + folder + Path.DirectorySeparatorChar;
                            (32: 66) folder = Path.DirectorySeparatorChar + folder + Path.DirectorySeparatorChar;
        ProjectExtensions.cs  (1 usage found)
            NuGet.VisualStudio  (1 usage found)
                ProjectExtensions  (1 usage found)
                    PathSeparatorChars  (1 usage found)
                        (67: 74) private static readonly char[] PathSeparatorChars = new[] { Path.DirectorySeparatorChar };
        RepositorySettings.cs  (1 usage found)
            NuGet.VisualStudio  (1 usage found)
                RepositorySettings  (1 usage found)
                    GetRepositoryPathFromConfig(string)  (1 usage found)
                        (221: 71) repositoryPath = repositoryPath.Replace('/', Path.DirectorySeparatorChar);
        VsWebsiteHandler.cs  (2 usages found)
            NuGet.VisualStudio  (2 usages found)
                VsWebsiteHandler  (1 usage found)
                    GetAssemblyReferences(IFileSystem, string, SemanticVersion, out string)  (1 usage found)
                        (134: 132) .Select(assembly => new FileAssemblyReference(assembly.Substring(packageName.Length).Trim(Path.DirectorySeparatorChar)));
                VsWebsiteHandler.FileAssemblyReference  (1 usage found)
                    FileAssemblyReference(string)  (1 usage found)
                        (185: 115) string pathExcludeLib = assemblyPath.Substring(Constants.LibDirectory.Length).Trim(System.IO.Path.DirectorySeparatorChar);
    <VsEvents>  (1 usage found)
        VsUtility.cs  (1 usage found)
            NuGet.VisualStudio  (1 usage found)
                VsUtility  (1 usage found)
                    GetProperName(this Project)  (1 usage found)
                        (330: 97) if (projectPath.Length > 0 && projectPath[projectPath.Length-1] == Path.DirectorySeparatorChar)

It seems that many of these usages (but not all of them) should be rewritten.

See also: http://aakinshin.net/en/blog/dotnet/nuget2-and-directoryseparatorchar/

@emgarten

This comment has been minimized.

Show comment
Hide comment
@emgarten

emgarten Feb 6, 2017

Collaborator

NuGet2 is legacy at this point, and it was never fully updated to work cross platform.

The newer NuGet.Frameworks and NuGet.Packaging libraries are tested on both Linux and OSX, try using those to read content files from a nupkg.

Collaborator

emgarten commented Feb 6, 2017

NuGet2 is legacy at this point, and it was never fully updated to work cross platform.

The newer NuGet.Frameworks and NuGet.Packaging libraries are tested on both Linux and OSX, try using those to read content files from a nupkg.

@AndreyAkinshin

This comment has been minimized.

Show comment
Hide comment
@AndreyAkinshin

AndreyAkinshin Feb 6, 2017

NuGet2 is legacy at this point, and it was never fully updated to work cross platform.

@emgarten, yeah, I know that. But we have a huge amount of legacy code too. And we use NuGet.Client 4.x for new code. However, it's hard to update all our codebase to the new API at once.

AndreyAkinshin commented Feb 6, 2017

NuGet2 is legacy at this point, and it was never fully updated to work cross platform.

@emgarten, yeah, I know that. But we have a huge amount of legacy code too. And we use NuGet.Client 4.x for new code. However, it's hard to update all our codebase to the new API at once.

@rrelyea rrelyea modified the milestones: Future-1, Future-2 Feb 8, 2017

@rrelyea

This comment has been minimized.

Show comment
Hide comment
@rrelyea

rrelyea Feb 8, 2017

Contributor

Unlikely to invest in, but placing in future for consideration.

Contributor

rrelyea commented Feb 8, 2017

Unlikely to invest in, but placing in future for consideration.

@emgarten

This comment has been minimized.

Show comment
Hide comment
@emgarten

emgarten Feb 8, 2017

Collaborator

Pull Requests are welcome for this

Collaborator

emgarten commented Feb 8, 2017

Pull Requests are welcome for this

@emgarten emgarten modified the milestones: Future-2, 4.6 Nov 9, 2017

@emgarten

This comment has been minimized.

Show comment
Hide comment
@emgarten

emgarten Nov 9, 2017

Collaborator

There are no plans to update NuGet 2.x at this time. Users should move to the 4.x packages.

Collaborator

emgarten commented Nov 9, 2017

There are no plans to update NuGet 2.x at this time. Users should move to the 4.x packages.

@emgarten emgarten closed this Nov 9, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment