Packages can't be retrieved from restored packages folders due to unexpected letter casing in nuspec file

[bhaeussermann]

NuGet Product Used

NuGet SDK

Product Version

NuGet.Commands package version 6.8.0

Worked before?

No response

Impact

It's more difficult to complete my work

Repro Steps & Context

I need to restore NuGet packages to folders via a Linux Docker container (of a Docker image nuget-restore) and include these folders in a second Docker image nuget-get-packages in order to load the packages from within a container of this image.

However, the casing of the restored nuspec file differs from what is expected by the LocalFolderUtility.GetPackagesV3() method and so the package is not loaded by that method.

When I perform either the package restore or get-packages operation directly on my Windows PC then getting the packages from the folders works fine (presumably due to file paths being case-insensitive in Windows). Therefore, the minimal reproduction of this issue for me is to perform both operations via Docker containers (I have Docker Desktop installed which is set to use Linux containers).

Here's my program for the first Docker image nuget-restore:

using NuGet.Commands;
using NuGet.Common;
using NuGet.Configuration;
using NuGet.Frameworks;
using NuGet.LibraryModel;
using NuGet.Packaging;
using NuGet.Packaging.Core;
using NuGet.ProjectModel;
using NuGet.Protocol;
using NuGet.Protocol.Core.Types;
using NuGet.Versioning;
using System.Runtime.Versioning;

const string packageFolderPath = @"/packages";
Directory.CreateDirectory(packageFolderPath);

await RestoreAsync(packageFolderPath);

static async Task RestoreAsync(string packageFolderPath)
{
    var packages = new[] { new PackageIdentity("NuGet.Commands", new NuGetVersion(6, 8, 0)) };

    var frameworkName = new FrameworkName(".NETCoreApp,Version=v6.0");
    var targetFramework = new NuGetFramework(frameworkName.Identifier, frameworkName.Version, string.Empty, new Version());
    var machineWideSettings = new MachineWideSettings();
    var defaultSettings = Settings.LoadDefaultSettings(null, null, machineWideSettings);

    using var sourceCacheContext = new SourceCacheContext();

    var restoreArgs = new RestoreArgs
    {
        AllowNoOp = true,
        CacheContext = sourceCacheContext,
        CachingSourceProvider = new CachingSourceProvider(new PackageSourceProvider(defaultSettings)),
        ConfigFile = null,
        DisableParallel = false,
        Runtimes = new HashSet<string>(),
        FallbackRuntimes = new HashSet<string>(),
        RequestProviders = new List<IRestoreRequestProvider>(),
        Sources = new List<string>(),
        GlobalPackagesFolder = null,
        HideWarningsAndErrors = false,
        Inputs = new List<string>(),
        IsLowercaseGlobalPackagesFolder = false,
        LockFileVersion = null,
        ValidateRuntimeAssets = null,
        Log = NullLogger.Instance,
        MachineWideSettings = machineWideSettings,
        PackageSaveMode = PackageSaveMode.Defaultv3
    };
    restoreArgs.PreLoadedRequestProviders.Add(
        new DependencyGraphSpecRequestProvider(
    new RestoreCommandProvidersCache(),
    GetDependencyGraph(packages)));

    var restoreSummary = await RestoreRunner.RunAsync(restoreArgs);

    var errors = restoreSummary.SelectMany(rs => rs.Errors);
    if (errors.Any())
    {
        throw new Exception("Failed restoring packages.",
            new AggregateException(errors.Select(e => new Exception(e.Message)).ToArray()));
    }


    DependencyGraphSpec GetDependencyGraph(IEnumerable<PackageIdentity> packages)
    {
        var dependencyGraphSpec = new DependencyGraphSpec();

        foreach (var package in packages)
        {
            VersionRange versionRange = new(package.Version);
            string uniqueName = $"{package.Id}-{targetFramework}-{versionRange.ToNormalizedString()}".ToLowerInvariant();
            string projectFilePath = Path.Combine(packageFolderPath, $"{package.Id}{package.Version}.csproj");
            var packageSpec = new PackageSpec
            {
                Name = uniqueName,
                FilePath = projectFilePath,
                Dependencies = { },
                TargetFrameworks =
                {
                    new TargetFrameworkInformation
                    {
                        TargetAlias = targetFramework!.GetShortFolderName(),
                        FrameworkName = targetFramework,
                        Dependencies =
                        {
                            new LibraryDependency
                            {
                                LibraryRange = new(package.Id, versionRange, LibraryDependencyTarget.Package)
                            }
                        }
                    }
                },
                RestoreMetadata = new()
                {
                    ProjectStyle = ProjectStyle.PackageReference,
                    ProjectName = uniqueName,
                    ProjectUniqueName = uniqueName,
                    OutputPath = Path.Combine(packageFolderPath, ".restore", package.Id, package.Version.ToString()),
                    ProjectPath = projectFilePath,
                    PackagesPath = packageFolderPath,
                    FallbackFolders = { },
                    Sources = { new PackageSource("https://api.nuget.org/v3/index.json") },
                    OriginalTargetFrameworks = { targetFramework.GetShortFolderName() },
                    TargetFrameworks =
                    {
                        new ProjectRestoreMetadataFrameworkInfo
                        {
                            TargetAlias = targetFramework.GetShortFolderName(),
                            FrameworkName = targetFramework
                        }
                    },
                    ProjectWideWarningProperties = new WarningProperties()
                }
            };

            dependencyGraphSpec.AddProject(packageSpec);
            dependencyGraphSpec.AddRestore(packageSpec.Name);
        }

        return dependencyGraphSpec;
    }
}

internal class MachineWideSettings : IMachineWideSettings
{
    private readonly Lazy<ISettings> settings;

    public MachineWideSettings()
    {
        var baseDirectory = NuGetEnvironment.GetFolderPath(NuGetFolderPath.MachineWideConfigDirectory);
        this.settings = new Lazy<ISettings>(() => NuGet.Configuration.Settings.LoadMachineWideSettings(baseDirectory));
    }

    public ISettings Settings => this.settings.Value;
}

Here's the Dockerfile for creating the image nuget-restore:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
COPY NuGetPackageRestore.csproj src/NuGetPackageRestore.csproj
COPY Program.cs src/Program.cs
RUN dotnet publish --output out src

FROM mcr.microsoft.com/dotnet/runtime:6.0 AS final
WORKDIR /app
COPY --from=build out .
ENTRYPOINT ["dotnet", "NuGetPackageRestore.dll"]

Build the image:

docker build -t nuget-restore .

Run the image, binding the packages folder to a folder on the host:

docker run -it --rm --mount type=bind,source=c:\temp\packages,destination=/packages nuget-restore

Here's my program for getting the packages from the restored folders:

using NuGet.Common;
using NuGet.Protocol;

const string packageFolderPath = @"/packages";
Directory.CreateDirectory(packageFolderPath);

GetPackages(packageFolderPath);

static void GetPackages(string packageFolderPath)
{
    var packages = LocalFolderUtility.GetPackagesV3(packageFolderPath, new ConsoleLogger());
    Console.WriteLine("Packages resolved: " + packages.Count());
    Console.WriteLine(string.Join(Environment.NewLine, packages.Select(p => $"{p.Identity.Id}, {p.Identity.Version}")));
}

internal class ConsoleLogger : LoggerBase
{
    public override void Log(ILogMessage message)
    {
        Console.WriteLine($"\t{message.Level}: {message.Message}");
    }

    public override Task LogAsync(ILogMessage message)
    {
        return Task.Run(() => Log(message));
    }
}

Here's the Dockerfile for creating the image nuget-get-packages. Note the step that copies the restored packages folder into the image.

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
COPY NuGetGetPackages.csproj src/NuGetGetPackages.csproj
COPY Program.cs src/Program.cs
RUN dotnet publish --output out src

FROM mcr.microsoft.com/dotnet/runtime:6.0 AS final
COPY packages packages
WORKDIR /app
COPY --from=build out .
ENTRYPOINT ["dotnet", "NuGetGetPackages.dll"]

Move the restored packages folder into the folder of the project and build the image:

docker build -t nuget-get-packages .

Run the image:

docker run -it --rm nuget-get-packages

The program output is:

Packages resolved: 0

Looking at the log-output, there are many logs like:

Debug: Missing /packages/nuget.commands/6.8.0/nuget.commands.nuspec

Looking at the restored packages folders I note that the relevant file name isn't nuget.commands.nuspec but NuGet.Commands.nuspec and this is clearly the reason the nuspec file isn't found and getting the packages fails.

Verbose Logs

No response

[jeffkl]

@bhaeussermann are you able to debug the code to figure out what's going on? If so, we'd accept a pull request if its an easy fix.

[bhaeussermann]

@jeffkl As far as I can tell the issue lies with the .nuspec file being generated with a different letter casing than what is expected when loading the packages. It is quite odd that when restoring from the Docker container the file is written as NuGet.Commands.nuspec, but when restoring from the Windows host it is written as nuget.commands.nuspec and there's no issue.

I found an effective work-around is to rename all .nuspec files to lower case after restoring the packages.

I'm not sure where to look at the code to troubleshoot. Can you tell me where in the code it is generating that .nuspec file?

[nkolev92]

@bhaeussermann

You're setting: IsLowercaseGlobalPackagesFolder = false in RestoreArgs.

[nkolev92]

I'll close this, but let us know if you're still having problems after removing that.