-
Notifications
You must be signed in to change notification settings - Fork 252
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cross-platform glob pattern and path issues #3584
Comments
We were using a windows path seperator when setting the TargetPath metadata on source files we added to the ItemGroup of artifacts to pack. This was causing the logic that splits the nuget package into a product and symbols package to place the source assets in the product package in an oddly named 'src\src" directory. The NuGet logic around dealing with path seperators cross platform has known issues (e.g. NuGet/Home#3584). For now, we will use the correct path seperator when adding the metadata. This seems to be the cleanest solution (I considered changing the path seperators in the GenerateNuSpec target, but I worry about doing path normalization there). Contributes to dotnet/corefx#12758. This actually fixes the issue, but of course they will need to tae buildtools update.
We were using a windows path seperator when setting the TargetPath metadata on source files we added to the ItemGroup of artifacts to pack. This was causing the logic that splits the nuget package into a product and symbols package to place the source assets in the product package in an oddly named 'src\src" directory. The NuGet logic around dealing with path seperators cross platform has known issues (e.g. NuGet/Home#3584). For now, we will use the correct path seperator when adding the metadata. This seems to be the cleanest solution (I considered changing the path seperators in the GenerateNuSpec target, but I worry about doing path normalization there). Contributes to dotnet/corefx#12758. This actually fixes the issue, but of course they will need to tae buildtools update.
Adding another issue to the list: Here's the description from there: Globbing patterns in nuspec using forward slashes do not match files. The same pattern with backslashes works. DetailsUsing NuGet.exe 3.5.0.1938 on Windows 10 Repro
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
<metadata>
<id>TestPackage</id>
<version>1.0.0</version>
<authors>name</authors>
<description>description</description>
</metadata>
<files>
<file src="folder/*.txt" target="dir1/" />
<file src="folder\*.txt" target="dir2/" />
</files>
</package>
Expected resultNuGet package contains:
Actual resultThe globbing pattern for
|
Threw in a couple of pack issues with forward/backward slashes. |
Added all related issues I could find that deal with paths cross-plat both pack and push related ones. |
The glob syntax doesn't seem to be documented anywhere. Would be nice to add this to the TODO list. In particular I'd love to know how to negate a match e.g. |
@RehanSaeed Let me know if you ever discovered the inner secrets of how globbing works. Running into an issue with One possible solution seems to be to disable auto globbing altogether <PropertyGroup>
<EnableDefaultContentItems>false</EnableDefaultContentItems>
</PropertyGroup> But that doesn't quite work for my scenario. |
NuGet v In v <file src="repro/dir/**" target="/" /> In Here is repro script (PowerShell). It'll download NuGet v5/6 binaries, create nuspec, package directory structure and 2 packages from the same nuspec using different NuGet versions. $repro_path = "$PSScriptRoot/repro/dir"
$nuspec_path = "$PSScriptRoot/repro.nuspec"
if (
-not (Test-Path -LiteralPath $repro_path -PathType Container)
) {
New-Item -Path $repro_path -ItemType Directory > $null
}
'repro' | Out-File -LiteralPath "$repro_path/repro.txt" -Encoding ascii
@'
<?xml version="1.0"?>
<package >
<metadata>
<id>Repro</id>
<description>repro</description>
<version>0.0.0</version>
<authors>repro</authors>
</metadata>
<files>
<file src="repro/dir/**" target="/" />
</files>
</package>
'@ | Out-File -LiteralPath $nuspec_path -Encoding ascii
'5.11.0', '6.0.0' | ForEach-Object {
$nuget_binary = "$PSScriptRoot/nuget.$_.exe"
if (
-not (Test-Path -LiteralPath $nuget_binary -PathType Leaf)
) {
Invoke-WebRequest -UseBasicParsing -Uri (
"https://dist.nuget.org/win-x86-commandline/v$_/nuget.exe"
) -OutFile $nuget_binary
}
& $nuget_binary @(
'pack'
'-Version'
$_
$nuspec_path
)
} |
Just to add on to my previous reply, I figured out what my problem was. Up until a recent version of NuGet, it was possible for a nuspec file to have duplicate or overlapping mapping rules. NU5050 was added to prevent this. While Zip files allow two files with the same name to exist in the same zip file, the behavior when unzipping those files is undefined. Thus, you could unpack a nuget package and whichever one of those two files could "win". In most cases, these files MIGHT even be identical values IF there upstream source was identical. While created with good intentions, this Nu5050 "feature" created a situation where, if you wanted to pack a Web site project using a custom nuspec, you needed some really verbose nuspec rules: <files>
<file src="bin\**\*" target="bin" />
<!-- We need to exclude the bin directory contents from each of the following includes to avoid NU5050 duplicate error. -->
<file src="**\*.ascx" exclude="bin\**\*" />
<file src="**\*.asmx" exclude="bin\**\*" />
<file src="**\*.aspx" exclude="bin\**\*" />
<file src="**\*.config" exclude="bin\**\*" />
<file src="**\*.cshtml" exclude="bin\**\*" />
<file src="**\*.css" exclude="bin\**\*" />
<file src="**\*.eot" exclude="bin\**\*" />
<file src="**\*.gif" exclude="bin\**\*" />
<file src="**\*.jpg" exclude="bin\**\*" />
<file src="**\*.js" exclude="bin\**\*" />
<file src="**\*.map" exclude="bin\**\*" />
<file src="**\*.master" exclude="bin\**\*" />
<file src="**\*.png" exclude="bin\**\*" />
<file src="**\*.sass" exclude="bin\**\*" />
<file src="**\*.scss" exclude="bin\**\*" />
<file src="**\*.skin" exclude="bin\**\*" />
<file src="**\*.svg" exclude="bin\**\*" />
<file src="**\*.ttf" exclude="bin\**\*" />
<file src="**\*.woff" exclude="bin\**\*" />
<file src="**\*.woff2" exclude="bin\**\*" />
<file src="*.asax" exclude="bin\**\*" />
<file src="*.htm" exclude="bin\**\*" />
<file src="*.xml" exclude="bin\**\*" />
<file src="*.sitemap" exclude="bin\**\*" />
</files> Unrelated, it also created a situation where you can't pack a .NET SDK csproj if the .NET SDK csproj contains no packable items. In other words, you can't use the special MSBuild csproj SDK types, like NoBuild, to create packaging of arbitrary content defined outside the csproj. Also, as far as I can tell, there is no way to inject fakes for the packable content so that MSBuild just passes that along to the nuspec. Also, if the MSBuild packable content also is covered by a nuspec wild card, the two will collide but the error reported by NU5050 isnt great. |
File separator handling in globbing expressions should really be documented and fixed. Bugs like the one reported by beatcracker cause issues in other projects, while the root cause is within NuGet. A portable solution (e.g., support slashes on all platforms) would be welcome. |
Just got bit by this today. Took ages to figure out that I needed to switch to backslash for Windows. Would love to have a cross-platform solution! |
@JonDouglas I dont know if this is something your team is looking into, but the general lack of documentation on this is frustrating. More and more people will get bit by this as people use Nuget 6 (upgrade to VS 2022 for example). |
Ran into this problem as well (on Windows 10 21H2). Forward slash in Changing from |
I also ran into this issue today doing a dotnet pack using a custom .nuspec file which used forward slashes in some paths (I dev on macOS), but my coworkers on windows couldn't pack the files due to them needing to use back slashes in those paths. |
NuGet/NuGet.Client#909 fixes a few issues, but there are more.
PathResolver.IsWildcardSearch(...)
only checks for presence of a star character (*
), butPathResolver.WildcardToRegex(...)
supports question mark character (?
).NuSpec examples show glob patterns with both forward slash (
/
) and backslash (\
), but it is not clear if NuGet intends to support both interchangeably on both Windows and Unix-flavored operating systems.Code to handle glob patterns/paths with forward- or backslashes is scattered around the code base (example), which makes bug fixing and testing more difficult.
dotnet pack
warnings #4596The text was updated successfully, but these errors were encountered: