archive/tar: write too long when calling crane export on mcr.microsoft.com/dotnet on windows

[ondrej-smola]

Describe the bug

export fails for mcr.microsoft.com/dotnet images

To Reproduce

crane export mcr.microsoft.com/dotnet/aspnet:6.0.5-focal

Expected behavior

should export without failing

Additional context

only reproducible on windows 10

[imjasonh]

I'm not able to reproduce this from macOS. crane export takes about 30s, but completes normally after a few runs.

I'm not sure why this would only manifest on Windows, as far as I know the underlying code to fetch manifests and blobs and un-tar them should be pure Go code, which should insulate us from platform-specific issues. Seems that's not the case though.

Can you try with --verbose to try to get an idea where the export is failing?

edit: And does it happen with any other images, when exporting on Windows?

[ondrej-smola]

Thank you for your quick response @imjasonh , It works also on linux ... it just fails on any Windows 10 machine I tried.
Logs attached for
crane export mcr.microsoft.com/dotnet/aspnet:6.0.5-focal aspnet.tar --verbose 2> log.txt

log.txt

[ondrej-smola]

I tried to go through code but also did not find any platform dependent code - this is why it really puzzled me :)

[ondrej-smola]

Same call from linux ... both go 1.18.4 and crane latest release
out_linux.txt

No it does not happend on other images I tested (nginx, internal ...)

[imjasonh]

https://stackoverflow.com/questions/38454850/getting-write-too-long-error-when-trying-to-create-tar-gz-file-from-file-and-d seems like it might be related.

However, we aren't using FileInfoHeader, we're just pulling the tar header directly from the tar blob in the registry, and passing it down. We seem to be hitting this on the very first layer we try to extract from the image.

https://github.com/google/go-containerregistry/blob/v0.10.0/pkg/v1/mutate/mutate.go#L300

Is there any image you can successfully crane export on Windows? That might help us figure out what's different about the tar contents of this image, and make us more resilient to handling that case.

[jonjohnsonjr]

I'm not sure what the equivalent of tar -tvf - is on windows, but can you do something like:

crane export mcr.microsoft.com/dotnet/aspnet:6.0.5-focal | tar -tvf -

So we can see what file is failing?

[ondrej-smola]

Error: archive/tar: write too long
drwxr-xr-x  0 0      0           0 kvě 31 17:43 usr
drwxr-xr-x  0 0      0           0 čvn 07 02:33 usr\\share
drwxr-xr-x  0 0      0           0 čvn 07 02:33 usr\\share\\dotnet
drwxrwxr-x  0 0      0           0 čvn 07 02:33 usr\\share\\dotnet\\shared
drwxr-xr-x  0 0      0           0 čvn 07 02:33 usr\\share\\dotnet\\shared\\Microsoft.AspNetCore.App
drwxrwxr-x  0 0      0           0 dub 18 14:03 usr\\share\\dotnet\\shared\\Microsoft.AspNetCore.App\\6.0.5
-rw-rw-r--  0 0      0          47 dub 18 14:02 usr\\share\\dotnet\\shared\\Microsoft.AspNetCore.App\\6.0.5\\.version
tar.exe: Damaged tar archive
tar.exe: Retrying...

Tar is present on Windows 10 so I could run exact command as posted

[ondrej-smola]

@imjasonh thanks for pointing me towards good direction ... https://github.com/google/go-containerregistry/blob/v0.10.0/pkg/v1/mutate/mutate.go#L300 is actually root of the problem as it does not handler error returned from tarWriter.WriteHeader(header) - when I logged the error I got

archive/tar: cannot encode header: Format specifies USTAR; and USTAR cannot encode Name="usr\\share\\dotnet\\shared\\Microsoft.AspNetCore.App\\6.0.5\\Microsoft.AspNetCore.Authentication.Abstractions.dll"

[ondrej-smola]

Related docker/engine#167

len("usr\share\dotnet\shared\Microsoft.AspNetCore.App\6.0.5\.version") == 69
len("usr\share\dotnet\shared\Microsoft.AspNetCore.App\6.0.5\Microsoft.AspNetCore.Authentication.Abstractions.dll") == 113

[ondrej-smola]

I am not expert in tar formats but can we take same direction as docker/engine? Options here https://serverfault.com/questions/250511/which-tar-file-format-should-i-use-it

Same logic also used here https://github.com/google/go-containerregistry/blob/v0.11.0/internal/windows/windows.go#L80 ?