Decompress layer tarballs when when writing to docker load

[rainersigwald]

The tarballs-of-tarballs emitted by docker load have bare .tar files inside them; ours should presumably look just like that.

However, when naively doing so (7c9a270), I ran into an error:

   Test method Test.Microsoft.NET.Build.Containers.Filesystem.EndToEnd.ApiEndToEndWithLocalLoad threw exception:
System.NotSupportedException: This operation is not supported.
  Stack Trace:
      at System.IO.Compression.GZipStream.get_Length()
   at System.Formats.Tar.TarHeader.GetTotalDataBytesToWrite()
   at System.Formats.Tar.TarHeader.WriteAsPaxSharedInternal(Span`1 buffer, Int64& actualLength)
   at System.Formats.Tar.TarHeader.WriteAsPaxInternalAsync(Stream archiveStream, Memory`1 buffer, CancellationToken cancellationToken)
   at System.Formats.Tar.TarHeader.WriteAsPaxAsync(Stream archiveStream, Memory`1 buffer, CancellationToken cancellationToken)
   at System.Formats.Tar.TarWriter.WriteEntryAsyncInternal(TarEntry entry, CancellationToken cancellationToken)
   at Microsoft.NET.Build.Containers.LocalDocker.WriteImageToStream(Image x, String name, String tag, Stream imageStream) in S:\System.Containers\Microsoft.NET.Build.Containers\LocalDocker.cs:line 75
   at Microsoft.NET.Build.Containers.LocalDocker.Load(Image x, String name, String tag, String baseName) in S:\System.Containers\Microsoft.NET.Build.Containers\LocalDocker.cs:line 29
   at Test.Microsoft.NET.Build.Containers.Filesystem.EndToEnd.ApiEndToEndWithLocalLoad() in S:\System.Containers\Test.Microsoft.NET.Build.Containers.Filesystem\EndToEnd.cs:line 71

[rainersigwald]

@carlossanlop, silly question but since it's around the tar APIs I figured I'd ping you.

What I'd like to do here is stream from an on-disk .tar.gz to a single .tar entry in another tarball. Is there any nicer way to solve this than to unzip the file to memory or disk?

[carlossanlop]

I see that you are setting an unseekable stream as the tar entry's data stream. When the tar API tries to write the entry to the archive, and calculate the number of bytes that the data stream contains, it can't because the Length property throws on unseekable streams.

There is a workaround, and there is also an improvement we could add to the tar APIs:

• Workaround: Do not set the gzs as the DataStream directly. Instead, copy it first to a MemoryStream, then set that memory stream as the tar entry's DataStream.
• Tar API improvement: We should either throw early when the user attempts to set an unseekable stream as a tar entry's DataStream, or we should find a creative way of internally retrieving the length of that stream when the entry is about to be written to an archive. I'll open an issue in runtime for this.