Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to stream Zip File in controller's Response.Body #731

Closed
roddone opened this issue Feb 22, 2022 · 2 comments · Fixed by #754
Closed

Unable to stream Zip File in controller's Response.Body #731

roddone opened this issue Feb 22, 2022 · 2 comments · Fixed by #754
Labels
bug zip Related to ZIP file format
Projects

Comments

@roddone
Copy link

roddone commented Feb 22, 2022

Steps to reproduce

  1. Create a project targeting .net 3.1 or higher
  2. create a zip using new ZipOutputStream(context.Response.Body) and PutNextEntryAsync()

Expected behavior

the zip file can be sent streamingly to the client by writing directily to context.Response.Body, without creating an intermediary MemoryStream.
The reason for not using a MemoryStream and writing directly to context.Response.Bodyis to prevent high memory consumption on the server when creating big zip files. I haven't found any library that can handle this correctly, except SharpZipLib which seems to be really closed to achieve this 😥

Actual behavior

when calling stream.PutNextEntryAsync(), a "System.NotSupportedException: Specified method is not supported" is thrown., complete stacktrace :

System.NotSupportedException: Specified method is not supported.
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpResponseStream.get_Position()
   at ICSharpCode.SharpZipLib.Zip.ZipOutputStream.<>c__DisplayClass28_0.<PutNextEntryAsync>b__0(Stream s) in C:\Users\roddone\Source\Repos\SharpZipLib\src\ICSharpCode.SharpZipLib\Zip\ZipOutputStream.cs:line 526
   at ICSharpCode.SharpZipLib.Core.StreamUtils.WriteProcToStreamAsync(Stream targetStream, MemoryStream bufferStream, Action`1 writeProc, CancellationToken ct) in C:\Users\roddone\Source\Repos\SharpZipLib\src\ICSharpCode.SharpZipLib\Core\StreamUtils.cs:line 281
   at ICSharpCode.SharpZipLib.Core.StreamUtils.WriteProcToStreamAsync(Stream targetStream, Action`1 writeProc, CancellationToken ct) in C:\Users\roddone\Source\Repos\SharpZipLib\src\ICSharpCode.SharpZipLib\Core\StreamUtils.cs:line 291
   at ICSharpCode.SharpZipLib.Zip.ZipOutputStream.PutNextEntryAsync(ZipEntry entry, CancellationToken ct) in C:\Users\roddone\Source\Repos\SharpZipLib\src\ICSharpCode.SharpZipLib\Zip\ZipOutputStream.cs:line 524
   at Program.<>c.<<<Main>$>b__0_0>d.MoveNext() in C:\Users\roddone\Source\Repos\SharpZipLib\TestStreaming\Program.cs:line 12
--- End of stack trace from previous location ---
   at Program.<>c.<<<Main>$>b__0_0>d.MoveNext() in C:\Users\roddone\Source\Repos\SharpZipLib\TestStreaming\Program.cs:line 19
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Version of SharpZipLib

Obtained from (only keep the relevant lines)

  • Compiled from source, latest commit from master

full sample to reproduce (must create a .net 6 project) :

using ICSharpCode.SharpZipLib.Zip;
using System.Text;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapGet("/", async (HttpContext context) =>
{
	await using (var stream = new ZipOutputStream(context.Response.Body))
	{
		//add file without compression
		await stream.PutNextEntryAsync(new ZipEntry("test.txt") { CompressionMethod = CompressionMethod.Stored });

		var bytes = Encoding.UTF8.GetBytes("ah ah !");
		await stream.WriteAsync(bytes);

		await stream.FinishAsync(CancellationToken.None);

	}
});

app.Run();
@piksel
Copy link
Member

piksel commented Feb 23, 2022

Yeah, this was overlooked when implementing the async solution...
The position of the stream should only be read when patching is actually needed.
As a workaround for now, you could use a wrapper around the stream that intercepts this:

class StreamWrapper: Stream 
{
	{
        Stream _innerStream;
		public StreamWrapper(Stream innerStream)
        {
            _innerStream = innerStream;
        }

		public override bool CanRead => _innerStream.CanRead;
		public override bool CanSeek => _innerStream.CanSeek;
		public override bool CanWrite => _innerStream.CanWrite;
		public override void Flush() => _innerStream.Flush();
		public override Task FlushAsync(CancellationToken ct) => _innerStream.FlushAsync(ct);
		public override long Length => _innerStream.Length;
		public override long Position
		{
			get => 0;
			set => _innerStream.Position = value;
		}

		public override int Read(byte[] buffer, int offset, int count) => _innerStream.Read(buffer, offset, count);
		public override Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken ct) => _innerStream.ReadAsync(buffer, offset, count, ct);
		public override long Seek(long offset, SeekOrigin origin) => _innerStream.Seek(offset, origin);
		public override void SetLength(long value) => _innerStream.SetLength(value);
		public override void Write(byte[] buffer, int offset, int count) => _innerStream.Write(buffer, offset, count);
		public override Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken ct) => _innerStream.WriteAsync(buffer, offset, count, ct);
}

and wrapping Body:

await using (var stream = new ZipOutputStream(new StreamWrapper(context.Response.Body)))

@piksel piksel added bug zip Related to ZIP file format labels Feb 23, 2022
@piksel piksel added this to Needs triage in Bugs via automation Feb 23, 2022
@piksel piksel moved this from Needs triage to High priority in Bugs Feb 23, 2022
@roddone
Copy link
Author

roddone commented Feb 23, 2022

Thanks for the workaround !

piksel added a commit that referenced this issue May 24, 2022
Bugs automation moved this from High priority to Closed May 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug zip Related to ZIP file format
Projects
No open projects
Bugs
  
Closed
Development

Successfully merging a pull request may close this issue.

2 participants