Uploading zero-byte stream, with length -1 fails

[LordMike]

We have a piece of code that doesn't know how long the input stream is (compressed data), so we pass -1 to Minios PutObjectAsync. This fails with the below exception

Minio API responded with message=The XML you provided was not well-formed or did not validate against our published schema.

   at Minio.MinioClient.ParseError(IRestResponse response) in /q/.q/sources/minio-dotnet/Minio/MinioClient.cs:line 363
   at Minio.MinioClient.HandleIfErrorResponse(IRestResponse response, IEnumerable`1 handlers, DateTime startTime) in /q/.q/sources/minio-dotnet/Minio/MinioClient.cs:line 505
   at Minio.MinioClient.ExecuteTaskAsync(IEnumerable`1 errorHandlers, IRestRequest request, CancellationToken cancellationToken) in /q/.q/sources/minio-dotnet/Minio/MinioClient.cs:line 351
   at Minio.MinioClient.CompleteMultipartUploadAsync(String bucketName, String objectName, String uploadId, Dictionary`2 etags, CancellationToken cancellationToken) in /q/.q/sources/minio-dotnet/Minio/ApiEndpoints/ObjectOperations.cs:line 348
   at Minio.MinioClient.PutObjectAsync(String bucketName, String objectName, Stream data, Int64 size, String contentType, Dictionary`2 metaData, CancellationToken cancellationToken) in /q/.q/sources/minio-dotnet/Minio/ApiEndpoints/ObjectOperations.cs:line 310
   at REDACTED.Networking.Minio.MinioTransport.SendAsync(String resource, Stream stream, REDACTED)

I've been sending in zero-length MemoryStreams and can confirm that if I send in a one-byte MemoryStream, the issue disappears.

[kannappanr]

@LordMike Thanks for filing this issue. We will take a look and get back to you.

[poornas]

@LordMike,when you specify length as -1, we internally use s3 multipart API since the stream size is unknown. However s3 multipart API does not allow 0 byte uploads, hence the MalformedXML error that you are seeing.

At the SDK level, we cannot fix this - your application needs to figure out the stream size or handle the error if -1 is passed as size for PutObjectAsync API on a zero byte stream

[LordMike]

Thanks for being quick. Can we do a multipart upload ourselves using the Minio SDK? -- else the only option we have seems to be handling this error...

Or reading the first byte, if it didn't exist, then it was a 0 byte stream, and we can do a put using that size, and if it did exist, we create a new stream that wraps the first byte and then reads from the original stream again..

[poornas]

@LordMike, the SDK does not provide multipart upload operation as an API call - however you could use the second approach to read the first byte and determine whether to call PutObjectAsync with 0 as size for a zero byte stream, or create a new stream that wraps first byte with original stream and pass it to PutObjectAsync with -1 as size.

[LordMike]

Would it be possible for the Minio SDK to get this feature? .. I imagine others would need this as well.

[harshavardhana]

@poornas if we see an EOF early with -1 then we can upload just the zero bytes with single part which is allowed in multipart uploads.

[poornas]

@harshavardhana, yes we could do that.