PutObject fails in version 2-preview (2.0.0.1) in console app

[evereq]

I have issues with PutObject (and PutObjectAsync) after upgrade to v2 preview (latest version 2.0.0.1 with fix for Async deadlocks).

Biggest issue is that even if I try to use old methods without async, it still fails (seems because such methods are just wrappers around async methods).

I got AmazonClientException with following message "Expected hash not equal to calculated hash" (happens during 'await s3Client.PutObjectAsync' call and can take a lot of time to wait till that exception throw even for small file).

Please note that I get such exception ONLY in console application. For some reason (and that's really strange), my huge ASP.NET MVC4 application works OK and success to execute PutObject requests.

I create repository on Github so one can reproduce issues I have (well, it's very basic console apps):
https://github.com/evereq/AWSAsyncTest

It contains 2 solutions, each one use different version of AWS SDK: one without Old prefix use new version 2.0.0.1 and one with Old prefix uses stable version 1.5.28.1.

You also need to add your own configuration inside AWSConfig.cs file: AWS Key, Secret, S3 Folder, etc.

After you configure try to run both apps:

• When you run Old console application, it attempts to put local file to S3 folder and succeed. So that means that old SDK 1.5.28.1 works well.
• When you run latest console application (without Old), it attempts first to put local file using normal (not async) approach and FAILS. Next it attempts to put same file using async approach and FAILS too. So that mean it's impossible to use latest AWS SDK (2.0.0.1) to put files inside S3 folder.

Please note that basically there are no differences between 2 solutions, except different version of AWS SDK and additional test for async workflow using it (but again, even non async workflow fails).

If it's possible to get solution or idea as soon as possible it will really help me!
I use console application to deploy changes in my project to live (i.e. continues deploy to www.evereq.com) and after SDK upgrade I just can't do it anymore :(
... and I really don't want to revert to v1.5 because already migrate tons of code to v2 and see that usage of async improve performance (when it works :D)

Any help really appreciated!

[PavelSafronov]

Thank you very much for providing such an easy-to-use repro case. :)

When filling the MemoryStream, you are not resetting the position to the start of the stream. If you call ms.Position = 0; before constructing the request, the position will reset and the request will succeed. V1 SDK did this, while the V2 SDK does not.

But setting the position is not the only possible solution:

• Since you already have a FileStream with your data, you can simply set the PutObjectRequest.InputStream to that stream.
• You can also construct the MemoryStream by passing in a byte array of your data (this would allow you to keep the file open for the shortest possible time). This can be done either by reading the data as you are doing it, or with File.ReadAllBytes.

On an unrelated note, when you are creating an instance of AmazonS3Config you are setting both ServiceURL and RegionEndpoint. You only need to set one of these. (If ServiceURL is set, the SDK ignores RegionEndpoint.) For ease of use, however, I would recommend to just using the RegionEndpoint.

[evereq]

Wow, thanks for an amazing, quick and right response!
Yep, reset of stream position helps and now console app works too :)
Secret was that in ASP.NET MVC app I do that, however in console app I read data other way (more simple) and fail with it.

I however want to make following notes: think it's good idea to list that (i.e. fact that new version of SDK does not reset stream position anymore) as breaking change because some other developers may step at the same issue (agree however that correct way to remove that line from SDK source code because it's client responsibility to ensure correct position in the stream... i.e. was totally my fault)

Thanks also for Tips about handing of streams for short period of time (they are correct and sample application can be improved with them, however in my production code it's more complicated and optimized already... for example I use async reading from source stream opened from web response instead of file in filesystem etc)

Re: ServiceUrl and RegionEndpoint I just can't found any info and decide to use both :D Now it's clear, thanks!

Thanks again, Amazing work!

🍻

[evereq]

Hi @PavelSafronov!

Would be amazing if you can take a look into new issue #27 for which same projects with SDK v2 have issues with Content-Encoding headers :)

Thanks!

P.S. hopefully last issue I need to be fixed in v2 to put my site latest version live :)