S3 Authorization Issue

[jonas-t-s]

Hey there,

I'm loving the app, but it stopped working. I get the error: S3 Authorization issue...

I'm using b2 as a back-end and the key suddenly stopped working. Thus I created a new key according to the guide and it still does not work. Funnily I can see the pictures online but not upload new items.

Do you have any idea what could be the problem here?

Thank you for your answer in advance.

[tomekit]

Hi @jonas-t-s,

Thanks for reporting this issue and it's great to hear you like the app.

Can I ask you to provide the logs from the application?
Once you go to the About page and click 7 times on the Version, you will enter the logs.

The recommended way is using the: "Send" option, however you can also Save them locally and send it directly to me on: traganowicz@syncaware.com

We would be looking specifically for the line which starts with: "Exception, file:" which shall provide what type of S3 exception it is.

[jonas-t-s]

Hi @tomekit

Thank you very much for your response. I provide you the link here: https://cdn.filestackcontent.com/zhAVa8TGSYGrS09FcsjJ

[tomekit]

That's extremely helpful, thanks !

This seem to be caused by the object locking setting within the bucket. Did you recently set the explicit default object lock for the bucket?

There is an open PR related to S3 protocol: aws/aws-sdk-php#1694 but it won't be resolved, so we may have to address it on our side. The issue is that we would have to generate MD5 for each single file, this however has performance/battery implications which we need to strongly consider.

In order to resolve this issue now, you can disable default Object Lock setting, in order to do so, please click on Object Lock:

then set it to: 0

Do you find the Object Lock setting useful? We will certainly be looking on supporting this and workaround the S3 default design.

[jonas-t-s]

This seem to be caused by the object locking setting within the bucket. Did you recently set the explicit default object lock for the bucket?

Yes I put it on when I created the bucket. I just increased it from 40 to 60 days lately.

Do you find the Object Lock setting useful? We will certainly be looking on supporting this and workaround the S3 default design.

Yes. Since I want that my photos are if they are online in my private bucket for a time. Whilst I don't know currently why I've enabled it, I see some scenarios:

• Ransomeware is unable to delete photos
• Case of emergency
• Legal proof that something happened.

In order to resolve this issue now, you can disable default Object Lock setting, in order to do so, please click on Object Lock:

I see, I will have to do this, but I'm happy to activate it again.

The issue is that we would have to generate MD5 for each single file, this however has performance/battery implications which we need to strongly consider.

I see the problem. What about giving the option to the user: Upload and calculate only when device is charged.
As long as you store the MD5 locally, I think it is not that heavy on performance/battery, since the number of photos is not that big I think.

[jonas-t-s]

Okey, removing the object lock indeed solved the problem

Thank you. Please notify me, as soon as you added object lock support to that. (Is just ignoring such an exception an option?)

PS: Edited my previous comment

[tomekit]

Unfortunately ignoring exception would just silent the issue, but file wouldn't be uploaded as S3 rejects such request.

I've moved this out to separate feature/issue: #17
I think the best way so far is to have the: "Calculate MD5" setting which would be automatically enabled once app come across this specific exception. In that case the additional MD5 calculations would affect only users which explicitly want this feature.
Then in the future we could think about optimizations like you've suggested, e.g. calculating MD5 locally when charger connected.

Thanks for your input. Are you already a Pro user? I would be happy to hand you the Promo Code to say thank you.

[jonas-t-s]

Unfortunately ignoring exception would just silent the issue, but file wouldn't be uploaded as S3 rejects such request.

Oh yeah, I see the problem.

I've moved this out to separate feature/issue: #17
I think the best way so far is to have the: "Calculate MD5" setting which would be automatically enabled once app come across this specific exception. In that case the additional MD5 calculations would affect only users which explicitly want this feature.
Then in the future we could think about optimizations like you've suggested, e.g. calculating MD5 locally when charger connected.

Thank you very much.

Thanks for your input. Are you already a Pro user? I would be happy to hand you the Promo Code to say thank you.

No problem, I love to help. No I'm not yet a Pro User. This would be very generous

[tomekit]

Hi @jonas-t-s,

Object lock default retention policies are now supported since the today's 1.5.3 release.

Once file upload functionality fails on upload due to retention policy being set, it switches on the MD5 header generation setting on the client-side. It's been tested successfully on Backblaze and AWS (they use different exceptions). The error messaging was also improved, so application reports the error directly instead of using the plain incorrect: "Authorization issue" message.

It came to our attention that AWS S3 supports stronger hashing algorithms than MD5. It it possible that other S3 back-ends do support them as well. We'll research this and implement SHA256 (most likely) for some or all of the back-ends depending on the availability. Calculating hashes client-side is required anyway for the two-way sync feature which we're actively working on and MD5 hashes don't meet the security standards.

If you happen to have time to confirm that, I would appreciate if you can close this issue.

Thanks !

[jonas-t-s]

Hey @tomekit

Thank you very much. I tried my best to test it and I will use this feature from now on in my day to day usage. And it seems to work.
What I've done to test it and what I've seen:

1. Made an object lock for default 60 Days.
2. Made two pictures
3. I got a message that MD5 verification is now enabled
4. Deleted one of the pictures from my phone. Got an error. Looking at the Log it got a non-fatal error (about that it cannot access the media. Got a message that the upload failed. Investigated it: In a second attempt it succeeded. The log said it was a network timeout.

So everything seems to work fine.

Thank you very much for your fast fix!!!

It came to our attention that AWS S3 supports stronger hashing algorithms than MD5. It it possible that other S3 back-ends do support them as well. We'll research this and implement SHA256 (most likely) for some or all of the back-ends depending on the availability. Calculating hashes client-side is required anyway for the two-way sync feature which we're actively working on and MD5 hashes don't meet the security standards.

In this field I got a bit of knowledge. Whilst SHA256 is more secure then MD5, there are still applications where MD5 is sufficient. (As long as the input-dataset is not to be manipulated by an advisory) But personally I would always prefer sha256 over MD5 since then you do not need to think about which attack scenario your in. But if the resources are limited MD5 may become relevant again, depending on the hardware. But some modern CPUs got hardware-accelerated SHA256 computing, which may be more efficient then calculating it normally. And so on.

---

I thank you again. Keep going on with the good work.