Confusing errors when not specifying bucket region

[Nikratio]

I've never used this bucket before, but:

> ./s3backer --blockSize=512K nikratio-test/s3backer ~/tmp/mnt  
s3backer: auto-detecting block size and total file size...
s3backer: error: auto-detection of filesystem size failed; please specify `--size'

./s3backer --blockSize=512K --size=50G nikratio-test/s3backer ~/tmp/mnt
s3backer: auto-detecting block size and total file size...
s3backer: auto-detection failed; using configured block size 512k and file size 50g
s3backer: http://s3.amazonaws.com/nikratio-test/s3backer/ appears to be already mounted (using mount token 0x00000001)
s3backer: reset mount token with `--reset-mounted-flag', or use `--force' to override

[Nikratio]

....or maybe the problem is something else?

./s3backer --blockSize=512K --size=50G --reset-mounted-flag nikratio-test/s3backer
s3backer: auto-detecting block size and total file size...
s3backer: auto-detection failed; using configured block size 512k and file size 50g
s3backer: resetting mount token for http://s3.amazonaws.com/nikratio-test/s3backer/
s3backer: error clearing s3 mount token: Input/output error

With debug logging:

./s3backer --blockSize=512K --size=50G --reset-mounted-flag --debug-http nikratio-test/s3backer

s3backer: auto-detecting block size and total file size...
*   Trying 52.217.201.128:80...
* Connected to s3.amazonaws.com (52.217.201.128) port 80 (#0)
> HEAD /nikratio-test/s3backer/00000000 HTTP/1.1
Host: s3.amazonaws.com
User-Agent: s3backer/1.6.2/1.6.2
Accept: */*
x-amz-date: 20220422T190609Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=0KTM97DC1DZTP27HXV02/20220422/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=b54c796205f273db510c42bf8058d2c4994aa7178f2628f23830de9b350acc1f

* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< x-amz-bucket-region: eu-west-2
< x-amz-request-id: 2DEYTHPHVA50MPM2
< x-amz-id-2: HlHJMzJDiRtl0d9un0kpxwCTiyV1ZAoDvF/NWNlv5I7tJYxvomv7ZDd/Td+CI4bBzvoU7bDjT14=
< Content-Type: application/xml
< Date: Fri, 22 Apr 2022 19:06:09 GMT
< Server: AmazonS3
< 
* Connection #0 to host s3.amazonaws.com left intact
s3backer: auto-detection failed; using configured block size 512k and file size 50g
s3backer: resetting mount token for http://s3.amazonaws.com/nikratio-test/s3backer/
*   Trying 52.217.201.128:80...
* Connected to s3.amazonaws.com (52.217.201.128) port 80 (#0)
> DELETE /nikratio-test/s3backer/s3backer-mounted HTTP/1.1
Host: s3.amazonaws.com
User-Agent: s3backer/1.6.2/1.6.2
Accept: */*
x-amz-date: 20220422T190609Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=0KTM97DC1DZTP27HXV02/20220422/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=[stuff]

* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< x-amz-bucket-region: eu-west-2
< x-amz-request-id: 2WHMR9B5X6ZTB6YS
< x-amz-id-2: 2ua6uHPJlVMokFdGgktQLOu7Ul2nr7g6NthbkM/K20yxFJbvQz/bVulMsSpr3Z1sWRdrjKxmjwc=
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Fri, 22 Apr 2022 19:06:09 GMT
< Server: AmazonS3
< 
* Failure writing output to destination
* Failed reading the chunked-encoded stream
* Closing connection 0
*   Trying 52.217.201.128:80...
* Connected to s3.amazonaws.com (52.217.201.128) port 80 (#0)
> DELETE /nikratio-test/s3backer/s3backer-mounted HTTP/1.1
Host: s3.amazonaws.com
User-Agent: s3backer/1.6.2/1.6.2
Accept: */*
x-amz-date: 20220422T190609Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=0KTM97DC1DZTP27HXV02/20220422/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=cd95fa8b0e9e1cc4dd85bd9e4dccc5466901592a49302a513d1af4d8bfd320a3

Maybe S3Backer is not honoring the redirect?

[Nikratio]

I think my guess is correct. The redirect is a result of the request being sent to the wrong S3 endpoint, due to s3backer not knowing which region the bucket is in.

With --region I'm having slightly more success - a different error:

./s3backer --blockSize=512K --size=50G --region eu-west-2 nikratio-test/s3backer ~/tmp/mnt --debug-http
s3backer: auto-detecting block size and total file size...
*   Trying 3.5.6.167:80...
* Connected to s3.amazonaws.com (3.5.6.167) port 80 (#0)
> HEAD /eu-west-2/00000000 HTTP/1.1
Host: s3.amazonaws.com
User-Agent: s3backer/1.6.2/1.6.2
Accept: */*
x-amz-date: 20220422T191136Z
x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256 Credential=0KTM97DC1DZTP27HXV02/20220422/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=3b9d4c0e6b75eb608f3d4c268e99bebe636659e50f08efb45b60a1db421508bf

* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< x-amz-request-id: 5Q97DP6W122ADRPQ
< x-amz-id-2: wTNfMUfZ/z2pb7CtAZQ6RbkxChmmvLTY4ellXW09qOJv8ld1sXYSo/qykUJEPx99LQDv2hZ/8lhIZe3rUMVmLA==
< Content-Type: application/xml
< Date: Fri, 22 Apr 2022 19:11:36 GMT
< Server: AmazonS3
< 
* Connection #0 to host s3.amazonaws.com left intact
s3backer: can't read data store meta-data: Operation not permitted

Credentials are taken from ~/.s3backer_passwd and have been freshly copied and pasted there from another s3 application (in which I'm able to access the bucket with these credentials).

[archiecobbs]

What do you suggest?

[Nikratio]

Well, ideally s3backer would honor the redirect response and transparently adjust the endpoint. If that's too complicated, exiting with a "ERROR: No/wrong region specified" error would be helpful :-).

[archiecobbs]

ideally s3backer would honor the redirect response and transparently adjust the endpoint.

Unfortunately, it doesn't look like Amazon tells us where to redirect to... i.e., there's no Location: header in the 301 response.

However, we can certainly output a warning message with a hint in this situation...

[archiecobbs]

Added warning in f76c7e8.

There is still the unresolved question of why you got 403 Forbidden, which would be due to some different problem.

If that's still occurring, please open a new ticket.

[Nikratio]

Hmm. That's surprising. Are you sure it's not filtered away / not printed for some reason? The API docs (https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html, search for "PermanentRedirect") are pretty clear about that.

In any case, you should be able to get the correct endpoint from the XML content of the message body.

[archiecobbs]

Are you sure it's not filtered away / not printed for some reason?

It's not appearing in the --debug-http output that you copied, so I'm assuming it's not there.

In any case, you should be able to get the correct endpoint from the XML content of the message body.

Probably not worth the trouble (IMHO), since with the warning/hint this should now be easy to detect and fix.