default region is unconfortable when using elsewhere than AWS

[fcomte]

Please specify whether your issue is about:

• a possible bug
• a question about package functionality
• a suggested code or documentation change, improvement to the code, or feature request

The requested url is prefixed by the region even if the AWS_S3_ENDPOINT is set.
Is it possible to set it to "" when an endpoint is set ?

[pierre-lamarche]

To back @fcomte suggestion, it seems like the workaround given in the README (specify region="" in every request) does work till R 3.6, but fails on R 4.0. I have been browsing the release notes, but the reason for this is still not obvious to me. Yet I would say that's a workaround not only kind of painful, but also meant to be unapplicable on some configurations.

[s-u]

@fcomte well, the main problem is that AWS_S3_ENDPOINT seems to be non-standard (not used by other AWS clients) and is pretty much undocumented. It appears to be only used as the base URL and the comment suggests that it was intended to be just used to the base, so setting it doesn't mean you don't use regions. I am working on the re-design of s3HTTP so I think we may want clean up the API as part of it, but that is only possible by breaking the current behavior.

@pierre-lamarche Your statement is false, region="" works just fine in R 4.0.0:

> bucketlist(base_url="127.0.0.1:9000", region="", use_https=FALSE,
     key="minioadmin", secret="minioadmin")
  Bucket             CreationDate
1   test 2020-05-27T02:15:24.942Z
> R.version.string
[1] "R version 4.0.0 (2020-04-24)"

so the difference must be something else. Please open a new issue with exact details of your configuration and make sure you're using latest version of the package.

[fcomte]

This behaviour is recent with aws.s3.

This library is the only one i know that add region in case of a specific endpoint is set.

I can quote for library

• paws (R)
• s3fs (python)
• hadoop
  For tools
• rocket-chat
• gitlab
• nexus
• codimd

All know the region and the endpoint and they work.

Back to previous behaviour of aws.s3 could be usefull

[fcomte]

But no tools use this AWS_S3_ENDPOINT.
For example i need to specify the base url with s3fs.

I would prefer to specify the endpoint url than setting region='' which is wrong and hack

[s-u]

@fcomte Agree, that's exactly what I said above. I wouldn't write it that way and I don't know why it was introduced the way it is, but that's what the current API is. I want to re-write the whole part as the request handling is very inconsistent right now, but that can only be done by breaking the current API.

[pierre-lamarche]

Ok, you got a point @s-u I indeed reproduced the error with the same configuration and R 3.6. I actually tried using a Docker image for RStudio server 3.6 and it was working ; turning on a full Ubuntu configuration, with RStudio desktop, it fails. Sorry for jumping too quickly on conclusions. Will try to investigate so as to open a useful issue.