[R] Using s3_bucket with non-AWS S3-compatible storage is confusing

[amoeba]

Describe the bug, including details regarding any error messages, version, and platform.

If I want to use arrow with non-AWS, S3-compatible storage, I might find the s3_bucket convenience function from our documentation [1, 2]. The documentation doesn't always mention non-AWS storage but I might see in the help page for s3_bucket that I can pass options to S3FileSystem$Create using dots and those options include endpoint_override. I might then try the following:

s3_bucket(
    "my-bucket",
    endpoint_override = "example.org", 
    anonymous = TRUE
)

However, this results in the following error:

# Error: IOError: Bucket 'my-bucket' not found
# /Users/bryce/src/apache/arrow/cpp/src/arrow/filesystem/s3fs.cc:829  ResolveRegionUncached(bucket)
# /Users/bryce/src/apache/arrow/cpp/src/arrow/filesystem/s3fs.cc:357  ResolveS3BucketRegion(bucket)
# /Users/bryce/src/apache/arrow/cpp/src/arrow/filesystem/filesystem.cc:719  S3Options::FromUri(uri, out_path)

You can see that we're trying to resolve the bucket's AWS region which fails as expected. Interestingly, the user that reported this issue to me found they could prevent this error by creating a bucket on AWS with the same name at which point the region resolution would succeed (pointlessly), the S3FileSystem createsd would respect their endpoint_override, and future calls like $ls() would target the correct endpoint. This could actually cause some significant frustration for non-AWS users.

If I go back to the docs and read further down on https://arrow.apache.org/docs/dev/r/articles/fs.html#file-systems-that-emulate-s3, I find that the recommended way to do what I want is directly through S3FileSystem$create:

S3FileSystem$create(
    endpoint_override = "example.org",
    anonymous = TRUE
)

I was made aware through other channels while researching this issue that this syntax would also have worked (though it's not documented sufficiently):

s3_bucket("s3://anonymous@my-bucket?endpoint_override=example.org")

With the increasing number of non-AWS providers of S3-compatible storage (e.g., Ceph, DigitalOcean], I only think we'll have more users of non-AWS, S3-compatible storage and those users will run into confusion over (1) how things work and (2) how things are documented.

I'd like to:

1. Do a pass over the documentation (online, and R help pages)
   a. To make the messaging clear that we support AWS, GCS, and S3-compatible storage providers
   b. Add more examples to s3_bucket to make it clearer how a user can go about doing that
2. Change the behavior of s3_bucket to special-case passing endpoint_override to skip region resolution and just jump directly to creating an S3FileSystem.

Component(s)

R

[PMassicotte]

Good call @amoeba !

[paleolimbot]

I agree the existing documentation of that is confusing (I found the enpoint override thing by searching my own gists, which is hardly sufficient). I'd be grateful for any improvements you can come up with for the S3 interface...I don't have any experience using S3 (other than keeping our tests passing) so I'm a poor candidate to come up with improvements (but would be more than happy to review any PRs!).

[amoeba]

FYI all: @cboettig filed a PR and makes the case for dramatically simplifying the function body in his top level comment. See #33918 (comment).