Skip to content

[R] Cannot override environmental variables setting AWS S3 region #32493

@asfimport

Description

@asfimport

If a user has set the AWS_DEFAULT_REGION, there is no way we can override that setting (especially to a null value) in a one-off call.  Consider the following example.  This fails:

library(arrow)
Sys.setenv(AWS_DEFAULT_REGION="data")

noaa <- s3_bucket("neon4cast-drivers/",
                  endpoint_override = "data.ecoforecast.org",
                  anonymous = TRUE) 

If the env var is not set or unset, this succeeds.  However, attempting to override the region does not help:

noaa <- s3_bucket("neon4cast-drivers/",
                  endpoint_override = "data.ecoforecast.org",
                  region = "us-east-1",
                  anonymous = TRUE) 

(nor does it help to set the region to "", or NULL. – note this is a MINIO host and so region is not needed anyway). 

Relatedly, one might expect that AWS_S3_ENDPOINT could be used instead of setting endpoint_override, but this does not seem to work either (lemme know if you want that in a separate issue thread).  At least on the plus side, the above code does not fail if AWS_S3_ENDPOINT is set to some other value than given in endpoint_override, so that is nice). 

Reporter: Carl Boettiger / @cboettig

Note: This issue was originally created as ARROW-17201. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions