Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible problem with s3 buckets that contain dots in their name #229

Closed
jpmarindiaz opened this issue May 23, 2020 · 3 comments
Closed

Comments

@jpmarindiaz
Copy link

There might be some unexpected scaping of dots . in bucket names.

I am able to run this:

 board_register_s3(bucket = "xxx.yyy.zzz")
 pin(mtcars, board = "xxx.yyy.zzz")

And I get this error:

Error in pin_download(path, name, board$name, extract = identical(extract,  : 
  Client error: (404) Not Found. Failed to download remote file: http://xxx.yyy.zzz.s3.amazonaws.com/mtcars

Funny thing is that the file gets uploaded correctly. But then pinfails when it tries to download the uploaded pin. Running again with options(pins.verbose = TRUE). The console outputs:

Storing mtcars into board xxx.yyy.zzz with type table
Checking 'change_age' header (time, change age, max age): 1590276706.44661, 1590276706.44656, 0
Checking 'etag' (old, new): , 
Downloading http://xxx.yyy.zzz.s3.amazonaws.com/mtcars/data.txt to /var/folders/2m/5p4nlx0d2_z_f8hsgvbwmpqc0000gn/T//RtmpeIjUCC/filef095a0c8833/data.txt
No encoding supplied: defaulting to UTF-8.
<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>NoSuchBucket</Code>
  <Message>The specified bucket does not exist</Message>
  <BucketName>xxx</BucketName>
  <RequestId>3C88ABA1354172CE</RequestId>
  <HostId>U9BamOkMZbT8/swohTQnl+uycdfY1clGoy1QMK/WY26pNghp1+ccpQ70lCMbiQjisH31OenBVVc=</HostId>
</Error>

Note that <BucketName>xxx</BucketName> is only showing the first part xxx of the bucket name... the part before the first dot.

Because of this even when the file gets uploaded and pinned(*), it does not retrieve it back. I tried to look at the pins source but I still don't understand it well to propose a solution.

(*) A quick note, when the download fails, I noticed that the data.txt is replaced in full and not appended to the previous one, losing the info of all pins in the bucket. Is this another issue? I believe the data.txt pin catalog should be updated right after successful upload and independent of the success of pin retrieval.

I also believe this hast to do with the latests comments in #188

@javierluraschi
Copy link
Contributor

@jpmarindiaz can you verify this one was also fixed by #237? You'll need to install with remotes::install_github("rstudio/pins"), thanks!

@jpmarindiaz
Copy link
Author

Yes, it works now. Thanks

@github-actions
Copy link

github-actions bot commented Sep 1, 2022

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.

@github-actions github-actions bot locked and limited conversation to collaborators Sep 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants