Skip to content

Update datasets download URL - #12840

Merged
rapids-bot[bot] merged 1 commit into
NVIDIA:branch-23.04from
jjacobelli:datasets
Feb 24, 2023
Merged

Update datasets download URL#12840
rapids-bot[bot] merged 1 commit into
NVIDIA:branch-23.04from
jjacobelli:datasets

Conversation

@jjacobelli

Copy link
Copy Markdown
Contributor

Description

Update datasets download URL to reduce latency and costs

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@jjacobelli jjacobelli added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Feb 24, 2023
@jjacobelli
jjacobelli requested a review from a team as a code owner February 24, 2023 09:51
@github-actions github-actions Bot added the Python Affects Python cuDF API. label Feb 24, 2023
Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>
@ajschmidt8

Copy link
Copy Markdown
Member

To provide additional context...

data.rapids.ai serves the contents of the rapidsai-data S3 bucket via an AWS CloudFront distribution.

The benefits of using a CloudFront distribution are:

  • dataset files are served from an endpoint closest to individual end-users. Therefore latency is decreased, which makes datasets download faster
  • data transfer costs out of AWS are cheaper

Therefore, it's in everyone's best interest to start using the new data.rapids.ai URLs for downloading datasets.

At some point in the future, the S3 URLs will be disabled and datasets will only be retrievable from data.rapids.ai.

@ajschmidt8

Copy link
Copy Markdown
Member

/merge

@rapids-bot
rapids-bot Bot merged commit 54ee14e into NVIDIA:branch-23.04 Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants