Add an option to retrieve presigned urls from to_remote_storage
call
#4196
Labels
kind/feature
New feature or request
to_remote_storage
call
#4196
Is your feature request related to a problem? Please describe.
to_remote_storage
method onRetrievalJob
class exports the dataset to a remote storage (s3/gcs) as parquet files and returns a list of file urls to the user. The location is usually preconfigured for an offline store as astaging_location
config. This behavior poses a problem for remote offline store deployments #4032 as the user will need to have a separate read access to the staging location.Describe the solution you'd like
Add an optional parameter to
to_remote_storage
method that will change the behavior of the method to return presigned urls to the caller rather than raw urls.The text was updated successfully, but these errors were encountered: