Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no possibility to specify credentials/profile in dvc import #1827

Closed
marie-ml opened this issue Apr 4, 2019 · 4 comments

Comments

Projects
None yet
2 participants
@marie-ml
Copy link

commented Apr 4, 2019

Hi,

For many use cases, when importing from s3, the s3 bucket from which data is imported requires credentials or a profile. Right now, it does not seem possible to specify these (even though it is possible for a s3 remote with dvc remote modify myremote profile myprofile).

I know that is it possible to work around that by using dvc run instead, but providing a way to specify a profile or credential would be a welcome feature in dvc import. It could look like:

dvc import s3://mybucket/myfile destination --profile myprofile

Thanks!

@efiop

This comment has been minimized.

Copy link
Member

commented Apr 4, 2019

Hi @marie-ml !

You could go about it this way:

dvc remote add mys3 s3://bucket/path
dvc remote modify mys3 profile myprofile
dvc import remote://mys3/myfile destination

Thanks,
Ruslan

@marie-ml

This comment has been minimized.

Copy link
Author

commented Apr 4, 2019

Hi @efiop ,

Thanks for the quick answer! And is there a way to specify where the destination.dvc file should be saved to ?

Thanks!

@efiop

This comment has been minimized.

Copy link
Member

commented Apr 4, 2019

@marie-ml There is no such option for dvc import itself right now 🙁 By default, it will simply place destination.dvc right beside where destination is located. However, you could workaround that by using a more flexible dvc run command like so:

dvc run -d remote://mys3/myfile -o destination -f path/to/file.dvc
@efiop

This comment has been minimized.

Copy link
Member

commented Apr 22, 2019

dvc import now supports -f #1895 . Closing this issue, please feel free to reopen 🙂

@efiop efiop closed this Apr 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.