Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

сlarify differences and use cases for external storage mechanisms? #566

Open
jorgeorpinel opened this issue Aug 17, 2019 · 2 comments

Comments

@jorgeorpinel
Copy link
Collaborator

commented Aug 17, 2019

I've noticed that in general the different mechanisms DVC has to handle external storage are confusing. This includes DVC remotes, external dependencies/outputs/cache, and dvc import, dvc import-url commands, or even more general concepts like "remote location" and "external data source".

We have independent docs explaining each one but no single document (or consistent references throughout) so users can clearly identify which mechanism does what or solves which problem – In fact I wonder whether the same external data problem could be solved with one or another mechanism (which could add to the confusion).

Well, we do have this page https://dvc.org/doc/user-guide/external-outputs#description (and issue #143) that try to combine external outputs and external cache into a single doc but I think the effort should probably encompass all of these mechanisms.

I'm not sure exactly how to address this so just opening this possible issue and cc @iterative/engineering for a discussion. Thanks

@jorgeorpinel

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 17, 2019

UPDATE: Also related #103, maybe #143, #497, and #563 (specially given #563 (comment) and #563 (comment)).

@jorgeorpinel jorgeorpinel changed the title Clarify differences and use cases for external storage? Clarify differences and use cases for external storage mechanisms? Aug 17, 2019

@shcheklein shcheklein changed the title Clarify differences and use cases for external storage mechanisms? сlarify differences and use cases for external storage mechanisms? Aug 20, 2019

@mroutis

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2019

@jorgeorpinel , I don't have any solution to this but I agree that having a page explaining how to deal with external storage would be helpful.

A good next step would be to have several questions that we want this page to answer.
Maybe searching on Discord for keywords like external storage and remote or something that will help to figure out the FAQs from the community.

What I've identified (from skimming Discord) is that there are a lot of questions related to remote configuration and usage. Currently, there's no single page to refer to for tips and tricks when setting up a remote. Users need to go through the remote sub commands and click on the expandable boxes to know about it. There was an issue already (currently closed) but I think it might be related #499

Another frequent question is about NFS / CIFS we have no entry about this in our docs (as you already noted in your comment)

@iterative iterative deleted a comment from mroutis Aug 20, 2019

@jorgeorpinel jorgeorpinel referenced this issue Aug 20, 2019
0 of 2 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.