-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
explicit storage directories, or no? #5
Comments
Note this is behavior in #3 currently. |
I think |
And one thing to watch out for: GoogleDrive, S3, and osfstorage are not always mappable to filesystems without special handling. All are happy to have a file named |
thanks @felliott ...but let me argue for the other behavior here: it is simply not useful to require people to descend into subdirectories to get access to specific files :). just to make life more complicated there's all sorts of things that we could do here ... some not necessarily orthogonal options:
I think designating Is there any other remote integration we should be looking to for guidance here, or has this not come up before? I note that there is no way to do a download of an entire project's files as a single zip - it's storage specific. |
OK, well, for now:
|
👍 to the "well, for now". As far as having the files in separate directories based on provider, I'd argue that finding files nowadays comes in two styles:
I agree that it may not be useful to go directory diving, but most of those folks have moved on to fuzzy-finding, so directories are somewhat irrelevant anyway. For myself, I like distinguishing between what's on my GDrive, what's on my osfstorage, and what's on my Dropbox. I use each of those for different purposes. I especially don't want those mixed into whatever GitHub project I have linked up. If I did that, my flake8 would become self-aware and murder me, and no jury would convict it. (Of course, that could just be an argument for making github/bitbucket/gitlab a special case). |
I like (what a surprise) the behaviour of having subdirectories for each storage. It makes it explicit where a file came from and avoids heuristics to deal with naming collisions. Another argument for sub dirs would be that you are probably keeping different things in the different storage providers (code on github, data on S3, ...) so you want them to be different. One thing I would like is an option |
One other thought: osf-cli should be idempotent, i.e.:
should result in fetch having to do nothing; and likewise
should do nothing. |
Hmm, I want to do be able to do:
osf fetch <working directory> <project>
cd <working directory>
jupyter notebook &
and have it work as expected.
|
I'd expect it to show me the subdirectories ... especially as you might have notebooks in multiple storages. I really don't like having files dropped in the root directory because I don't know what a good strategy is for handling conflicts when two or more storages give you the same path. So I still prefer the |
We now have basic config file support. I think adding to the config file a way to specify mappings from "name of the storage on osf.io" to "local path" is the way to go. By default we continue to do what |
Currently,
python -m osfclient fetch foo 7g6vu
puts files under subdirectories named after the storage.It looks to me like the Right Approach is to use the
attr['materialized_path']
attribute for filenames, which is whatls
currently does --but I don't know what happens if a file from one osfstorage steps on a file from another osfstorage. Any tips @felliott?
The text was updated successfully, but these errors were encountered: