Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sub Directory support question #5

Closed
phpfui opened this issue Dec 20, 2021 · 6 comments
Closed

Sub Directory support question #5

phpfui opened this issue Dec 20, 2021 · 6 comments

Comments

@phpfui
Copy link
Contributor

phpfui commented Dec 20, 2021

First of all, this is a great wrapper around blob storage engines! I was going to have to write this from scratch, so I am glad I found this.

I am writing a Storage copy command line utility (happy to add it to this repo once finished if you like). Basically it asks for a source server and a destination server, then iterates through the blobs on the source server and adds them to the destination server. This is great for downloading from a cloud server to a local server for testing.

I pointed the source server to a random local directory with sub directories, and to my surprise, I found it iterated through all the files in all directories. I then tried to add these "blobs" into an empty directory without the corresponding sub directories, and it crashed on the first directory. Looks like the target sub directories are not being created on a write.

So the question is, should it iterate through sub directories on read, or should it create sub directories on write? Or neither?

Totally understand if sub directories are not supported. But in that case, it should probably not enumerate sub directories.

And thanks again for this very nicely designed library.

@jchristn
Copy link
Owner

Hi @phpfui first off thank you for the kind words! They are very much appreciated.

The intent of the library is to make interaction consistent across repositories. Thus for file, even though it has subdirectory support, object (S3, etc) does not (though it supports prefixes that make it look as though directories are supported).

So for file, it should recurse on read and, on write, create subdirectories as needed. I'll take a look ASAP and see what needs to be fixed to ensure the library is consistent with this statement!

Thanks, Joel

@jchristn
Copy link
Owner

Hi @phpfui could you give 2.1.4.5 a try and let me know if this resolves the issue? https://www.nuget.org/packages/BlobHelper/2.1.4.5

@phpfui
Copy link
Contributor Author

phpfui commented Dec 21, 2021

I can confirm it works! Thank you very much for the prompt resolution.

Let me know if you would like to add a Blob server copy program to the repo. Right now it only does DiskFile and Azure, but I think I could make it do all supported type fairly easily. Would just copy blobs between servers. Great way to backup a cloud server locally as well.

@jchristn
Copy link
Owner

Thanks @phpfui I'd love to see it! And thanks for confirming that everything works as expected now :)

@phpfui
Copy link
Contributor Author

phpfui commented Dec 21, 2021

I'll see if I can put in a PR this week. C# is not my forte, as I am a PHP guy (DUH!), but C# has a lot of C++ stuff (former C++ guy), so it is fun to work in a language that supports method and operator overloading and strict types.

@jchristn
Copy link
Owner

That's awesome @phpfui ! Please don't feel like you have to do a PR around it, if you want to zip/attach to this issue I'm also happy to do integration work on my end if it fits cleanly into the library! Hope you have a nice week and holiday break.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants