-
Notifications
You must be signed in to change notification settings - Fork 676
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
s3sync support #86
Comments
Hi. Thanks for pointing this out. I already considered this a while ago but for some reason I believe I got ahold of an old version that seemed to be either abandoned or just didn't work, so I assumed it was no longer supported. I just briefly checked out s3sync.rb and successfully got it up and running (with Ruby 1.8.7 and REE, but not with Ruby 1.9.2 due to the fact that it can't find the md5 encryptor library. I haven't looked in to this yet myself but if you know why I get the following error and how to fix it, it'd save me some research time.
If not then I'll see why it occurs. Probably used to be in Ruby's core and was removed since 1.9 or something. However since I got it to work under 1.8 and it seems like a very useful addition to the Backup gem I will most certainly implement this very soon. It also has the ability to mirror using the Previously, before I implemented Syncers this was the reason why I didn't use the s3sync approach, because from what I read there was no way to get the real RSync ( diff ) effect on files on the S3 filesystem, so it'd always transfer the full bytes, and since storages in Backup always bundled everything up as a single .tar.gz.enc file, the whole syncing purpose would be pointless since it'd always transfer the whole file. However since the introduction of Syncers in Backup, where you can specify a whole directory (and sub-directories) to directly sync to a remote location, bypassing the whole copy/tar/compress/encrypt procedure then s3sync will regardless be an very awesome addition to the Backup gem. So let me know if you have any info on the above error which only occurs in Ruby 1.9, and whether you know if a file that only changed a few bytes will be completely re-uploaded. Thanks! |
Something tells me that S3Sync hasn't been updated to work with Ruby 1.9.2 :/ |
i dont think it has been updated in a while. I've been using 1.2.6 for the past 3 years. I think it's been abandoned. Let me look to see if I can get it working under 1.9.x in the next few weeks. |
Thanks. Yeah actually for Ruby 1.9.2 the path to the md5 library is
Hopefully there aren't much more issues that need to be made compatible, then we can make it Ruby 1.9 compatible and add S3Sync to Backup. |
I just found this on GitHub, a Ruby 1.9.2 compatible S3Sync: https://github.com/aproxacs/s3sync However now I run it with Ruby 1.8.7 and i get errors ha. But this is a good step in the right direction. Going to see what can be done. In the worst case we now have 2 separate libraries that work the same, but one is compaitble for Ruby1.8 and the other for Ruby1.9 |
Update: Ruby 1.9 compatible S3Sync Ruby 1.8 compatible S3Sync The user will have to ensure the correct one is installed for their Ruby version. |
Well I actually got it working with Backup. I'm just running some more tests on it, and writing more specs, and i'll push a gem soon that includes the S3 Syncer. :) hurray. Works both with Ruby 1.8 and Ruby 1.9, but you gotta use a different gem depending on which you want to use (as stated above in my comment) but that's absolutely no problem, just a different |
Boom: 69e81d0 This commit integrates S3Sync with the Backup gem. Going to update the Syncer Wiki page. |
And it has been released.
and check out the docs here: https://github.com/meskyanichi/backup/wiki/Syncers |
Awesome thanks for the fast response! |
You're welcome! I know I'll enjoy this one as well : ) |
Took me a while to figure this out. Could you add it to the wiki for S3? |
@ponny add what to the wiki? the S3syncer? It's here already: https://github.com/meskyanichi/backup/wiki/Syncers |
Erk. Right you are. I must be blind. |
It would be great if there was support for s3sync and s3cmd http://www.s3sync.net/wiki
The text was updated successfully, but these errors were encountered: