A Ruby gem for comparing file-like objects from various data sources
Ruby
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
lib
spec
.gitignore
.travis.yml
Gemfile
README.md
Rakefile
changelog.md
filey-diff.gemspec

README.md

Filey diff

[Build Status] (http://travis-ci.org/laurilehmijoki/filey-diff)

A Ruby library for comparing file-like objects from various data sources.

Central concepts

Filey

A file-like object. Can be, for example, a file system file or an AWS S3 object.

Data source

Provides Fileys.

The current built-in data sources support Amazon Web Services S3 and file system.

Operations

List outdated files

Given two data sources A and B, list the changed files that A has but B doesn't.

require 'aws-sdk'
require 'filey-diff'

s3 = AWS::S3.new(:access_key_id => 'some-id',
                 :secret_access_key => 'some-secret')
s3_bucket = s3.buckets['your-s3-bucket-name']

s3_data_source = Filey::DataSources::AwsSdkS3.new(s3_bucket)
fs_data_source = Filey::DataSources::FileSystem.new('/tmp/site-root')
Filey::Comparison.list_changed(fs_data_source, s3_data_source).each { |filey|
  puts "File #{filey.full_path} has different contents on local file system than on S3"
}

List missing files

Given two data sources A and B, list the files that A has but B doesn't.

Filey::Comparison.list_missing(fs_data_source, s3_data_source).each { |filey|
  puts "File #{filey.full_path} is missing from S3"
}

List changed files

Given two data sources A and B, list the files on A that have a different MD5 hash than the corresponding file on B.

Filey::Comparison.list_outdated(fs_data_source, s3_data_source).each { |filey|
  puts "File #{filey.full_path} is newer on local file system than on S3"
}

AWS SDK data source

Specifing custom concurrency level

The concurrency level determines the amount of parallel operations that the AwsSdkS3 data source performs against the S3 API.

config = { :concurrency_level => 1000 }
s3_data_source = Filey::DataSources::AwsSdkS3.new(s3_bucket, config)

Example use cases

Arnie has a blog on AWS S3. He has just finished a new post and wants to upload only the new post into S3. With the help of Filey diff Arnie can write a Ruby program that uploads only the new post and nothing else.

License

Copyright (C) 2012 Lauri Lehmijoki

Distributed under the Apache-2.0 license http://www.apache.org/licenses/LICENSE-2.0.html