Setup and transform your data before export
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
config
lib
spec
.gitignore
.rspec
.rubocop.yml
.travis.yml
CODE_OF_CONDUCT.md
Gemfile
Guardfile
LICENSE.txt
README.md
Rakefile
export.gemspec

README.md

Build Status

The idea is implement a simple way to describe what and how should the data be exported.

Example:

Export.transform User do
  replace :full_name, -> { FFaker::Name.name }
  replace :password, 'password'
  replace :email, -> (r) { "#{r.email.split('@').first}@example.com" }
  ignore :created_at, :updated_at
end

And then is possible to apply your rules with the tables:

dump = Export::TransformData.new(User)
result = dump.process(User.all)

And export the tranformed values to a file:

File.open('results.json', 'w+') {|f|f.puts result.to_json }

Currently you can also specify a dump schema, to fetch a specific scenario of data:

Export.dump 'last 3 monts user' do
  model User, -> { where(["created_at > ?",  3.months.ago]) }

  ignore AuditableItem

  on_fetch_data {|table, data| puts "Exported #{data.size} from #{table}" }

  on_fetch_error do |table, error, full_trace|
    puts "Ops! something goes wrong importing #{table}", error, full_trace
  end
end

Imagine that you also have a table orders that depends on users. It will automatically load only the orders related to the current user.

The same will happen with order_items that depends of what orders are being exported.

  • all include all records from n *tables
  • model receives a class and allow you specify a scope directly in the model class.

How to test

We have an example on examples/rails_test that you can use:

cd examples/rails_test
rake db:setup    # Populate with some seeds
rake export:init # Generate the default configuration

It will generate an initial suggestion with setting up the configuration:

Export.table 'users' do
  replace :full_name, -> { FFaker::Name.name }
  replace :password, 'password'
  replace :email, -> (r) { "#{r.email.split('@').first}@example.com" }
end

Export.table 'addresses'

To test all the process:

rake export:dump

Check the normalized results under results.json file.

TODO

  • Make it load the generated dump file
  • Explore SQL, yml and and other data formats
  • Port lib/tasks/export.rake from rails example to the lib
  • Allow use fake :full_name syntax
Export.table 'users' do
  fake :full_name
  fake :email
end