Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Upgrade yaml_db to latest version (11678745cb1)

  • Loading branch information...
commit cbb2106784c84a6fffc3b2396b22c6b0e550cc1e 1 parent 03e5e20
@FooBarWidget FooBarWidget authored
View
35 vendor/plugins/yaml_db/README
@@ -1,35 +0,0 @@
-= YamlDb
-
-YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
-
-This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
-
-Any database that has an ActiveRecord adapter should work.
-
-== Usage
-
-rake db:data:dump -> Dump contents of Rails database to db/data.yml
-rake db:data:load -> Load contents of db/data.yml into the database
-
-Further, there are tasks db:dump and db:load which do the entire database (the equivalent of running db:schema:dump followed by db:data:load).
-
-== Examples
-
-One common use would be to switch your data from one database backend to another. For example, let's say you wanted to switch from SQLite to MySQL. You might execute the following steps:
-
-1. rake db:dump
-
-2. Edit config/database.yml and change your adapter to mysql, set up database params
-
-3. mysqladmin create [database name]
-
-4. rake db:load
-
-== Credits
-
-Created by Orion Henry and Adam Wiggins. Major updates by Ricardo Chimal, Jr.
-
-Patches contributed by Michael Irwin, Tom Locke, and Tim Galeckas.
-
-Send questions, feedback, or patches to the Heroku mailing list: http://groups.google.com/group/heroku
-
View
48 vendor/plugins/yaml_db/README.markdown
@@ -0,0 +1,48 @@
+# YamlDb
+
+YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
+
+This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
+
+Any database that has an ActiveRecord adapter should work.
+
+## Installation
+
+This plugin can now be installed as a gem via
+
+ gem install yaml_db
+
+Note that when doing this, the rake tasks won't automatically be available to your app. You'll have to explicitly include them via the approaches followed: [here](http://ggr.com/how-to-include-a-gems-rake-tasks-in-your-rails-app.html), [here](https://rails.lighthouseapp.com/projects/8994/tickets/510-rake-tasks-not-included-for-gems), [and here](https://rails.lighthouseapp.com/projects/8994/tickets/59)
+
+## Usage
+
+ rake db:data:dump -> Dump contents of Rails database to db/data.yml
+ rake db:data:load -> Load contents of db/data.yml into the database
+
+Further, there are tasks db:dump and db:load which do the entire database (the equivalent of running db:schema:dump followed by db:data:load). Also, there are other tasks recently added that allow the export of the database contents to/from multiple files (each one named after the table being dumped or loaded).
+
+ rake db:data:dump_dir -> Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)
+ rake db:data:load_dir -> Load contents of db/data_dir into database
+
+In addition, we have plugins whereby you can export your database to/from various formats. We only deal with yaml and csv right now, but you can easily write tools for your own formats (such as Excel or XML). To use another format, just load setting the "class" parameter to the class you are using. This defaults to "YamlDb::Helper" which is a refactoring of the old yaml_db code. We'll shorten this to use class nicknames in a little bit.
+
+## Examples
+
+One common use would be to switch your data from one database backend to another. For example, let's say you wanted to switch from SQLite to MySQL. You might execute the following steps:
+
+1. rake db:dump
+
+2. Edit config/database.yml and change your adapter to mysql, set up database params
+
+3. mysqladmin create [database name]
+
+4. rake db:load
+
+## Credits
+
+Created by Orion Henry and Adam Wiggins. Major updates by Ricardo Chimal, Jr.
+
+Patches contributed by Michael Irwin, Tom Locke, and Tim Galeckas.
+
+Send questions, feedback, or patches to the Heroku mailing list: http://groups.google.com/group/heroku
+
View
19 vendor/plugins/yaml_db/Rakefile
@@ -1,6 +1,25 @@
require 'rake'
require 'spec/rake/spectask'
+begin
+ require 'jeweler'
+ Jeweler::Tasks.new do |gem|
+ gem.name = "yaml_db"
+ gem.summary = %Q{yaml_db allows export/import of database into/from yaml files}
+ gem.description = %Q{
+YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
+This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
+Any database that has an ActiveRecord adapter should work
+}
+ gem.email = "nate@ludicast.com"
+ gem.homepage = "http://github.com/ludicast/yaml_db"
+ gem.authors = ["Adam Wiggins","Orion Henry"]
+ end
+ Jeweler::GemcutterTasks.new
+rescue LoadError
+ puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
+end
+
desc "Run all specs"
Spec::Rake::SpecTask.new('spec') do |t|
t.spec_files = FileList['spec/*_spec.rb']
View
1  vendor/plugins/yaml_db/VERSION
@@ -0,0 +1 @@
+0.1.0
View
78 vendor/plugins/yaml_db/lib/csv_db.rb
@@ -0,0 +1,78 @@
+#require 'FasterCSV'
+module CsvDb
+ module Helper
+ def self.loader
+ Load
+ end
+
+ def self.dumper
+ Dump
+ end
+
+ def self.extension
+ "csv"
+ end
+ end
+
+ class Load < SerializationHelper::Load
+ def self.load_documents(io, truncate = true)
+ tables = {}
+ curr_table = nil
+ io.each do |line|
+ if /BEGIN_CSV_TABLE_DECLARATION(.+)END_CSV_TABLE_DECLARATION/ =~ line
+ curr_table = $1
+ tables[curr_table] = {}
+ else
+ if tables[curr_table]["columns"]
+ tables[curr_table]["records"] << FasterCSV.parse(line)[0]
+ else
+ tables[curr_table]["columns"] = FasterCSV.parse(line)[0]
+ tables[curr_table]["records"] = []
+ end
+ end
+ end
+
+ tables.each_pair do |table_name, contents|
+ load_table(table_name, contents, truncate)
+ end
+ end
+ end
+
+ class Dump < SerializationHelper::Dump
+
+ def self.before_table(io,table)
+ io.write "BEGIN_CSV_TABLE_DECLARATION#{table}END_CSV_TABLE_DECLARATION\n"
+ end
+
+ def self.dump(io)
+ tables.each do |table|
+ before_table(io, table)
+ dump_table(io, table)
+ after_table(io, table)
+ end
+ end
+
+ def self.after_table(io,table)
+ io.write ""
+ end
+
+ def self.dump_table_columns(io, table)
+ io.write(table_column_names(table).to_csv)
+ end
+
+ def self.dump_table_records(io, table)
+
+ column_names = table_column_names(table)
+
+ each_table_page(table) do |records|
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
+ records.each do |record|
+ io.write(record.to_csv)
+ end
+ end
+ end
+
+ end
+
+
+end
View
195 vendor/plugins/yaml_db/lib/serialization_helper.rb
@@ -0,0 +1,195 @@
+module SerializationHelper
+
+ class Base
+ attr_reader :extension
+
+ def initialize(helper)
+ @dumper = helper.dumper
+ @loader = helper.loader
+ @extension = helper.extension
+ end
+
+ def dump(filename)
+ disable_logger
+ @dumper.dump(File.new(filename, "w"))
+ reenable_logger
+ end
+
+ def dump_to_dir(dirname)
+ Dir.mkdir(dirname)
+ tables = @dumper.tables
+ tables.each do |table|
+ io = File.new "#{dirname}/#{table}.#{@extension}", "w"
+ @dumper.before_table(io, table)
+ @dumper.dump_table io, table
+ @dumper.after_table(io, table)
+ end
+ end
+
+ def load(filename, truncate = true)
+ disable_logger
+ @loader.load(File.new(filename, "r"), truncate)
+ reenable_logger
+ end
+
+ def load_from_dir(dirname, truncate = true)
+ Dir.entries(dirname).each do |filename|
+ if filename =~ /^[.]/
+ next
+ end
+ @loader.load(File.new("#{dirname}/#{filename}", "r"), truncate)
+ end
+ end
+
+ def disable_logger
+ @@old_logger = ActiveRecord::Base.logger
+ ActiveRecord::Base.logger = nil
+ end
+
+ def reenable_logger
+ ActiveRecord::Base.logger = @@old_logger
+ end
+ end
+
+ class Load
+ def self.load(io, truncate = true)
+ ActiveRecord::Base.connection.transaction do
+ load_documents(io, truncate)
+ end
+ end
+
+ def self.truncate_table(table)
+ begin
+ ActiveRecord::Base.connection.execute("TRUNCATE #{SerializationHelper::Utils.quote_table(table)}")
+ rescue Exception
+ ActiveRecord::Base.connection.execute("DELETE FROM #{SerializationHelper::Utils.quote_table(table)}")
+ end
+ end
+
+ def self.load_table(table, data, truncate = true)
+ column_names = data['columns']
+ if truncate
+ truncate_table(table)
+ end
+ load_records(table, column_names, data['records'])
+ reset_pk_sequence!(table)
+ end
+
+ def self.load_records(table, column_names, records)
+ if column_names.nil?
+ return
+ end
+ columns = column_names.map{|cn| ActiveRecord::Base.connection.columns(table).detect{|c| c.name == cn}}
+ quoted_column_names = column_names.map { |column| ActiveRecord::Base.connection.quote_column_name(column) }.join(',')
+ quoted_table_name = SerializationHelper::Utils.quote_table(table)
+ records.each do |record|
+ quoted_values = record.zip(columns).map{|c| ActiveRecord::Base.connection.quote(c.first, c.last)}.join(',')
+ ActiveRecord::Base.connection.execute("INSERT INTO #{quoted_table_name} (#{quoted_column_names}) VALUES (#{quoted_values})")
+ end
+ end
+
+ def self.reset_pk_sequence!(table_name)
+ if ActiveRecord::Base.connection.respond_to?(:reset_pk_sequence!)
+ ActiveRecord::Base.connection.reset_pk_sequence!(table_name)
+ end
+ end
+
+
+ end
+
+ module Utils
+
+ def self.unhash(hash, keys)
+ keys.map { |key| hash[key] }
+ end
+
+ def self.unhash_records(records, keys)
+ records.each_with_index do |record, index|
+ records[index] = unhash(record, keys)
+ end
+
+ records
+ end
+
+ def self.convert_booleans(records, columns)
+ records.each do |record|
+ columns.each do |column|
+ next if is_boolean(record[column])
+ record[column] = (record[column] == 't' or record[column] == '1')
+ end
+ end
+ records
+ end
+
+ def self.boolean_columns(table)
+ columns = ActiveRecord::Base.connection.columns(table).reject { |c| silence_warnings { c.type != :boolean } }
+ columns.map { |c| c.name }
+ end
+
+ def self.is_boolean(value)
+ value.kind_of?(TrueClass) or value.kind_of?(FalseClass)
+ end
+
+ def self.quote_table(table)
+ ActiveRecord::Base.connection.quote_table_name(table)
+ end
+
+ end
+
+ class Dump
+ def self.before_table(io, table)
+
+ end
+
+ def self.dump(io)
+ tables.each do |table|
+ before_table(io, table)
+ dump_table(io, table)
+ after_table(io, table)
+ end
+ end
+
+ def self.after_table(io, table)
+
+ end
+
+ def self.tables
+ ActiveRecord::Base.connection.tables.reject { |table| ['schema_info', 'schema_migrations'].include?(table) }
+ end
+
+ def self.dump_table(io, table)
+ return if table_record_count(table).zero?
+
+ dump_table_columns(io, table)
+ dump_table_records(io, table)
+ end
+
+ def self.table_column_names(table)
+ ActiveRecord::Base.connection.columns(table).map { |c| c.name }
+ end
+
+
+ def self.each_table_page(table, records_per_page=1000)
+ total_count = table_record_count(table)
+ pages = (total_count.to_f / records_per_page).ceil - 1
+ id = table_column_names(table).first
+ boolean_columns = SerializationHelper::Utils.boolean_columns(table)
+ quoted_table_name = SerializationHelper::Utils.quote_table(table)
+
+ (0..pages).to_a.each do |page|
+ sql = ActiveRecord::Base.connection.add_limit_offset!("SELECT * FROM #{quoted_table_name} ORDER BY #{id}",
+ :limit => records_per_page, :offset => records_per_page * page
+ )
+ records = ActiveRecord::Base.connection.select_all(sql)
+ records = SerializationHelper::Utils.convert_booleans(records, boolean_columns)
+ yield records
+ end
+ end
+
+ def self.table_record_count(table)
+ ActiveRecord::Base.connection.select_one("SELECT COUNT(*) FROM #{SerializationHelper::Utils.quote_table(table)}").values.first.to_i
+ end
+
+ end
+
+end
View
237 vendor/plugins/yaml_db/lib/yaml_db.rb
@@ -1,180 +1,67 @@
require 'rubygems'
require 'yaml'
require 'active_record'
-
+require 'serialization_helper'
module YamlDb
- def self.dump(filename)
- disable_logger
- YamlDb::Dump.dump(File.new(filename, "w"))
- reenable_logger
- end
-
- def self.load(filename)
- disable_logger
- YamlDb::Load.load(File.new(filename, "r"))
- reenable_logger
- end
-
- def self.disable_logger
- @@old_logger = ActiveRecord::Base.logger
- ActiveRecord::Base.logger = nil
- end
-
- def self.reenable_logger
- ActiveRecord::Base.logger = @@old_logger
- end
-end
-
-
-module YamlDb::Utils
- def self.chunk_records(records)
- yaml = [ records ].to_yaml
- yaml.sub!("--- \n", "")
- yaml.sub!('- - -', ' - -')
- yaml
- end
-
- def self.unhash(hash, keys)
- keys.map { |key| hash[key] }
- end
-
- def self.unhash_records(records, keys)
- records.each_with_index do |record, index|
- records[index] = unhash(record, keys)
- end
-
- records
- end
-
- def self.convert_booleans(records, columns)
- records.each do |record|
- columns.each do |column|
- next if is_boolean(record[column])
- record[column] = (record[column] == 't' or record[column] == '1')
- end
- end
- records
- end
-
- def self.boolean_columns(table)
- columns = ActiveRecord::Base.connection.columns(table).reject { |c| c.type != :boolean }
- columns.map { |c| c.name }
- end
-
- def self.is_boolean(value)
- value.kind_of?(TrueClass) or value.kind_of?(FalseClass)
- end
-
- def self.quote_table(table)
- ActiveRecord::Base.connection.quote_table_name(table)
- end
-end
-
-
-module YamlDb::Dump
- def self.dump(io)
- tables.each do |table|
- dump_table(io, table)
- end
- end
-
- def self.tables
- ActiveRecord::Base.connection.tables.reject { |table| ['schema_info', 'schema_migrations'].include?(table) }
- end
-
- def self.dump_table(io, table)
- return if table_record_count(table).zero?
-
- dump_table_columns(io, table)
- dump_table_records(io, table)
- end
-
- def self.dump_table_columns(io, table)
- io.write("\n")
- io.write({ table => { 'columns' => table_column_names(table) } }.to_yaml)
- end
-
- def self.dump_table_records(io, table)
- table_record_header(io)
-
- column_names = table_column_names(table)
-
- each_table_page(table) do |records|
- rows = YamlDb::Utils.unhash_records(records, column_names)
- io.write(YamlDb::Utils.chunk_records(records))
- end
- end
-
- def self.table_record_header(io)
- io.write(" records: \n")
- end
-
- def self.table_column_names(table)
- ActiveRecord::Base.connection.columns(table).map { |c| c.name }
- end
-
- def self.each_table_page(table, records_per_page=1000)
- total_count = table_record_count(table)
- pages = (total_count.to_f / records_per_page).ceil - 1
- id = table_column_names(table).first
- boolean_columns = YamlDb::Utils.boolean_columns(table)
- quoted_table_name = YamlDb::Utils.quote_table(table)
-
- (0..pages).to_a.each do |page|
- sql = ActiveRecord::Base.connection.add_limit_offset!("SELECT * FROM #{quoted_table_name} ORDER BY #{id}",
- :limit => records_per_page, :offset => records_per_page * page
- )
- records = ActiveRecord::Base.connection.select_all(sql)
- records = YamlDb::Utils.convert_booleans(records, boolean_columns)
- yield records
- end
- end
-
- def self.table_record_count(table)
- ActiveRecord::Base.connection.select_one("SELECT COUNT(*) FROM #{YamlDb::Utils.quote_table(table)}").values.first.to_i
- end
-end
-
-
-module YamlDb::Load
- def self.load(io)
- ActiveRecord::Base.connection.transaction do
- YAML.load_documents(io) do |ydoc|
- ydoc.keys.each do |table_name|
- next if ydoc[table_name].nil?
- load_table(table_name, ydoc[table_name])
- end
- end
- end
- end
-
- def self.truncate_table(table)
- begin
- ActiveRecord::Base.connection.execute("TRUNCATE #{YamlDb::Utils.quote_table(table)}")
- rescue Exception
- ActiveRecord::Base.connection.execute("DELETE FROM #{YamlDb::Utils.quote_table(table)}")
- end
- end
-
- def self.load_table(table, data)
- column_names = data['columns']
- truncate_table(table)
- load_records(table, column_names, data['records'])
- reset_pk_sequence!(table)
- end
-
- def self.load_records(table, column_names, records)
- quoted_column_names = column_names.map { |column| ActiveRecord::Base.connection.quote_column_name(column) }.join(',')
- quoted_table_name = YamlDb::Utils.quote_table(table)
- records.each do |record|
- ActiveRecord::Base.connection.execute("INSERT INTO #{quoted_table_name} (#{quoted_column_names}) VALUES (#{record.map { |r| ActiveRecord::Base.connection.quote(r) }.join(',')})")
- end
- end
-
- def self.reset_pk_sequence!(table_name)
- if ActiveRecord::Base.connection.respond_to?(:reset_pk_sequence!)
- ActiveRecord::Base.connection.reset_pk_sequence!(table_name)
- end
- end
-end
+ module Helper
+ def self.loader
+ YamlDb::Load
+ end
+
+ def self.dumper
+ YamlDb::Dump
+ end
+
+ def self.extension
+ "yml"
+ end
+ end
+
+
+ module Utils
+ def self.chunk_records(records)
+ yaml = [ records ].to_yaml
+ yaml.sub!("--- \n", "")
+ yaml.sub!('- - -', ' - -')
+ yaml
+ end
+
+ end
+
+ class Dump < SerializationHelper::Dump
+
+ def self.dump_table_columns(io, table)
+ io.write("\n")
+ io.write({ table => { 'columns' => table_column_names(table) } }.to_yaml)
+ end
+
+ def self.dump_table_records(io, table)
+ table_record_header(io)
+
+ column_names = table_column_names(table)
+
+ each_table_page(table) do |records|
+ rows = SerializationHelper::Utils.unhash_records(records, column_names)
+ io.write(YamlDb::Utils.chunk_records(records))
+ end
+ end
+
+ def self.table_record_header(io)
+ io.write(" records: \n")
+ end
+
+ end
+
+ class Load < SerializationHelper::Load
+ def self.load_documents(io, truncate = true)
+ YAML.load_documents(io) do |ydoc|
+ ydoc.keys.each do |table_name|
+ next if ydoc[table_name].nil?
+ load_table(table_name, ydoc[table_name], truncate)
+ end
+ end
+ end
+ end
+
+end
View
3  vendor/plugins/yaml_db/spec/base.rb
@@ -3,5 +3,4 @@
$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../lib')
require 'yaml_db'
-
-
+require 'serialization_helper'
View
53 vendor/plugins/yaml_db/spec/serialization_helper_base_spec.rb
@@ -0,0 +1,53 @@
+require File.dirname(__FILE__) + '/base'
+
+describe SerializationHelper::Base do
+
+ before do
+ @io = StringIO.new
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
+ ActiveRecord::Base.connection = mock('connection')
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
+ end
+
+ def stub_helper!
+ @helper = mock("MyHelper")
+ @dumper = mock("MyDumper");
+ @loader = mock("MyLoader");
+ @helper.stub!(:dumper).and_return(@dumper)
+ @helper.stub!(:loader).and_return(@loader)
+ @helper.stub!(:extension).and_return("yml")
+ @dumper.stub!(:tables).and_return([ActiveRecord::Base.connection.tables[0]])
+ @dumper.stub!(:before_table).and_return(nil)
+ @dumper.stub!(:after_table).and_return(nil)
+ end
+
+ context "for multi-file dumps" do
+ before do
+ File.should_receive(:new).once.with("dir_name/mytable.yml", "w").and_return(@io)
+ Dir.should_receive(:mkdir).once.with("dir_name")
+ stub_helper!
+ @dumper.should_receive(:dump_table).once.with(@io, "mytable")
+ end
+
+ it "should create the number of files that there are tables" do
+ SerializationHelper::Base.new(@helper).dump_to_dir "dir_name"
+ end
+
+ end
+
+ context "for multi-file loads" do
+
+ before do
+ stub_helper!
+ @loader.should_receive(:load).once.with(@io, true)
+ File.should_receive(:new).once.with("dir_name/mytable.yml", "r").and_return(@io)
+ Dir.stub!(:entries).and_return(["mytable.yml"])
+ end
+
+ it "should insert into then umber of tables that there are files" do
+ SerializationHelper::Base.new(@helper).load_from_dir "dir_name"
+ end
+
+ end
+
+end
View
65 vendor/plugins/yaml_db/spec/serialization_helper_dump_spec.rb
@@ -0,0 +1,65 @@
+require File.dirname(__FILE__) + '/base'
+
+describe SerializationHelper::Dump do
+
+ before do
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
+ ActiveRecord::Base.connection = mock('connection')
+ ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a'), mock('b', :name => 'b') ])
+ ActiveRecord::Base.connection.stub!(:select_one).and_return({"count"=>"2"})
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ])
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
+ end
+
+ before(:each) do
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
+ @io = StringIO.new
+ end
+
+ it "should return a list of column names" do
+ SerializationHelper::Dump.table_column_names('mytable').should == [ 'a', 'b' ]
+ end
+
+ it "should return a list of tables without the rails schema table" do
+ SerializationHelper::Dump.tables.should == ['mytable']
+ end
+
+ it "should return the total number of records in a table" do
+ SerializationHelper::Dump.table_record_count('mytable').should == 2
+ end
+
+ it "should return all records from the database and return them when there is only 1 page" do
+ SerializationHelper::Dump.each_table_page('mytable') do |records|
+ records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
+ end
+ end
+
+ it "should paginate records from the database and return them" do
+ ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 } ], [ { 'a' => 3, 'b' => 4 } ])
+
+ records = [ ]
+ SerializationHelper::Dump.each_table_page('mytable', 1) do |page|
+ page.size.should == 1
+ records.concat(page)
+ end
+
+ records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
+ end
+
+ it "should dump a table's contents to yaml" do
+ SerializationHelper::Dump.should_receive(:dump_table_columns)
+ SerializationHelper::Dump.should_receive(:dump_table_records)
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
+ end
+
+ it "should not dump a table's contents when the record count is zero" do
+ SerializationHelper::Dump.stub!(:table_record_count).with('mytable').and_return(0)
+ SerializationHelper::Dump.should_not_receive(:dump_table_columns)
+ SerializationHelper::Dump.should_not_receive(:dump_table_records)
+ SerializationHelper::Dump.dump_table(@io, 'mytable')
+ end
+
+
+
+end
View
77 vendor/plugins/yaml_db/spec/serialization_helper_load_spec.rb
@@ -0,0 +1,77 @@
+require File.dirname(__FILE__) + '/base'
+
+describe SerializationHelper::Load do
+ before do
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
+
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
+ ActiveRecord::Base.connection = mock('connection')
+ ActiveRecord::Base.connection.stub!(:transaction).and_yield
+ @io = StringIO.new
+ end
+
+ it "should truncate the table" do
+ ActiveRecord::Base.connection.stub!(:execute).with("TRUNCATE mytable").and_return(true)
+ ActiveRecord::Base.connection.should_not_receive(:execute).with("DELETE FROM mytable")
+ SerializationHelper::Load.truncate_table('mytable')
+ end
+
+ it "should delete the table if truncate throws an exception" do
+ ActiveRecord::Base.connection.should_receive(:execute).with("TRUNCATE mytable").and_raise()
+ ActiveRecord::Base.connection.should_receive(:execute).with("DELETE FROM mytable").and_return(true)
+ SerializationHelper::Load.truncate_table('mytable')
+ end
+
+
+ it "should call reset pk sequence if the connection adapter is postgres" do
+ ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(true)
+ ActiveRecord::Base.connection.should_receive(:reset_pk_sequence!).with('mytable')
+ SerializationHelper::Load.reset_pk_sequence!('mytable')
+ end
+
+ it "should not call reset pk sequence for other adapters" do
+ ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(false)
+ ActiveRecord::Base.connection.should_not_receive(:reset_pk_sequence!)
+ SerializationHelper::Load.reset_pk_sequence!('mytable')
+ end
+
+ it "should insert records into a table" do
+ mca = mock('a',:name => 'a')
+ mcb = mock('b', :name => 'b')
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([mca , mcb ])
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('b').and_return('b')
+ ActiveRecord::Base.connection.stub!(:quote).with(1, mca).and_return("'1'")
+ ActiveRecord::Base.connection.stub!(:quote).with(2, mcb).and_return("'2'")
+ ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
+ ActiveRecord::Base.connection.stub!(:quote).with(4, mcb).and_return("'4'")
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('1','2')")
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('3','4')")
+
+ SerializationHelper::Load.load_records('mytable', ['a', 'b'], [[1, 2], [3, 4]])
+ end
+
+ it "should quote column names that correspond to sql keywords" do
+ mca = mock('a',:name => 'a')
+ mccount = mock('count', :name => 'count')
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([mca , mccount ])
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
+ ActiveRecord::Base.connection.stub!(:quote_column_name).with('count').and_return('"count"')
+ ActiveRecord::Base.connection.stub!(:quote).with(1, mca).and_return("'1'")
+ ActiveRecord::Base.connection.stub!(:quote).with(2, mccount).and_return("'2'")
+ ActiveRecord::Base.connection.stub!(:quote).with(3, mca).and_return("'3'")
+ ActiveRecord::Base.connection.stub!(:quote).with(4, mccount).and_return("'4'")
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('1','2')")
+ ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('3','4')")
+
+ SerializationHelper::Load.load_records('mytable', ['a', 'count'], [[1, 2], [3, 4]])
+ end
+
+ it "should truncate the table and then load the records into the table" do
+ SerializationHelper::Load.should_receive(:truncate_table).with('mytable')
+ SerializationHelper::Load.should_receive(:load_records).with('mytable', ['a', 'b'], [[1, 2], [3, 4]])
+ SerializationHelper::Load.should_receive(:reset_pk_sequence!).with('mytable')
+
+ SerializationHelper::Load.load_table('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] })
+ end
+end
View
31 vendor/plugins/yaml_db/spec/serialization_utils_spec.rb
@@ -0,0 +1,31 @@
+require File.dirname(__FILE__) + '/base'
+
+describe SerializationHelper::Utils, " convert records utility method" do
+ before do
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
+ ActiveRecord::Base.connection = mock('connection')
+ end
+
+ it "returns an array of hash values using an array of ordered keys" do
+ SerializationHelper::Utils.unhash({ 'a' => 1, 'b' => 2 }, [ 'b', 'a' ]).should == [ 2, 1 ]
+ end
+
+ it "should unhash each hash an array using an array of ordered keys" do
+ SerializationHelper::Utils.unhash_records([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ], [ 'b', 'a' ]).should == [ [ 2, 1 ], [ 4, 3 ] ]
+ end
+
+ it "should return true if it is a boolean type" do
+ SerializationHelper::Utils.is_boolean(true).should == true
+ SerializationHelper::Utils.is_boolean('true').should_not == true
+ end
+
+ it "should return an array of boolean columns" do
+ ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a',:type => :string), mock('b', :name => 'b',:type => :boolean) ])
+ SerializationHelper::Utils.boolean_columns('mytable').should == ['b']
+ end
+
+ it "should quote the table name" do
+ ActiveRecord::Base.connection.should_receive(:quote_table_name).with('values').and_return('`values`')
+ SerializationHelper::Utils.quote_table('values').should == '`values`'
+ end
+end
View
51 vendor/plugins/yaml_db/spec/yaml_dump_spec.rb
@@ -1,10 +1,9 @@
require File.dirname(__FILE__) + '/base'
describe YamlDb::Dump do
- before do
- File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
- ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true)
+ before do
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
ActiveRecord::Base.connection = mock('connection')
ActiveRecord::Base.connection.stub!(:tables).and_return([ 'mytable', 'schema_info', 'schema_migrations' ])
ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a'), mock('b', :name => 'b') ])
@@ -13,8 +12,9 @@
YamlDb::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
end
- before(:each) do
- @io = StringIO.new
+ before(:each) do
+ File.stub!(:new).with('dump.yml', 'w').and_return(StringIO.new)
+ @io = StringIO.new
end
it "should return a formatted string" do
@@ -23,17 +23,6 @@
@io.read.should == " records: \n"
end
- it "should return a list of column names" do
- YamlDb::Dump.table_column_names('mytable').should == [ 'a', 'b' ]
- end
-
- it "should return a list of tables without the rails schema table" do
- YamlDb::Dump.tables.should == ['mytable']
- end
-
- it "should return the total number of records in a table" do
- YamlDb::Dump.table_record_count('mytable').should == 2
- end
it "should return a yaml string that contains a table header and column names" do
YamlDb::Dump.stub!(:table_column_names).with('mytable').and_return([ 'a', 'b' ])
@@ -49,24 +38,6 @@
EOYAML
end
- it "should return all records from the database and return them when there is only 1 page" do
- YamlDb::Dump.each_table_page('mytable') do |records|
- records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
- end
- end
-
- it "should paginate records from the database and return them" do
- ActiveRecord::Base.connection.stub!(:select_all).and_return([ { 'a' => 1, 'b' => 2 } ], [ { 'a' => 3, 'b' => 4 } ])
-
- records = [ ]
- YamlDb::Dump.each_table_page('mytable', 1) do |page|
- page.size.should == 1
- records.concat(page)
- end
-
- records.should == [ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ]
- end
-
it "should return dump the records for a table in yaml to a given io stream" do
YamlDb::Dump.dump_table_records(@io, 'mytable')
@io.rewind
@@ -79,16 +50,6 @@
EOYAML
end
- it "should dump a table's contents to yaml" do
- YamlDb::Dump.should_receive(:dump_table_columns)
- YamlDb::Dump.should_receive(:dump_table_records)
- YamlDb::Dump.dump_table(@io, 'mytable')
- end
- it "should not dump a table's contents when the record count is zero" do
- YamlDb::Dump.stub!(:table_record_count).with('mytable').and_return(0)
- YamlDb::Dump.should_not_receive(:dump_table_columns)
- YamlDb::Dump.should_not_receive(:dump_table_records)
- YamlDb::Dump.dump_table(@io, 'mytable')
- end
+
end
View
68 vendor/plugins/yaml_db/spec/yaml_load_spec.rb
@@ -2,69 +2,24 @@
describe YamlDb::Load do
before do
- YamlDb::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
+ SerializationHelper::Utils.stub!(:quote_table).with('mytable').and_return('mytable')
- ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true)
+ silence_warnings { ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true) }
ActiveRecord::Base.connection = mock('connection')
ActiveRecord::Base.connection.stub!(:transaction).and_yield
end
before(:each) do
@io = StringIO.new
- end
-
- it "should truncate the table" do
- ActiveRecord::Base.connection.stub!(:execute).with("TRUNCATE mytable").and_return(true)
- ActiveRecord::Base.connection.should_not_receive(:execute).with("DELETE FROM mytable")
- YamlDb::Load.truncate_table('mytable')
- end
-
- it "should delete the table if truncate throws an exception" do
- ActiveRecord::Base.connection.should_receive(:execute).with("TRUNCATE mytable").and_raise()
- ActiveRecord::Base.connection.should_receive(:execute).with("DELETE FROM mytable").and_return(true)
- YamlDb::Load.truncate_table('mytable')
- end
-
- it "should insert records into a table" do
- ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
- ActiveRecord::Base.connection.stub!(:quote_column_name).with('b').and_return('b')
- ActiveRecord::Base.connection.stub!(:quote).with(1).and_return("'1'")
- ActiveRecord::Base.connection.stub!(:quote).with(2).and_return("'2'")
- ActiveRecord::Base.connection.stub!(:quote).with(3).and_return("'3'")
- ActiveRecord::Base.connection.stub!(:quote).with(4).and_return("'4'")
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('1','2')")
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,b) VALUES ('3','4')")
-
- YamlDb::Load.load_records('mytable', ['a', 'b'], [[1, 2], [3, 4]])
- end
-
- it "should quote column names that correspond to sql keywords" do
- ActiveRecord::Base.connection.stub!(:quote_column_name).with('a').and_return('a')
- ActiveRecord::Base.connection.stub!(:quote_column_name).with('count').and_return('"count"')
- ActiveRecord::Base.connection.stub!(:quote).with(1).and_return("'1'")
- ActiveRecord::Base.connection.stub!(:quote).with(2).and_return("'2'")
- ActiveRecord::Base.connection.stub!(:quote).with(3).and_return("'3'")
- ActiveRecord::Base.connection.stub!(:quote).with(4).and_return("'4'")
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('1','2')")
- ActiveRecord::Base.connection.should_receive(:execute).with("INSERT INTO mytable (a,\"count\") VALUES ('3','4')")
-
- YamlDb::Load.load_records('mytable', ['a', 'count'], [[1, 2], [3, 4]])
- end
-
- it "should truncate the table and then load the records into the table" do
- YamlDb::Load.should_receive(:truncate_table).with('mytable')
- YamlDb::Load.should_receive(:load_records).with('mytable', ['a', 'b'], [[1, 2], [3, 4]])
- YamlDb::Load.should_receive(:reset_pk_sequence!).with('mytable')
-
- YamlDb::Load.load_table('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] })
- end
+ end
+
it "should call load structure for each document in the file" do
YAML.should_receive(:load_documents).with(@io).and_yield({ 'mytable' => {
'columns' => [ 'a', 'b' ],
'records' => [[1, 2], [3, 4]]
- } })
- YamlDb::Load.should_receive(:load_table).with('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] })
+ } } )
+ YamlDb::Load.should_receive(:load_table).with('mytable', { 'columns' => [ 'a', 'b' ], 'records' => [[1, 2], [3, 4]] },true)
YamlDb::Load.load(@io)
end
@@ -74,15 +29,4 @@
YamlDb::Load.load(@io)
end
- it "should call reset pk sequence if the connection adapter is postgres" do
- ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(true)
- ActiveRecord::Base.connection.should_receive(:reset_pk_sequence!).with('mytable')
- YamlDb::Load.reset_pk_sequence!('mytable')
- end
-
- it "should not call reset pk sequence for other adapters" do
- ActiveRecord::Base.connection.should_receive(:respond_to?).with(:reset_pk_sequence!).and_return(false)
- ActiveRecord::Base.connection.should_not_receive(:reset_pk_sequence!)
- YamlDb::Load.reset_pk_sequence!('mytable')
- end
end
View
26 vendor/plugins/yaml_db/spec/yaml_utils_spec.rb
@@ -1,10 +1,6 @@
require File.dirname(__FILE__) + '/base'
describe YamlDb::Utils, " convert records utility method" do
- before do
- ActiveRecord::Base = mock('ActiveRecord::Base', :null_object => true)
- ActiveRecord::Base.connection = mock('connection')
- end
it "turns an array with one record into a yaml chunk" do
YamlDb::Utils.chunk_records([ %w(a b) ]).should == <<EOYAML
@@ -22,26 +18,4 @@
EOYAML
end
- it "returns an array of hash values using an array of ordered keys" do
- YamlDb::Utils.unhash({ 'a' => 1, 'b' => 2 }, [ 'b', 'a' ]).should == [ 2, 1 ]
- end
-
- it "should unhash each hash an array using an array of ordered keys" do
- YamlDb::Utils.unhash_records([ { 'a' => 1, 'b' => 2 }, { 'a' => 3, 'b' => 4 } ], [ 'b', 'a' ]).should == [ [ 2, 1 ], [ 4, 3 ] ]
- end
-
- it "should return true if it is a boolean type" do
- YamlDb::Utils.is_boolean(true).should == true
- YamlDb::Utils.is_boolean('true').should_not == true
- end
-
- it "should return an array of boolean columns" do
- ActiveRecord::Base.connection.stub!(:columns).with('mytable').and_return([ mock('a',:name => 'a',:type => :string), mock('b', :name => 'b',:type => :boolean) ])
- YamlDb::Utils.boolean_columns('mytable').should == ['b']
- end
-
- it "should quote the table name" do
- ActiveRecord::Base.connection.should_receive(:quote_table_name).with('values').and_return('`values`')
- YamlDb::Utils.quote_table('values').should == '`values`'
- end
end
View
38 vendor/plugins/yaml_db/tasks/yaml_db_tasks.rake
@@ -6,18 +6,40 @@ namespace :db do
task(:load => [ "db:schema:load", "db:data:load" ])
namespace :data do
- def db_dump_data_file
- "#{RAILS_ROOT}/db/data.yml"
+ def db_dump_data_file (extension = "yml")
+ "#{dump_dir}/data.#{extension}"
+ end
+
+ def dump_dir(dir = "")
+ "#{RAILS_ROOT}/db#{dir}"
+ end
+
+ desc "Dump contents of database to db/data.extension (defaults to yaml)"
+ task :dump => :environment do
+ format_class = ENV['class'] || "YamlDb::Helper"
+ helper = format_class.constantize
+ SerializationHelper::Base.new(helper).dump db_dump_data_file helper.extension
+ end
+
+ desc "Dump contents of database to curr_dir_name/tablename.extension (defaults to yaml)"
+ task :dump_dir => :environment do
+ format_class = ENV['class'] || "YamlDb::Helper"
+ dir = ENV['dir'] || "#{Time.now.to_s.gsub(/ /, '_')}"
+ SerializationHelper::Base.new(format_class.constantize).dump_to_dir dump_dir("/#{dir}")
end
- desc "Dump contents of database to db/data.yml"
- task(:dump => :environment) do
- YamlDb.dump db_dump_data_file
+ desc "Load contents of db/data.extension (defaults to yaml) into database"
+ task :load => :environment do
+ format_class = ENV['class'] || "YamlDb::Helper"
+ helper = format_class.constantize
+ SerializationHelper::Base.new(helper).load (db_dump_data_file helper.extension)
end
- desc "Load contents of db/data.yml into database"
- task(:load => :environment) do
- YamlDb.load db_dump_data_file
+ desc "Load contents of db/data_dir into database"
+ task :load_dir => :environment do
+ dir = ENV['dir'] || "base"
+ format_class = ENV['class'] || "YamlDb::Helper"
+ SerializationHelper::Base.new(format_class.constantize).load_from_dir dump_dir("/#{dir}")
end
end
end
View
67 vendor/plugins/yaml_db/yaml_db.gemspec
@@ -0,0 +1,67 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE DIRECTLY
+# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+ s.name = %q{yaml_db}
+ s.version = "0.1.0"
+
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+ s.authors = ["Adam Wiggins", "Orion Henry"]
+ s.date = %q{2009-12-27}
+ s.description = %q{
+YamlDb is a database-independent format for dumping and restoring data. It complements the the database-independent schema format found in db/schema.rb. The data is saved into db/data.yml.
+This can be used as a replacement for mysqldump or pg_dump, but only for the databases typically used by Rails apps. Users, permissions, schemas, triggers, and other advanced database features are not supported - by design.
+Any database that has an ActiveRecord adapter should work
+}
+ s.email = %q{nate@ludicast.com}
+ s.extra_rdoc_files = [
+ "README"
+ ]
+ s.files = [
+ "README",
+ "Rakefile",
+ "VERSION",
+ "about.yml",
+ "init.rb",
+ "lib/csv_db.rb",
+ "lib/serialization_helper.rb",
+ "lib/yaml_db.rb",
+ "spec/base.rb",
+ "spec/serialization_helper_base_spec.rb",
+ "spec/serialization_helper_dump_spec.rb",
+ "spec/serialization_helper_load_spec.rb",
+ "spec/serialization_utils_spec.rb",
+ "spec/yaml_dump_spec.rb",
+ "spec/yaml_load_spec.rb",
+ "spec/yaml_utils_spec.rb",
+ "tasks/yaml_db_tasks.rake"
+ ]
+ s.homepage = %q{http://github.com/ludicast/yaml_db}
+ s.rdoc_options = ["--charset=UTF-8"]
+ s.require_paths = ["lib"]
+ s.rubygems_version = %q{1.3.5}
+ s.summary = %q{yaml_db allows export/import of database into/from yaml files}
+ s.test_files = [
+ "spec/base.rb",
+ "spec/serialization_helper_base_spec.rb",
+ "spec/serialization_helper_dump_spec.rb",
+ "spec/serialization_helper_load_spec.rb",
+ "spec/serialization_utils_spec.rb",
+ "spec/yaml_dump_spec.rb",
+ "spec/yaml_load_spec.rb",
+ "spec/yaml_utils_spec.rb"
+ ]
+
+ if s.respond_to? :specification_version then
+ current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+ s.specification_version = 3
+
+ if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+ else
+ end
+ else
+ end
+end
+
Please sign in to comment.
Something went wrong with that request. Please try again.