Reads CSV schema dump and populates a fixture file for Rails
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


This is a command line utility that reads CSV schema dump and creates fixture files for Rails application. We used it to migrate an old database of an Django-based application to a new Rails-based application. Since the schemas are different, we need a configurable utility that can map an old table to new table(s).

To use this utility

  • Export your old schema in CSV format using a utility such as Sequel Pro. The CSV files should have headers as their first row.
  • Write configuration file for tables that need to be mapped to new tables
  • Write custom attribute mapping for fields of old tables that do not map directly to new fields
  • Run the utlity

Writing configuration files

configuration file for each old table needs to be written in this format:

  new_table_field_name: old_table_field_name

The name of the configuration file should be same as the old table data dump file, except for the file suffix. So, if the old table dump file is "xxx_customers.csv", the configuration for that table should be named as "xxx_customers.conf"

Note: Django-based apps have a common records table which is named "commons_commoninfo", which needs to be joined with each tables and then processed together. The tables which need to be joined with this "commoninfo" table have a field named "commoninfo_ptr_id". For such tables, the old_table_field_name should be the field name in the commoninfo table field name. And the "commoninfo_ptr_id" should be written as "id" as that's what is the name of the field in commons_commoninfo table.

Writing custom attribute mapping config

Also, we had some fields which needed some custom processing based on their current value. For example, the old table field may be a status value which needs to be looked into a table to find its new status value. Such custom attribute mapping needs to be described in a config file that should be named as "custom_attribute_mapping.conf" in the same schema directory.

The custom attribute mapping file should be written in this format:

    old_field_value: new_field_value

So, basically, in this file, the we need to describe the mapping of the old field value to the new field value.

Running the program

Install perl, perlbrew and cpanm. And then install the cpan modules for yaml and CSV processing:

cpanm YAML::Syck
cpanm Text::CSV_XS

Run the application like this:

./ <schema_directory_location>