Skip to content

iolo/mongolfier

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DEPRECATION NOTICE

This project is abandoned.

mongolfier

A Simple MySQL to MongoDB Migration Tool

Montgolfier Balloon

NOTE: not "montgolfier" but "mongolfier". ;)

install and run

local install and run

$ npm install mongolfier
$ node ./node_modules/.bin/mongolfier --help

global install and run

$ npm install -g mongolfier
$ mongolfier --help

simple migration with bulk-copy

To copy all rows from users table on test mysql database into users collection on test mongo db.

$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b users

NOTE use -d flag to test WITHOUT real insert operations:

$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b -d users

NOTE use -e flag to remove all existing documents in the collection:

$ mongolfier -s mysql://root@localhost/test -o mongodb://localhost/test -b -e users

complex migration with template(EXPERIMENTAL)

For complex migration, you need to write a config file and template files and use -c option:

$ mongolfier -c config.json

usage

$ mongolfier -help
  Usage: mongolfier [OPTIONS] [COLLECTION_NAME ...]

  Options:

    -h, --help              output usage information
    -V, --version           output the version number
    -c, --config [FILE]     path to config file(js or json)
    -s, --mysql [URL]       mysql connection url
    -o, --mongo [URL]       mongodb connection url
    -e, --empty-collection  make empty collection before migration
    -b, --bulk-copy         do bulk copy if no template available
    -d, --dry-run           do not insert into collection

configuration

config file(json)

  • NOTE: no comment allowed here. this file should conforms to strict json format.
  • NOTE: {{ and }} is just a marker. replace it with yours.
{
  "mysql": "mysql://user:password@host:port/database",
  "mongo": "mongodb://user:password@host:port/db",
  "context": {
    "{{custom_attr_key}}": "{{custom_attr_value}}",
    ...
  },
  "before": [
    "{{before_script.js}}",
    ...
  ],
  "after": [
    "{{after_script.js}}",
    ...
  ]
  "collections": [
    {
      "collection": "{{mongo_collection_name}}",
      "template": "{{path_to_collection_template}}",
      "query": "{{mysql_select_query}}"
    },
    {
      "collection": "{{mongo_collection_name}}",
      "template": "{{path_to_collection_template}}",
      "query": [
        "{{mysql_select_query_part1}}",
        "{{mysql_select_query_part2}}",
        ...
      ]
    },
    ...
  ]
}
  • context is optional. this could be accessed via $ variable across processing all collections.
  • before is optional. these scripts are evalulated before processing the first collection.
  • after is optional. these scripts are evalulated after processing the last collection.
  • template is optional(default: {{collection name}}.json).
  • query is string or array. array will be joined to a query.

mapping template/script

NOTE: comment allowed here. this file is javascript or something. ;)

NOTE: {{ and }} is just a marker. replace it with yours.

simple mapping template

({
  // use mongodb ObjectID
  "_id": new mongo.ObjectID(),
  // use mysql column values
  "{{mongo_field_name}}": ${{mysql_column_name}},
  ...
})

complex mapping template

var id = new mongo.ObjectID();
...
({
  "_id": id,
  "foo": ($bar).toUpperCase(),
  "sum": $num1 + $num2,
  "now": new Date(),
  ...
})
  • NOTE on the enclosing braces the result. do not use return keyword.

async mapping template(EXPERIMENTAL)

var d = q.defer();
var id = new mongo.ObjectID();
setTimeout(function () {
  ...
  d.resolve({
    "_id": id,
    "foo": ($bar).toUpperCase(),
    "sum": $num1 + $num2,
    "now": new Date(),
    ...
  })
}, 100);
d.promise;
  • NOTE on the last line d.promise;. do not use return keyword.

predefined objects

TBD... ;)


That's all folks.

May the SOURCE be with you...

About

**DEPRECATED** mysql to mongodb migration tool

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published