The ultimate solution for populating your MongoDB database.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.ci
cli
core
docker-image
docs
examples
.dockerignore
.gitignore
.prettierrc
.travis.yml
CONTRIBUTING.md
LICENSE
README.md
lerna.json
package-lock.json
package.json
tslint.json

README.md

Mongo Seeding

Mongo Seeding

GitHub release Build Status MIT license

The ultimate solution for populating your MongoDB database 🚀

Define MongoDB documents in JSON, JavaScript or even TypeScript file(s). Use JS library, install CLI or run Docker image to import them!

Introduction

Mongo Seeding is a flexible set of tools for importing data into MongoDB database.

It's great for:

  • testing database queries, automatically or manually
  • preparing ready-to-go development environment for your application
  • setting initial state for your application

How does it work?

  1. Define documents for MongoDB import in JSON, JavaScript or TypeScript file(s). To learn, how to do that, read the import data definition guide. To see some examples, navigate to the examples directory.

  2. Use one of the Mongo Seeding tools, depending on your needs:

  3. ???

  4. Profit!

Motivation

There are many tools for MongoDB data import out there, including the official one - mongoimport. Why should you choose Mongo Seeding?

Problem #1: JSON used for import data definition

Every tool I found before creating Mongo Seeding support only JSON files. In my opinion, that is not the most convenient way of data definition. The biggest problems are data redundancy and lack of ability to write logic.

Imagine that you want to import 10 very similar documents into authors collection. Every document is identical - except the name:

{
    "name": "{NAME_HERE}",
    "email": "example@example.com",
    "avatar": "https://placekitten.com/300/300"
}

With every tool I've ever found, you would need to create 10 separate JSON files, or one file with array of objects. Of course, the latter option is better, but anyway you end up with a file looking like this:

[
    {
        "name": "John",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Joanne",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Bob",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Will",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Chris",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Mike",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Anna",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Jack",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Peter",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    },
    {
        "name": "Paul",
        "email": "example@example.com",
        "avatar": "https://placekitten.com/300/300"
    }
]

That doesn't look good - you did probably hear about DRY principle.

Imagine that now you have to change authors' email. You would probably use search and replace. But what if you would need change the data shape completely? This time you can also use IDE features like multiple cursors etc., but hey - it's a waste of time. What if you had a much more complicated data shape?

If you could use JavaScript to define the authors documents, it would be much easier and faster to write something like this:

const names = ["John", "Joanne", "Bob", "Will", "Chris", "Mike", "Anna", "Jack", "Peter", "Paul"];

module.exports = names.map(name => ({
    name,
    email: "example@example.com",
    avatar: "https://placekitten.com/300/300",
}))

Obviously, in JavaScript files you can also import other files - external libraries, helper methods etc. It's easy to write some data randomization rules - which are mostly essential for creating development sample data. Consider the following example of people collection import:

const { getObjectId } = require("../../helpers/index");

const names = ["John", "Joanne", "Bob", "Will", "Chris", "Mike", "Anna", "Jack", "Peter", "Paul"];

const min = 18;
const max = 100;

module.exports = names.map(name => ({
    firstName: name,
    age: Math.floor(Math.random() * (max - min + 1)) + min,
    _id: getObjectId(name),
}))

The difference should be noticeable. This way of defining import data feels just right. And yes, you can do that in Mongo Seeding. But, JSON files are supported as well.

Problem #2: No data model validation

In multiple JSON files which contains MongoDB documents definition, it's easy to make a mistake, especially in complex data structure. Sometimes a typo results in invalid data. See the example below for people collection definition:

[
    {
        "name": "John",
        "email": "john@mail.de",
        "age": 18,
    },
    {
        "name": "Bob",
        "emial": "bob@example.com",
        "age": "none",
    },
]

Because of a typo, Bob has email field empty. Also, there is a non-number value for age key. The same problem would exist in JavaScript data definition. But, if you was able to use TypeScript...

export interface Person {
  name: string;
  email: string;
  age: number;
}
// import interface defined above
import { Person } from '../../models/index';

const people: Person[] = [
    {
        name: "John",
        email: "john@mail.de",
        age: 18,
    },
    {
        name: "Bob",
        emial: "bob@example.com", // <-- error underlined in IDE
        age: "none", //  <-- error underlined in IDE
    },
];

export = people;

If you used types, you would instantly see that you made mistakes - not only during import, but much earlier, in your IDE.

The Mongo Seeding CLI and Mongo Seeding Docker Image have TypeScript runtime built-in. It means that you can take advantage of static type checking in TypeScript data definition files (.ts extension). You can use Mongo Seeding library in your projects with TypeScript runtime and enable importing TS files as well.

Problem #3: No ultimate solution

Tools like this should be as flexible as possible. Some developers need just CLI tool, and some want to import data programmatically. Before writing Mongo Seeding, I needed a ready-to-use Docker image and found none. Dockerizing an application is easy, but it takes time.

That's why Mongo Seeding consists of:

All tools you'll ever need for seeding your MongoDB database.

Contribution

Before you contribute to this project, read CONTRIBUTING.md file.