New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database seed file support #391

Merged
merged 15 commits into from Aug 11, 2014

Conversation

Projects
None yet
7 participants
@nsantorello
Contributor

nsantorello commented Jul 22, 2014

Hello,

I've been using knex and bookshelf, and they've been great! I come from a rails background and found knex missing seed files, so I've developed the functionality. Included in this pull request are:

  • Seed functionality (described below) and corresponding tests and website documentation
  • Made migration config directory relative instead of absolute (might have been a bug?)

Seed functionality

Create seed files with knex seed:make <name>, similar to creating migrations. Example: knex seed:make test_data

Run seed files with knex seed:run. Looks at seeds.directory property in knexfile.js for the given environment and executes all seed files.

Seed files are intended to destroy existing data if necessary and then seed with correct data. Example seed file:

'use strict';

exports.seed = function(knex, Promise) {
    return Promise.join(
        // Remove any already existing payment plans
        knex('payment_plans').del(),

        // Seed initial payment plans
        knex('payment_plans').insert({name: 'Free', description: '30 day trial', cost: 0}),
        knex('payment_plans').insert({name: 'Basic', description: '10 users', cost: 2000}),
        knex('payment_plans').insert({name: 'Unlimited', description: 'unlimited users', cost: 5000})
    );
};

Example knexfile config:

  development: {
      client: ...,
      connection: { ... },
      seeds: {
          directory: 'seeds/dev'
      }
  },

Tests are included as well and cover things similar to what the migration tests cover.

Some notes:

  • It appears that gulp build modifies the files in the browser directory. I have excluded those files in this pull request because there is a tremendous amount of differences between the one that my machine generates and what is in git.
  • Made migration CLI output absolute paths instead of just filenames.
  • There is some copy/pasta between functionality in lib/migrate/index.js and lib/seed/index.js. It would be good in the future to refactor to share that functionality.

There is a repo you can clone that shows the seed file functionality:
https://github.com/nsantorello/knex-seed-test

Comments, thoughts, and questions are welcome!

@tgriesser

This comment has been minimized.

Owner

tgriesser commented Jul 22, 2014

This is awesome! Taking a look now!

@amcgee

This comment has been minimized.

amcgee commented Jul 22, 2014

+1!

@tgriesser

This comment has been minimized.

Owner

tgriesser commented Jul 28, 2014

This is really awesome. I'm going to get this in there this week and cut a new minor release with a few other changes that are in the works.

@nsantorello

This comment has been minimized.

Contributor

nsantorello commented Jul 28, 2014

Great, let me know if there's anything I can do in the interim.

@briandela

This comment has been minimized.

Contributor

briandela commented Aug 5, 2014

This is great - can't wait to start using it.

@briandela

This comment has been minimized.

Contributor

briandela commented Aug 10, 2014

Tim, would love to start using this as soon as it's available so please do let me know if there is anything I can do to help get it out there.

tgriesser added a commit that referenced this pull request Aug 11, 2014

@tgriesser tgriesser merged commit 5b876d6 into tgriesser:master Aug 11, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details
@briandela

This comment has been minimized.

Contributor

briandela commented Aug 18, 2014

@tgriesser Just wondering if this change will be pushed to a new release on npm soon? No worries if not - was just wondering.

Also, just to reiterate - knex/bookshelf are awesome and we really appreciate all the effort you put into this.

@tgriesser

This comment has been minimized.

Owner

tgriesser commented Aug 22, 2014

@briandela yep, very soon - sorry I've been backlogged with things lately, just looking to make one more change and then I'll cut a minor version release. Starting on it now.

Also thanks! Glad you're enjoying them!

@bmac

This comment has been minimized.

Contributor

bmac commented Aug 31, 2014

Really excited for this to land in a release. Its going to solve some issues for me.

@tgriesser tgriesser referenced this pull request Sep 2, 2014

Closed

Changelog for 0.7 #460

@vohof

This comment has been minimized.

vohof commented Sep 5, 2014

Excited as well.

@briandela

This comment has been minimized.

Contributor

briandela commented Oct 1, 2014

Hey @tgriesser - just wondering if there is anything which I can do to help with getting this shipped?

@tgriesser

This comment has been minimized.

Owner

tgriesser commented Oct 1, 2014

I'm going to forget the event support I wanted to add for 0.7 and just ship what's in there now... I know a lot of people have been waiting on it, sorry about that. Working on the changelog and such now.

@tgriesser

This comment has been minimized.

Owner

tgriesser commented Oct 1, 2014

Shipped.

@briandela

This comment has been minimized.

Contributor

briandela commented Oct 1, 2014

Amazing! Thanks!

@briandela

This comment has been minimized.

Contributor

briandela commented Oct 2, 2014

@nsantorello This is awesome. Had a quick question about how you use this, if you have a second. Do you use it for seeding data in production? For example, when using a migration to add a new table in production, do you use a seed file to insert seed data for that? I've been using migrations for that, as only the migrations which need to run, run - it seems like the seed functionality will always run every seed file.

@nsantorello

This comment has been minimized.

Contributor

nsantorello commented Oct 2, 2014

@briandela Glad you like it! I structure my seed files so that they're idempotent. Although it's rails related, this link explains the general idea: http://stackoverflow.com/a/18575279/94824

Let me know if that answers your question--I can post an example seed file if it helps.

@briandela

This comment has been minimized.

Contributor

briandela commented Oct 2, 2014

@nsantorello Makes sense - thanks. Would still love to see some examples if you were able to share.

@nsantorello

This comment has been minimized.

Contributor

nsantorello commented Oct 4, 2014

@briandela It can be as simple as this depending on your use case:

exports.seed = function(knex, Promise) {
    return Promise.join(
        knex('payment_plans').del(), // Delete ALL existing payment plans
        // Insert seed payment plans
        knex('payment_plans').insert({id: 1, name: 'Free', description: '30 day trial', cost: 0}),
        knex('payment_plans').insert({id: 2, name: 'Basic', description: '10 users', cost: 2000}),
        knex('payment_plans').insert({id: 3, name: 'Unlimited', description: 'unlimited users', cost: 5000})
    );
};

If you have seed data in a table mixed with non-seed data, you can emulate upsert functionality:

exports.seed = function(knex, Promise) {
    function upsert(table, element)
    {
      var selector = knex(table).where({id: element.id}); // Query specific element
      return selector.then(function(rows) {
        if (rows.length == 0) {
          return knex(table).insert(element); // Insert if doesn't exist
        } else {
          return selector.update(element); // Update if does exist
        }
      });
    }

    var plans = [{id: 1, name: 'Free', description: '30 day trial', cost: 0},
      {id: 2, name: 'Basic', description: '10 users', cost: 2000},
      {id: 3, name: 'Unlimited', description: 'unlimited users', cost: 5000}];

    var transactions = [];

    plans.forEach(function(plan) {
      transactions.push(upsert('payment_plans', plan));
    });

    return Promise.all(transactions);
};

Let me know if that answers your question.

@fcsonline

This comment has been minimized.

fcsonline commented on lib/seed/stub/js.stub in f8d01e9 Oct 8, 2014

A missing return here?

This comment has been minimized.

Contributor

nsantorello replied Oct 8, 2014

Not only is it missing a return, it should actually return a Promise. See the seed file examples here:
#391 (comment)

This comment has been minimized.

fcsonline replied Oct 9, 2014

Exactly. Thanks. 👍

nsantorello added a commit to nsantorello/knex that referenced this pull request Oct 13, 2014

@nsantorello nsantorello referenced this pull request Oct 13, 2014

Closed

Fix seed stub files #528

bendrucker added a commit that referenced this pull request Oct 13, 2014

Merge pull request #528 from nsantorello/master
* Updated seed file documentation
* Sample seed files now correctly return Promises #391
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment