Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database seed file support #391

Merged
merged 15 commits into from Aug 11, 2014
Merged

Database seed file support #391

merged 15 commits into from Aug 11, 2014

Conversation

@nsantorello
Copy link
Contributor

@nsantorello nsantorello commented Jul 22, 2014

Hello,

I've been using knex and bookshelf, and they've been great! I come from a rails background and found knex missing seed files, so I've developed the functionality. Included in this pull request are:

  • Seed functionality (described below) and corresponding tests and website documentation
  • Made migration config directory relative instead of absolute (might have been a bug?)

Seed functionality

Create seed files with knex seed:make <name>, similar to creating migrations. Example: knex seed:make test_data

Run seed files with knex seed:run. Looks at seeds.directory property in knexfile.js for the given environment and executes all seed files.

Seed files are intended to destroy existing data if necessary and then seed with correct data. Example seed file:

'use strict';

exports.seed = function(knex, Promise) {
    return Promise.join(
        // Remove any already existing payment plans
        knex('payment_plans').del(),

        // Seed initial payment plans
        knex('payment_plans').insert({name: 'Free', description: '30 day trial', cost: 0}),
        knex('payment_plans').insert({name: 'Basic', description: '10 users', cost: 2000}),
        knex('payment_plans').insert({name: 'Unlimited', description: 'unlimited users', cost: 5000})
    );
};

Example knexfile config:

  development: {
      client: ...,
      connection: { ... },
      seeds: {
          directory: 'seeds/dev'
      }
  },

Tests are included as well and cover things similar to what the migration tests cover.

Some notes:

  • It appears that gulp build modifies the files in the browser directory. I have excluded those files in this pull request because there is a tremendous amount of differences between the one that my machine generates and what is in git.
  • Made migration CLI output absolute paths instead of just filenames.
  • There is some copy/pasta between functionality in lib/migrate/index.js and lib/seed/index.js. It would be good in the future to refactor to share that functionality.

There is a repo you can clone that shows the seed file functionality:
https://github.com/nsantorello/knex-seed-test

Comments, thoughts, and questions are welcome!

@tgriesser
Copy link
Member

@tgriesser tgriesser commented Jul 22, 2014

This is awesome! Taking a look now!

@amcgee
Copy link

@amcgee amcgee commented Jul 22, 2014

+1!

@tgriesser
Copy link
Member

@tgriesser tgriesser commented Jul 28, 2014

This is really awesome. I'm going to get this in there this week and cut a new minor release with a few other changes that are in the works.

@nsantorello
Copy link
Contributor Author

@nsantorello nsantorello commented Jul 28, 2014

Great, let me know if there's anything I can do in the interim.

@briandela
Copy link
Contributor

@briandela briandela commented Aug 5, 2014

This is great - can't wait to start using it.

@briandela
Copy link
Contributor

@briandela briandela commented Aug 10, 2014

Tim, would love to start using this as soon as it's available so please do let me know if there is anything I can do to help get it out there.

tgriesser added a commit that referenced this pull request Aug 11, 2014
@tgriesser tgriesser merged commit 5b876d6 into knex:master Aug 11, 2014
1 check passed
1 check passed
@tgriesser
continuous-integration/travis-ci The Travis CI build passed
Details
@briandela
Copy link
Contributor

@briandela briandela commented Aug 18, 2014

@tgriesser Just wondering if this change will be pushed to a new release on npm soon? No worries if not - was just wondering.

Also, just to reiterate - knex/bookshelf are awesome and we really appreciate all the effort you put into this.

@tgriesser
Copy link
Member

@tgriesser tgriesser commented Aug 22, 2014

@briandela yep, very soon - sorry I've been backlogged with things lately, just looking to make one more change and then I'll cut a minor version release. Starting on it now.

Also thanks! Glad you're enjoying them!

@bmac
Copy link
Contributor

@bmac bmac commented Aug 31, 2014

Really excited for this to land in a release. Its going to solve some issues for me.

@tgriesser tgriesser mentioned this pull request Sep 2, 2014
@cyrusdavid
Copy link

@cyrusdavid cyrusdavid commented Sep 5, 2014

Excited as well.

@briandela
Copy link
Contributor

@briandela briandela commented Oct 1, 2014

Hey @tgriesser - just wondering if there is anything which I can do to help with getting this shipped?

@tgriesser
Copy link
Member

@tgriesser tgriesser commented Oct 1, 2014

I'm going to forget the event support I wanted to add for 0.7 and just ship what's in there now... I know a lot of people have been waiting on it, sorry about that. Working on the changelog and such now.

@tgriesser
Copy link
Member

@tgriesser tgriesser commented Oct 1, 2014

Shipped.

@briandela
Copy link
Contributor

@briandela briandela commented Oct 1, 2014

Amazing! Thanks!

@briandela
Copy link
Contributor

@briandela briandela commented Oct 2, 2014

@nsantorello This is awesome. Had a quick question about how you use this, if you have a second. Do you use it for seeding data in production? For example, when using a migration to add a new table in production, do you use a seed file to insert seed data for that? I've been using migrations for that, as only the migrations which need to run, run - it seems like the seed functionality will always run every seed file.

@nsantorello
Copy link
Contributor Author

@nsantorello nsantorello commented Oct 2, 2014

@briandela Glad you like it! I structure my seed files so that they're idempotent. Although it's rails related, this link explains the general idea: http://stackoverflow.com/a/18575279/94824

Let me know if that answers your question--I can post an example seed file if it helps.

@briandela
Copy link
Contributor

@briandela briandela commented Oct 2, 2014

@nsantorello Makes sense - thanks. Would still love to see some examples if you were able to share.

@nsantorello
Copy link
Contributor Author

@nsantorello nsantorello commented Oct 4, 2014

@briandela It can be as simple as this depending on your use case:

exports.seed = function(knex, Promise) {
    return Promise.join(
        knex('payment_plans').del(), // Delete ALL existing payment plans
        // Insert seed payment plans
        knex('payment_plans').insert({id: 1, name: 'Free', description: '30 day trial', cost: 0}),
        knex('payment_plans').insert({id: 2, name: 'Basic', description: '10 users', cost: 2000}),
        knex('payment_plans').insert({id: 3, name: 'Unlimited', description: 'unlimited users', cost: 5000})
    );
};

If you have seed data in a table mixed with non-seed data, you can emulate upsert functionality:

exports.seed = function(knex, Promise) {
    function upsert(table, element)
    {
      var selector = knex(table).where({id: element.id}); // Query specific element
      return selector.then(function(rows) {
        if (rows.length == 0) {
          return knex(table).insert(element); // Insert if doesn't exist
        } else {
          return selector.update(element); // Update if does exist
        }
      });
    }

    var plans = [{id: 1, name: 'Free', description: '30 day trial', cost: 0},
      {id: 2, name: 'Basic', description: '10 users', cost: 2000},
      {id: 3, name: 'Unlimited', description: 'unlimited users', cost: 5000}];

    var transactions = [];

    plans.forEach(function(plan) {
      transactions.push(upsert('payment_plans', plan));
    });

    return Promise.all(transactions);
};

Let me know if that answers your question.

@fcsonline

This comment has been minimized.

Copy link

@fcsonline fcsonline commented on lib/seed/stub/js.stub in f8d01e9 Oct 8, 2014

A missing return here?

This comment has been minimized.

Copy link
Contributor Author

@nsantorello nsantorello replied Oct 8, 2014

Not only is it missing a return, it should actually return a Promise. See the seed file examples here:
#391 (comment)

This comment has been minimized.

Copy link

@fcsonline fcsonline replied Oct 9, 2014

Exactly. Thanks. 👍

nsantorello added a commit to nsantorello/knex that referenced this pull request Oct 13, 2014
@nsantorello nsantorello mentioned this pull request Oct 13, 2014
bendrucker added a commit that referenced this pull request Oct 13, 2014
* Updated seed file documentation
* Sample seed files now correctly return Promises #391
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants