Support multiple migration directories

[kibertoad]

Feature discussion / request

There are two clear use-cases when having multiple migration directories is beneficial:

1. Large long-running project with huge amount of migrations becomes more manageable if you can split migrations by year;
2. If project uses migrations for versioned data seeding/modification, it may be more convenient to have separate directory for structural and separate directory for data migrations.

In order to achieve this goal, it should be possible to pass an array of paths as 'directory' parameter in Migrations API.

Migration location within provided list should be interchangeable as far as check for missing migrations is concerned, so that migration could e. g. be moved from "latest" to "2017" at any given moment.

[kibertoad]

@elhigu This is something that my current project could use so I could work on a PR for this if you would approve such change :)

[elhigu]

Maybe related issue: #607 not exactly the same I think.

I suppose what you are proposing is doable and pretty easy; just go through all migration directories, collect filenames, sort them and run migrations as currently (and store only migration filename).

Writing tests for this is most difficult, where you should move migrations from one directory to another and add new migration directory on the fly etc.

Just to be sure. This change won't allow to lose any of old migration files?

[kibertoad]

Yup, tests gonna take a while :).
Which loss do you have in mind? You mean that knex should still require old migration files to be present in at least one of the specified directories? Then yeah, I don't think that we should make the check less strict than it currently is in this regard.

[elhigu]

This sounds good to me, I don't have any problems for accepting PR for this. So @kibertoad go ahead if you are up to implement this 👍

[kibertoad]

@elhigu Started working on it, one question I have - there was a rather strange decision to put Migrator class inside migrate/index.js which makes finding it less trivial than it should be. Would you mind if I extracted it into migrator.js?

[elhigu]

Yeah, that should be just fine. Is this feature now ready and it works backwards compatible way?

[kibertoad]

@elhigu I've only done the refactoring part for now which would enable me to do the actual change easily. If you could review #2695 so that it could be merged, actual feature would have a much more readable diff :).

[elhigu]

Its merged now :)

[elhigu]

Implemented in #2735

[ShreelekhaB]

Is there a documentation on how to use this feature ? I want to create multiple migration directories and how to use make command to specify which directory to use and similar related documentation ?

[elhigu]

knex/documentation#129

If you need just one directory that changes, then you might need to use environment variables in knexfile.js to pass directory name there. If multiple directories are listed in the knexfile, all of then are used.