[v3 beta10] Usage with js bundlers

[sowizz]

Hello,

I'm using Umzug to run migrations. Project's tree (removed unrelated stuff):

.
├── build
├── db
│   └── database.sqlite3
├── package-lock.json
├── package.json
└── src
    ├── index.js
    ├── migrations
    │   ├── 20200628183646-create-config.js
    │   ├── 20200628183704-create-model.js
    │   ├── 20200628183814-create-feature.js
    │   ├── 20200628183850-create-features-to-config.js
    │   ├── 20200628183909-create-models-to-config.js
    │   ├── 20200904133751-create-user.js
    │   ├── 20200925132248-create-setting.js
    │   └── 20201008143604-create-label.js
    └── models
        ├── config.js
        ├── feature.js
        ├── featureToConfig.js
        ├── index.js
        ├── label.js
        ├── model.js
        ├── modelToConfig.js
        ├── setting.js
        └── user.js

I'm building the sources with babel src -d build command, so that all the files are compiled and the same structure as above is created within build directory.

Now, I'm wondering why creating Umzug instance inside index.js with following options:

// Create Umzug instance.
const umzug = new Umzug({
  migrations: { glob: './migrations/*.js' },
  context: db.sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize: db.sequelize }),
  logger: console
});

makes it unable to find pending migrations. If I change the glob to './build/migrations/*.js' everything is fine.

It seems like prepending glob path with build/ solves the issue completely but the main reason I'm asking is:

I'm using pkg module to create executable and when using migrations: { glob: './build/migrations/*.js' }, I need to copy build/migrations directory to location that is next to executable file and then it does not work at all. Migrations are found and Umzug tries to run them but:

1. If I use compiled version of migration files, I receive following error: Error: Cannot find module '@babel/runtime/helpers/interopRequireDefault'
2. If I use "raw source" version of migration files it is not able to find Sequelize: Error: Cannot find module 'sequelize'.

It seems like they're run "out of the scope". In both scenarios from above copying node_modules to location next to executable fixes the issue, but since node_modules are already bundled within executable it's pointless.

I was able to achieve my goal by manually importing all migration files and using the array of migrations syntax, but it is less flexible and you have to remember that new migrations have to be imported and then added to array. I'm pretty sure it's something simple that I'm not aware of. Any ideas?

[mmkal]

I haven't used pkg before, and this project uses typescript rather than babel so I'm not totally sure how they might be interacting with each other.

One thing that might be worth trying from eyeballing your project tree. Instead of explicitly making glob ./build/migrations/*.js, you could tell the umzug instance to look for migrations relative to itself:

const umzug = new Umzug({
-  migrations: { glob: './migrations/*.js' },
+  migrations: { glob: ['./migrations/*.js', { cwd: __dirname }] },
  context: db.sequelize.getQueryInterface(),
  storage: new SequelizeStorage({ sequelize: db.sequelize }),
  logger: console
});

Then when you generated a bundle, put the migrations folder next to the bundle output so the relative paths match up.

Also worth looking at: seems like pkg lets you specify "assets" and "scripts" in the config so you don't have to manually copy them: https://github.com/vercel/pkg#config

Possibly adding the migrations to scripts instead of copying would fix your babel import issue - the module probably isn't found because the whole point of pkg is to get rid of the requirement on node_modules.

If you can create a small repro of this issue (git repo I can clone, install, build, run and see the error on) I can take a look, and maybe add an example to the examples folder, or see if there's anything that could be added to umzug to make this easier.

[sowizz]

@mmkal Thank you for the response, I'll try the solution you suggested and If it does not work as expected I'll create mini-repo, but no sooner than over the weekend since I have to do that in my own time.

I'm using pkg exactly for this reason - to "get rid" of the node_modules or any other directories for that matter, since our end-users are not computer-savvy so the setup has to be as minimal as possible.

pkg is obviously configured:

"pkg": {
    "scripts": [ "build/**/*.js" ],
    "assets": [ "node_modules/**/*","www/**/*", "logs/*.log", "logs/*.json", "db/*.sqlite3", "files/*.tar" ],
    "targets": [ "host"  ]
  }

but it is still required to copy the migrations folder manually.

[mmkal]

@sowizz There's now an example of one way that you can workaround the issue of bundlers wiping out the filesystem structure by using codegen to reduce the overhead of manually maintaining an array of migrations: https://github.com/sequelize/umzug/tree/master/examples/7.bundling-codegen. It uses parcel rather than pkg but the principle is the same.