javascript knexfile can't run typescript migrations #3849

prk3 · 2020-05-10T10:11:02Z

Environment

Knex version: 0.21.1
Database + version: postgres 12
OS: Linux

Bug

Description

When the knexfile has js extension, typescript migrations and seeds can't be run. The error thrown suggests these files are not loaded with typescript loader. Changing knexfile extension to ts fixes the problem.

This is rather unexpected, as the documentation suggests that knex can run migrations of different types, no matter what the knexfile extension is.

loadExtensions: array of file extensions which knex will treat as migrations.

Error message

import Knex from 'knex';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1047:16)
    at Module._compile (internal/modules/cjs/loader.js:1097:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
    at Module.load (internal/modules/cjs/loader.js:977:32)
    at Function.Module._load (internal/modules/cjs/loader.js:877:14)
    at Module.require (internal/modules/cjs/loader.js:1019:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at FsMigrations.getMigration (/home/przemek/DEV/sundae/sundae-collab-demo-api/node_modules/knex/lib/migrate/sources/fs-migrations.js:82:12)
    at /home/przemek/DEV/sundae/sundae-collab-demo-api/node_modules/knex/lib/migrate/Migrator.js:76:67
    at Array.some (<anonymous>)

Knexfile content

require('dotenv').config();

const defaultSettings = {
  // client
  // connection
};

module.exports.development = {
  ...defaultSettings,
  migrations: {
    loadExtensions: ['.ts'],
    extension: 'ts',
  },
  seeds: {
    loadExtensions: ['.ts'],
  },
};

module.exports.test = {
  ...defaultSettings,
  migrations: {
    loadExtensions: ['.ts'],
    extension: 'ts',
  },
  seeds: {
    loadExtensions: ['.ts'],
  },
};

module.exports.production = {
  ...defaultSettings,
  migrations: {
    directory: 'build/migrations',
  },
  seeds: {
    directory: 'build/seeds',
  },
};

Feature discussion / request

Explain what is your use case

I want to run Typescript migrations and seeds in development environment. In production, where Typescript and ts-node might not be present, compiled migrations and seeds should be used instead. Ideally, one js knexfile would handle both cases.

Explain what kind of feature would support this

Knexfile extension should not influence migration and seed loading. When loadExtensions includes .ts, typescript migrations and seeds should be loaded and run.

The text was updated successfully, but these errors were encountered:

love8587 · 2020-05-20T11:07:50Z

@prk3 You could try with npx knex migrate:latest

prk3 · 2020-05-21T15:28:34Z

@love8587
That's how I run migrations. Here is the exact console output. Notice that ts-node/register is used only when knexfile has .ts extension.

$ npx knex migrate:latest
/home/przemek/DEV/sundae/sundae-collab-demo-api/migrations/20200414144724_recipes.ts:1
import Knex from 'knex';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1047:16)
    at Module._compile (internal/modules/cjs/loader.js:1097:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
    at Module.load (internal/modules/cjs/loader.js:977:32)
    at Function.Module._load (internal/modules/cjs/loader.js:877:14)
    at Module.require (internal/modules/cjs/loader.js:1019:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at FsMigrations.getMigration (/home/przemek/DEV/sundae/sundae-collab-demo-api/node_modules/knex/lib/migrate/sources/fs-migrations.js:82:12)
    at /home/przemek/DEV/sundae/sundae-collab-demo-api/node_modules/knex/lib/migrate/Migrator.js:76:67
    at Array.some (<anonymous>)
$ mv knexfile.js knexfile.ts
$ npx knex migrate:latest
Requiring external module ts-node/register
Batch 1 run: 2 migrations
$

Heverson · 2020-06-09T20:20:52Z

Hello @prk3
I have the same problem, when run knex migrate in heroku cli

Running knex --knexfile knexfile.ts migrate:latest on ⬢ backend-ecolab... up, run.5056 (Free)
Failed to load external module ts-node/register
Failed to load external module typescript-node/register
Failed to load external module typescript-register
Failed to load external module typescript-require
Failed to load external module sucrase/register/ts
Failed to load external module @babel/register
Error: Cannot find module 'ts-node/register'
Require stack:
- /app/knexfile.ts
- /app/node_modules/knex/bin/cli.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:966:15)
    at Function.Module._load (internal/modules/cjs/loader.js:842:27)
    at Module.require (internal/modules/cjs/loader.js:1026:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/app/knexfile.ts:1:1)
    at Module._compile (internal/modules/cjs/loader.js:1138:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)
    at Module.load (internal/modules/cjs/loader.js:986:32)
    at Function.Module._load (internal/modules/cjs/loader.js:879:14)
    at Module.require (internal/modules/cjs/loader.js:1026:19)

lorefnon · 2020-06-10T18:54:57Z

@prk3 If you specify loadExtensions: ['.js'] in the config for production env it should work as expected.

The default value of loadExtensions comes from here which will have to be overridden if you have some files matching those extensions which you don't want to process.

love8587 · 2020-06-12T01:29:49Z

@lorefnon that's right. I found out it from document and resolved it.

abcd-ca · 2020-06-12T01:37:21Z

I can't get it to run a knexfile.ts that has typescript syntax such as,

import dotenv from 'dotenv'

dotenv.config();

without the error,

import dotenv from 'dotenv';
^^^^^^

SyntaxError: Cannot use import statement outside a module

I have tried running it like,

knex --knexfile ./knexfile.ts migrate:make create_users_table -x ts (tried with and without the --knexfile option and with and without the -x ts option)

If I remove the import from knexfile.ts it just hides the problem because the rest of the file is the same as javascript – it creates the migration file with a .ts extension but any imports or requires of typescript code give the same error so I think we need to focus on why it's not interpreting knexfile.ts as typescript. Further, to get rid of the "Failed to load external module ts-node/register" message I did yarn add --dev ts-node which results in the line, "Requiring external module ts-node/register" being displayed before it throws the SyntaxError about import.

I don't know if TS is broken for the knex CLI or if I'm just missing something

abcd-ca · 2020-06-12T01:41:57Z

~~I just noticed that 0.21.1 in the changelog has a bug fix that looks related which I suspect might be causing this side effect issue.~~

This was incorrect, I tried with 0.21.0 and it didn't fix it. I got the knexfile.ts to work with the import by changing "module": "esnext" to "module": "commonjs" in tsconfig.json. The problem is in my case is that I am using Next.js 9 (9.4.0) and when I run the app, Next.js changes the module setting back to esnext which is apparently a known issue. So, I think this is not a knex problem but if you don't know that module must be commonjs then you're in trouble – I'd recommend adding it to the docs.

abcd-ca · 2020-06-12T01:59:30Z

Is there a way to have a separate tsconfig.json file just for the knex CLI as a workaround?

love8587 · 2020-06-12T08:02:15Z

@abcd-ca I resolved it below code.

const extension =
  process.env.NODE_CONFIG_ENV === 'production' || process.env.NODE_CONFIG_ENV === 'staging'
    ? 'js'
    : 'ts';

console.log('extension ============> ', extension);

  migrations: {
    tableName: 'knex_migrations',
    extension: extension,
    directory: path.join(BASE_PATH, 'migrations'),
    disableMigrationsListValidation: true,
    loadExtensions: [`.${extension}`],
  },
  seeds: {
    extension: extension,
    directory: path.join(BASE_PATH, 'seeds'),
  },

abcd-ca · 2020-06-12T17:33:16Z

@love8587 Yeah that's a good solution to your problem, thanks for sharing! In my case I realized it was a bit of a different problem than yours. In my case I couldn't get knex CLI to work with typescript files at all. In my case it was because my tsconfig.json didn't have module set to commonjs as it requires. Conversely, my UI framework Next.js requires module to be esnext. For now I'm switching back and forth when doing knex CLI activity vs running the server.

lorefnon · 2020-06-12T17:59:53Z

@abcd-ca You can use:

TS_NODE_COMPILER_OPTIONS='{ "module": "commonjs" }' knex migrate:latest

knex uses ts-node's require hook for .ts extension and that can accept TS compiler options through TS_NODE_COMPILER_OPTIONS env variable. These options will be merged into what you have in your tsconfig.json.

abcd-ca · 2020-06-12T19:04:44Z

thanks!

abcd-ca · 2020-06-12T20:57:50Z

@lorefnon That gave me this alternative idea:

created a tsconfig.knexcli.json containing an override for module,

{
  "extends": "./tsconfig",
  "compilerOptions": {
    "module": "commonjs"
  }
}

Then when running my commands,

TS_NODE_PROJECT=tsconfig.knexcli.json knex migrate:latest

Just slightly cleaner because the JSON when in an npm script needs to escape the quotes and gets a bit busy looking. Thanks!

lorefnon · 2020-06-13T04:45:57Z

@abcd-ca Yes, that certainly looks like a cleaner approach.

mrsufgi · 2020-06-29T06:27:53Z

Thanks for your solution @abcd-ca.
Seems like this thread is a mix of two issues.

knexfile.js doesn't transpile .ts migration files by default.
knex cli doesn't support 'esnext' code, even with node14.4

the workaround for 1 was changing to knexfile.ts, it should be enough for most people (as I've seen it, "commonjs" is the most common module resolution). but the workaround for 2 only indicates that we have issues with esm :)

for ts/node maybe we need to run it with

node --loader ts-node/esm.mjs

somewhere down the pipeline?

Adverbly · 2020-07-31T01:38:02Z

For those seeing

Failed to load external module ts-node/register
Failed to load external module typescript-node/register
Failed to load external module typescript-register
Failed to load external module typescript-require
Failed to load external module sucrase/register/ts
Failed to load external module @babel/register
Error: Cannot find module 'ts-node/register'
...
SyntaxError: Cannot use import statement outside a module

for me the fix was that I was missing ts-node from my package.json. A simple

npm install -D ts-node

was all I needed.

Isaac-alencar · 2020-08-24T13:21:53Z

I can't get it to run a knexfile.ts that has typescript syntax such as,
import dotenv from 'dotenv'

dotenv.config();
without the error,
import dotenv from 'dotenv';
^^^^^^

SyntaxError: Cannot use import statement outside a module
I have tried running it like,

knex --knexfile ./knexfile.ts migrate:make create_users_table -x ts (tried with and without the --knexfile option and with and without the -x ts option)

If I remove the import from knexfile.ts it just hides the problem because the rest of the file is the same as javascript – it creates the migration file with a .ts extension but any imports or requires of typescript code give the same error so I think we need to focus on why it's not interpreting knexfile.ts as typescript. Further, to get rid of the "Failed to load external module ts-node/register" message I did yarn add --dev ts-node which results in the line, "Requiring external module ts-node/register" being displayed before it throws the SyntaxError about import.

I don't know if TS is broken for the knex CLI or if I'm just missing something

I have the same issue, and your suggest to add ts-node works to me

cbaron · 2020-12-07T04:38:38Z

@abcd-ca's method of

    "module": "commonjs",

in tsconfig.json worked for me as well

DininduKanchana · 2021-05-26T12:42:01Z

loadExtensions: [.${extension}],

@abcd-ca I resolved it below code.

const extension =
  process.env.NODE_CONFIG_ENV === 'production' || process.env.NODE_CONFIG_ENV === 'staging'
    ? 'js'
    : 'ts';

console.log('extension ============> ', extension);

  migrations: {
    tableName: 'knex_migrations',
    extension: extension,
    directory: path.join(BASE_PATH, 'migrations'),
    disableMigrationsListValidation: true,
    loadExtensions: [`.${extension}`],
  },
  seeds: {
    extension: extension,
    directory: path.join(BASE_PATH, 'seeds'),
  },

I did all the things but this solved my issue. Thank you very much. Spent 3+ hours on this.

jeanbispo · 2022-08-04T20:30:04Z

I resolve with:

yarn add ts-node -D

and add in package.json scripts:

{
  "scripts": {
     ...,
     "knex": "TS_NODE_COMPILER_OPTIONS='{ \"module\": \"commonjs\" }' npx knex"
  }
}

now use yarn knex <your command>

hahayusuf · 2023-02-15T00:21:00Z

Still having issues with this.

This is the error I am getting:

SyntaxError: Cannot use import statement outside a module

opatajoshua · 2023-10-17T22:56:28Z

I got it to work by generating a TS knexfile instead knex init -x ts and moving my js config into it

jowparks mentioned this issue Aug 16, 2020

Broken CLI TypeScript support #3807

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

javascript knexfile can't run typescript migrations #3849

javascript knexfile can't run typescript migrations #3849

prk3 commented May 10, 2020

love8587 commented May 20, 2020 •

edited

prk3 commented May 21, 2020

Heverson commented Jun 9, 2020

lorefnon commented Jun 10, 2020 •

edited

love8587 commented Jun 12, 2020

abcd-ca commented Jun 12, 2020

abcd-ca commented Jun 12, 2020 •

edited

abcd-ca commented Jun 12, 2020

love8587 commented Jun 12, 2020 •

edited

abcd-ca commented Jun 12, 2020

lorefnon commented Jun 12, 2020 •

edited

abcd-ca commented Jun 12, 2020

abcd-ca commented Jun 12, 2020 •

edited

lorefnon commented Jun 13, 2020

mrsufgi commented Jun 29, 2020

Adverbly commented Jul 31, 2020 •

edited

Isaac-alencar commented Aug 24, 2020

cbaron commented Dec 7, 2020

DininduKanchana commented May 26, 2021

jeanbispo commented Aug 4, 2022

hahayusuf commented Feb 15, 2023 •

edited

opatajoshua commented Oct 17, 2023

javascript knexfile can't run typescript migrations #3849

javascript knexfile can't run typescript migrations #3849

Comments

prk3 commented May 10, 2020

Environment

Bug

Feature discussion / request

love8587 commented May 20, 2020 • edited

prk3 commented May 21, 2020

Heverson commented Jun 9, 2020

lorefnon commented Jun 10, 2020 • edited

love8587 commented Jun 12, 2020

abcd-ca commented Jun 12, 2020

abcd-ca commented Jun 12, 2020 • edited

abcd-ca commented Jun 12, 2020

love8587 commented Jun 12, 2020 • edited

abcd-ca commented Jun 12, 2020

lorefnon commented Jun 12, 2020 • edited

abcd-ca commented Jun 12, 2020

abcd-ca commented Jun 12, 2020 • edited

lorefnon commented Jun 13, 2020

mrsufgi commented Jun 29, 2020

Adverbly commented Jul 31, 2020 • edited

Isaac-alencar commented Aug 24, 2020

cbaron commented Dec 7, 2020

DininduKanchana commented May 26, 2021

jeanbispo commented Aug 4, 2022

hahayusuf commented Feb 15, 2023 • edited

opatajoshua commented Oct 17, 2023

love8587 commented May 20, 2020 •

edited

lorefnon commented Jun 10, 2020 •

edited

abcd-ca commented Jun 12, 2020 •

edited

love8587 commented Jun 12, 2020 •

edited

lorefnon commented Jun 12, 2020 •

edited

abcd-ca commented Jun 12, 2020 •

edited

Adverbly commented Jul 31, 2020 •

edited

hahayusuf commented Feb 15, 2023 •

edited