Generating migrations with cli

[SleeveShirtholes]

Issue Description

Expected Behavior

Migrations get generated using the cli

Actual Behavior

I am following the documentation outlined here https://typeorm.io/migrations and I feel like the documentation is way different then what is actually happening in the cli. I have tried just running the following command:

typeorm migration:generate -n PostRefactoring

When I run this command I get the following:

Not enough non-option arguments: got 0, need at least 1

So then I modify the command to something like this:

Missing required argument: dataSource

So then from there I add the datasource since that is what it is asking for:

npx typeorm migration:generate test -d app-data-source.ts (I found the -d somewhere online while investigating this issue)

But once I run that command I get the following error:

File must contain a TypeScript / JavaScript code and export a DataSource instance.

I have been trying to get this to work for days and I am about ready to throw the towl in and use a different ORM...which I don't want to do. I know that everything is working fine because I can start up the app and the app connects to the database and creates rows in the database. So it is just something with the cli and trying to generate the migrations.

Here is what is in my app-data-source.ts file:

import { DataSource } from "typeorm";
import { User } from "./entitiy/User";

const myDataSource = new DataSource({
  type: "postgres",
  host: "localhost",
  port: 5432,
  username: "postgres",
  password: "",
  database: "postgres",
  entities: [User],
  migrations: ["/migrations/"],
  logging: true,
  synchronize: true,
});


export default myDataSource;

This is the contents of my app.ts file:

import * as express from "express";
import { Request, Response } from "express";
import myDataSource from "./app-data-source";
import { User } from "./entitiy/User";

// establish database connection
myDataSource
  .initialize()
  .then(() => {
    console.log("Data Source has been initialized!");
  })
  .catch((err) => {
    console.error("Error during Data Source initialization:", err);
  });

// create and setup express app
const app = express();
app.use(express.json());

// register routes
app.get("/users", async function (req: Request, res: Response) {
  const users = await myDataSource.getRepository(User).find();
  res.json(users);
});

app.get("/users/:id", async function (req: Request, res: Response) {
  const results = await myDataSource.getRepository(User).findOneBy({
    id: +req.params.id,
  });
  return res.send(results);
});

app.post("/users", async function (req: Request, res: Response) {
  const user = await myDataSource.getRepository(User).create(req.body);
  const results = await myDataSource.getRepository(User).save(user);
  return res.send(results);
});

app.put("/users/:id", async function (req: Request, res: Response) {
  const user = await myDataSource.getRepository(User).findOneBy({
    id: +req.params.id,
  });
  myDataSource.getRepository(User).merge(user, req.body);
  const results = await myDataSource.getRepository(User).save(user);
  return res.send(results);
});

app.delete("/users/:id", async function (req: Request, res: Response) {
  const results = await myDataSource.getRepository(User).delete(req.params.id);
  return res.send(results);
});

// start express server
app.listen(3000);

All these files were setup using the instructions outlined in the express typeorm documentation:
Example with Express

Which BTW, this also has issues and doesn't run from what is outlined in the directions. I had to make some modifications to actually make the app run compared to what is outlined in that guide. One of the changes I made in the data source file was putting the actual Entity name in the entity property of the datasource compared to the guide that has you put the path in.

Anyway, am I crazy...am I doing something wrong? You guys might not have enough information to answer the first question but can you at least help with trying to figure out how to make the migrations work?

Steps to Reproduce

Set up you app using the guide outlined above to use express

My Environment

Dependency	Version
Operating System	MAC OS Monterrey
Node.js version	v16.14.2
Typescript version	4.5.2
TypeORM version	0.3.4

Relevant Database Driver(s)

DB Type	Reproducible
aurora-mysql	no
aurora-postgres	no
better-sqlite3	no
cockroachdb	no
cordova	no
expo	no
mongodb	no
mysql	no
nativescript	no
oracle	no
postgres	yes
react-native	no
sap	no
sqlite	no
sqlite-abstract	no
sqljs	no
sqlserver	no

Are you willing to resolve this issue by submitting a Pull Request?

• ✖️ Yes, I have the time, and I know how to start.
• ✖️ Yes, I have the time, but I don't know how to start. I would need guidance.
• ✖️ No, I don’t have the time, but I can support (using donations) development.
• ✅ No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.

[Sreejith288]

CLI works fine till version - 0.2.45.
After the version bump, above mentioned message is displayed.
I had upgraded my entire project to 0.3 seeing the deprecated changes for the connection configurations (deprecated createconnection), but now migration is blocked.
Does anyone know what the new command should be like for version 0.3+ or is this a bug?

Looping in (@pleerock )

[anoutsider]

Had the same issue, the workaround I found for now is to use the compiled js file

For example

npx typeorm migration:generate test -d dist/app-data-source.js

As the cli command fails to import the .ts file for some reason, probably because of the file url it tries to generate, also for some reason it will only import a ts file if the closest package.json file is set to a type => module

File URL ->

typeorm/src/util/ImportUtils.ts

Line 15 in eacecce

: pathToFileURL(filePath).toString(),

package.json module type check ->

typeorm/src/util/ImportUtils.ts

Line 32 in eacecce

const isModule = (packageJson as any)?.type === "module"

[matthewdowns]

How did 0.3.0 get past the testing phase when the CLI is completely broken?

[AashJ]

+1, pretty unusable atm, will switch off for my newer projects

[evskorobogatij]

I have the same issue

[evskorobogatij]

Had the same issue, the workaround I found for now is to use the compiled js file

For example

npx typeorm migration:generate test -d dist/app-data-source.js

As the cli command fails to import the .ts file for some reason, probably because of the file url it tries to generate, also for some reason it will only import a ts file if the closest package.json file is set to a type => module

File URL ->

typeorm/src/util/ImportUtils.ts

Line 15 in eacecce

: pathToFileURL(filePath).toString(),

package.json module type check ->

typeorm/src/util/ImportUtils.ts

Line 32 in eacecce

const isModule = (packageJson as any)?.type === "module"

This is not work

npx typeorm migration:generate -d .\dist\db\ormconfig.js Init

And I have the same issue

[msalsbery]

Thanks to @pleerock and this post I now have migration creation/generation working.

The CLI now requires the full path to the output file, so if you have an npm script like

    "typeorm": "typeorm-ts-node-esm -d ./config/data-source.ts"

then an example call to that script to generate a migration could look something like

npm run typeorm -- migration:generate ./src/database/migrations/my-migration-name

@pleerock thank you for that!

[SleeveShirtholes]

I also got the migrations to work but only by compiling my code and using .js data-source file. I still can't get the migrations to work using the .ts files. I also can't get the entities path in the data-source file to work and have to declare each entity.

[kylevillegas93]

@SleeveShirtholes - may want to try: ['dist/src/entities/**/*{.js,.ts}']

[msalsbery]

I also got the migrations to work but only by compiling my code and using .js data-source file. I still can't get the migrations to work using the .ts files. I also can't get the entities path in the data-source file to work and have to declare each entity.

@SleeveShirtholes In my example I'm using all TS. In addition to @kylevillegas93 's reply, my data-source.ts looks like

import { DataSource } from 'typeorm';
import { SqlServerConnectionOptions } from 'typeorm/driver/sqlserver/SqlServerConnectionOptions';
// eslint-disable-next-line node/no-missing-import
import { CustomNamingStrategy } from '../src/database/models/naming-strategy';

const options: SqlServerConnectionOptions = {
  type: 'mssql',
  url: '',
  host: 'localhost',
  port: 1433,
  dropSchema: false,
  synchronize: false,
  extra: {
    trustServerCertificate: true
  },
  options: {
    useUTC: true,
    abortTransactionOnError: true, // SET XACT_ABORT ON|OFF
    enableArithAbort: true
  },
  username: '...',
  password: '...',
  database: '...',
  logging: false, // TypeORM true, false, ['query', 'error', 'schema', 'warn', 'info', 'log']
  logger: 'advanced-console', // TypeORM 'advanced-console' (default), 'simple-console', 'file'
  entities: ['src/database/models/**/*.ts'],
  migrations: ['src/database/migrations/**/*.ts'],
  subscribers: ['src/database/subscribers/**/*.ts'],
  namingStrategy: new CustomNamingStrategy()
};

export default new DataSource(options);

Hope that helps in some way...

[SleeveShirtholes]

@msalsbery @kylevillegas93

So I just tried both suggestions and still can't get migrations to work with the .ts file. Here is my .ts file:

import { DataSource } from "typeorm";
import { PostgresConnectionOptions } from "typeorm/driver/postgres/PostgresConnectionOptions";

const options: PostgresConnectionOptions = {
  type: "postgres",
  host: process.env.PGHOST,
  port: 5432,
  username: process.env.PGUSER,
  password: process.env.PGPASSWORD,
  database: process.env.PGDATABASE,
  entities: ["src/**/entities/*{.js,.ts}"],
  // entities: [
  //   BudgetAppUser,
  //   Account,
  //   UserAccount,
  //   Biller,
  //   AccountBiller,
  //   AccountBill,
  //   Frequency,
  //   BillerType,
  // ],
  migrations: ["./build/**/migrations/**.js"],
  synchronize: false,
  logging: false,
};

export default new DataSource(options);

Here is my package.json file:

{
  "name": "back_end",
  "version": "1.0.0",
  "description": "",
  "main": "index.ts",
  "scripts": {
    "start": "nodemon src/index.ts",
    "typeorm": "typeorm-ts-node-esm -d ./src/data-source.ts"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "body-parser": "^1.19.2",
    "express": "^4.17.3",
    "pg": "^8.7.3",
    "reflect-metadata": "^0.1.13",
    "typeorm": "^0.3.3",
    "typescript": "^4.6.2"
  },
  "devDependencies": {
    "@types/express": "^4.17.13",
    "@types/node": "^17.0.21",
    "nodemon": "^2.0.15",
    "ts-node": "^10.7.0"
  }
}

The error I am getting now is:
sh: typeorm-ts-node-esm: not found

I was getting this error before:
File must contain a TypeScript / JavaScript code and export a DataSource instance.

But I think the reason I am getting that error is because I didn't originally have the typeorm script in my package.json.

Also, I am working within a docker container so don't know if that has anything to do with it.

[msalsbery]

@SleeveShirtholes here's my relevant package.json stuff... typeorm version is different than yours I noticed...

{
  "dependencies": {
    "mssql": "^8.0.2",
    "reflect-metadata": "^0.1.13",
    "typeorm": "^0.3.4"
  },
  "devDependencies": {
    "@types/node": "^17.0.23",
    "@types/tedious": "^4.0.7",
    "ts-node": "10.7.0",
    "typescript": "^4.6.2"
  },
  "scripts": {
    "typeorm": "typeorm-ts-node-esm -d ./config/data-source.ts"
  }
}

Note: use typeorm-ts-node-commonjs in the typeorm script if not using esm

[SleeveShirtholes]

@msalsbery I am getting this error now when running the following command:

Error:
env: can't execute 'node --require ts-node/register': No such file or directory

Command:
npm run typeorm -- migration:generate ./src/budget_app/migrations/initialMigration

Do you know why that is? Sorry to keep bugging you.

[msalsbery]

@SleeveShirtholes are you sure all dependencies are updated to latest versions and installed? That looks like it's not seeing ts-node

[SleeveShirtholes]

@msalsbery Yea thats the weird thing...I have ts-node installed. I also confirmed that ts-node is in my node_modules folder and it is there. Not sure why I am getting this error.

[msalsbery]

@SleeveShirtholes hmm, I got nothin ;) If you added the typeorm script to package.json and that script runs typeorm-ts-node-esm then the only difference from what I'm doing is I'm not running in a container

[SleeveShirtholes]

@msalsbery Ok...it does look like its the container that is the issue. I ran the command outside the container and everything worked fine. Thanks for your help!

[ricardo85x]

I could not make the run command work

Here is my files:

./src/database/data_source.ts

import { DataSource } from "typeorm";

const AppDataSource = new DataSource({
  type: process.env.DB_TYPE,
  host: process.env.DB_HOST,
  username: process.env.DB_USERNAME,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_DATABASE,
  port: process.env.DB_PORT,
  entities: [],
  migrations: ["./src/database/migrations/*.ts"],
});

export default AppDataSource;

AppDataSource.initialize()
  .then(() => console.log("Data Source has been initialized"))
  .catch((error) => console.error("Error initializing Data Source", error));

./src/database/migrations/1648475278557-CreateCategories.ts

import { MigrationInterface, QueryRunner, Table } from "typeorm";

export class CreateCategories1648475278557 implements MigrationInterface {
  public async up(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.createTable(
      new Table({
        name: "categories",
        columns: [
          {
            name: "id",
            type: "uuid",
            isPrimary: true,
          },
          {
            name: "name",
            type: "varchar",
          },
        ],
      })
    );
  }

  public async down(queryRunner: QueryRunner): Promise<void> {
    await queryRunner.dropTable("categories");
  }
}

Command I'm using to run the migration

yarn typeorm-ts-node-esm -d ./src/database/data_source.ts migration:run

I always got the error:

Error during migration run:
Error: Invalid file path given: "/Users/myusername/Documents/code/backend/nodejs/typeorm-project/src/database/data_source.ts". File must contain a TypeScript / JavaScript code and export a DataSource instance.

I tried to convert this two files to JS, but still got the same error.
I'm using 0.3.4

[MrPostie]

For me the following is working:

package.json

...
"scripts": {
 ...
  "typeorm": "node --require ts-node/register ./node_modules/typeorm/cli.js"
}

I use a ormconfig.ts in the root of the project:

import { DataSource } from 'typeorm';

const devDataSource = new DataSource({
  type: 'mariadb',
  host: 'localhost',
  port: 3306,
  username: 'user',
  password: 'password',
  database: 'database',
  entities: ['**/*.entity.js'],
  synchronize: false,
  migrations: ['src/migrations/*.js'],
});

export default devDataSource;

Then I run the migrations with yarn typeorm migration:run -d ormconfig.ts and that works.

Maybe this works for others too.

My problem however is that the ormconfig.ts doesn't use process.env.HOSTNAME from the .env file in the root.

[myflowpl]

I'v just tried all examples here with no success

still getting Error: Invalid file path given: "C:\Users\username\nest-server\orm.ts". File must contain a TypeScript / JavaScript code and export a DataSource instance. error`

and this file is present, tried with .js files, but no success, tried ts-node, typeorm-ts-node-commonjs, typeorm-ts-node-esm and every where got the same error

[myflowpl]

I'v just tried all examples here with no success

still getting Error: Invalid file path given: "C:\Users\username\nest-server\orm.ts". File must contain a TypeScript / JavaScript code and export a DataSource instance. error`

and this file is present, tried with .js files, but no success, tried ts-node, typeorm-ts-node-commonjs, typeorm-ts-node-esm and every where got the same error

I found the problem 🎉

It looks like @MrPostie solution is working, in my case there was problem with my ormconfig.ts, I used old options for cli and it's not present in 0.3 version, and the problem is ts-node did not print the compilation error, it only shows that the file is missing. After I run npm run start:dev I noticed the ts compilation error, I fixed it, run the npm run typeorm migration:run command nad it's working. here are my files:

package.json

"scripts": {
  "typeorm": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js -d ormconfig.ts"
}

ormconfig.ts

import { DataSource } from 'typeorm';

const source = new DataSource({
  "type": "sqlite",
  "database": "./storage/nest.db",
  "entities": ["src/**/*.entity.ts"],
  "migrationsTableName": "migrations",
  "migrations": ["src/db/migrations/*.ts"],
});

export default source;

command

npm run typeorm migration:run

it also works with this package.json

"scripts": {
  "typeorm": "typeorm-ts-node-commonjs -d ormconfig.ts"
}

Thanks 💪

[ricardo85x]

MrPostie
Actually the only problem was the file with the Datasouce export that can't read the .env file

then this command:

yarn typeorm-ts-node-commonjs migration:run -d src/database/data_source.ts

works normally IF it is NOT using the .env variables

Now we need to figure out how make this file read the .env variables

[MrPostie]

Somewhere I read that the cli doesn't parse/read .env files. I've solved it using the dotenv package in the ormconfig.ts. Installed the package with yarn install dotenv @types/dotenv -D. Added import 'dotenv/config'; at the top of the ormconfig.ts and now process.env.XXX can be used.

[myflowpl]

Yes, you have to use dotenv or @nestjs/config witch is using dotenv internally.

OK, i had some other issue, some of the commands don't work, for example migration:create, what I found is that dataSource option -d is required in one command and breaks others, so you can't do general configuration like with typeorm v0.2

so my final working code is like this:

/src/db/ormconfig.ts

import { DataSource } from 'typeorm';
import { SqliteConnectionOptions } from 'typeorm/driver/sqlite/SqliteConnectionOptions';

const source = new DataSource({
  "type": "sqlite",
  "database": "./storage/nest.db",
  "entities": ["src/**/*.entity.ts"],
  "migrationsTableName": "migrations",
  "migrations": ["src/db/migrations/*.ts"],
});

export default source;

package.json

{
  "scripts": {
    "typeorm": "typeorm-ts-node-commonjs -d ./src/db/ormconfig.ts",
    "migration:generate": "npm run typeorm migration:generate",
    "migration:show": "npm run typeorm migration:show",
    "migration:run": "npm run typeorm migration:run",
    "migration:revert": "npm run typeorm migration:revert",
    "migration:create": "typeorm-ts-node-commonjs migration:create"
  }
}

In your terminal use it like this:

npm run migration:generate src/db/migrations/init-photos

npm run migration:create src/db/migrations/update-photos-data

npm run migration:show

npm run migration:run

npm run migration:revert

[msalsbery]

Yes, you have to use dotenv or @nestjs/config witch is using dotenv internally.

OK, i had some other issue, some of the commands don't work, for example migration:create, what I found is that dataSource option -d is required in one command and breaks others, so you can't do general configuration like with typeorm v0.2

so my final working code is like this:

/src/db/ormconfig.ts

import { DataSource } from 'typeorm';
import { SqliteConnectionOptions } from 'typeorm/driver/sqlite/SqliteConnectionOptions';

const source = new DataSource({
  "type": "sqlite",
  "database": "./storage/nest.db",
  "entities": ["src/**/*.entity.ts"],
  "migrationsTableName": "migrations",
  "migrations": ["src/db/migrations/*.ts"],
});

export default source;

package.json

{
  "scripts": {
    "typeorm": "typeorm-ts-node-commonjs -d ./src/db/ormconfig.ts",
    "migration:generate": "npm run typeorm migration:generate",
    "migration:show": "npm run typeorm migration:show",
    "migration:run": "npm run typeorm migration:run",
    "migration:revert": "npm run typeorm migration:revert",
    "migration:create": "typeorm-ts-node-commonjs migration:create"
  }
}

In your terminal use it like this:

npm run migration:generate src/db/migrations/init-photos

npm run migration:create src/db/migrations/update-photos-data

npm run migration:show

npm run migration:run

npm run migration:revert

@myflowpl How does migration:run work without -d option? How does cli find the migration? Do you have an ormconfig file still that it’s picking up using deprecated behavior?

I’m just curious, as I wouldn’t think it should work without specifying a DataSource and I’d want to make sure my cli and app runs are always using the same DataSource…

edit: Also, I use -d option on all cli commands without issue, so I’m curious why that doesn’t work for you

[myflowpl]

@myflowpl How does migration:run work without -d option? How does cli find the migration? Do you have an ormconfig file still that it’s picking up using deprecated behavior?

I’m just curious, as I wouldn’t think it should work without specifying a DataSource and I’d want to make sure my cli and app runs are always using the same DataSource…

edit: Also, I use -d option on all cli commands without issue, so I’m curious why that doesn’t work for you

@msalsbery note that I did npm script called typeorm where i defined data source -d option, then i use it in other commands like npm run typeorm migration:generate so I'm not using pure typeorm cli command, but one configured by me. Then when I have to use command without -d option i use directly the official cli, for example: typeorm-ts-node-commonjs migration:create.

My goal is to have predefined npm scripts so my team doesn't have to remember any extra options, they just give simple command to generate the migration and run it, and me as a one who knows how configurations works, I prepare the scripts.

That interesting it works for you with the -d option for all commands, may be it works whey you use cli directly without npm scripts, I wasted lots of time to make it working, and now it works so I'm going to leave it like it is without further investigation

[msalsbery]

@myflowpl I reread and noticed it was only the create that didn’t use -d and realized that operation wouldn’t need the database so that’s why it works. Sorry for the confusion and thank you for setting my head right!