CLI does not support async DataSource

[anton-pavlov-deel]

Issue Description

Starting from the 0.3.0 version, CLI requires to specify the file with DataSource instance instead of ormconfig. When I am trying to export async created DataSource it fails

Expected Behavior

Command ran successfully

Actual Behavior

Command failed with error: Error: Given data source file must contain export of a DataSource instance

Steps to Reproduce

Have your DataSource defined in the file:

const buildDataSource = async () => {
  // Provided config-like functionality just for example, it will fail anyway with just async function
  const dbConfig = await config.get('database');

  return new DataSource({
    type: 'mysql',
    host: dbConfig.host,
    port: 3306,
    username: dbConfig.username,
    password: dbConfig.password,
    database: dbConfig.database,
    logging: true,
    entities: typeormEntities,
    migrations: [ './src/components/database/migrations/**/*.ts' ]
  })
};

export default buildDataSource();

This works though:

const buildDataSource = () => {
  return new DataSource({
    type: 'mysql',
    host: '127.0.0.1',
    port: 3306,
    username: 'test',
    password: 'test',
    database: 'test',
    logging: true,
    entities: typeormEntities,
    migrations: [ './src/components/database/migrations/**/*.ts' ]
  })
};

export default buildDataSource();

My Environment

Dependency	Version
Operating System	Ubuntu 20.04.4 LTS
Node.js version	v16.13.0
Typescript version	4.6.3
TypeORM version	0.3.4

Relevant Database Driver(s)

I tested with mysql only but I do not think it is a driver problem at all

DB Type	Reproducible
aurora-mysql	no
aurora-postgres	no
better-sqlite3	no
cockroachdb	no
cordova	no
expo	no
mongodb	no
mysql	yes
nativescript	no
oracle	no
postgres	no
react-native	no
sap	no
spanner	no
sqlite	no
sqlite-abstract	no
sqljs	no
sqlserver	no

Are you willing to resolve this issue by submitting a Pull Request?

• ✖️ Yes, I have the time, and I know how to start.
• ✖️ Yes, I have the time, but I don't know how to start. I would need guidance.
• ✖️ No, I don’t have the time, but I can support (using donations) development.
• ✅ No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.

[Ginden]

If you are using ESM modules, try export default await .

[anton-pavlov-deel]

Unfortunately, I am using commonjs

[anton-pavlov-deel]

@Ginden I think the solution could be here smth like that:

        for (let fileExport in dataSourceFileExports) {
            const unpromisifiedExport = dataSourceFileExports[fileExport] instanceof Promise
                ? await dataSourceFileExports[fileExport]
                : dataSourceFileExports[fileExport];

            if (
                InstanceChecker.isDataSource(unpromisifiedExport)
            ) {
                dataSourceExports.push(unpromisifiedExport)
            }
        }

What do you think?

[Ginden]

Yes, it makes sense. Though, you don't need to make instanceof Promise check, we need to replace lines:

const dataSourceExports = []
for (let fileExport in dataSourceFileExports) {
    if (
        InstanceChecker.isDataSource(dataSourceFileExports[fileExport])
    ) {
        dataSourceExports.push(dataSourceFileExports[fileExport])
    }
}

with:

const dataSourceExports = []
for (let fileExport in dataSourceFileExports) {
    const fileExportValue = await dataSourceFileExports[fileExport];
    if (
        InstanceChecker.isDataSource(fileExportValue)
    ) {
        dataSourceExports.push(fileExportValue)
    }
}

Please, feel free to open PR for that. @pavlovanton

[anton-pavlov-deel]

@Ginden ^ Check pls

[SimonCockx]

A quick and very dirty hack which can be used until this gets resolved:

import { DataSource, DataSourceOptions, InstanceChecker } from 'typeorm';
import * as dotenv from 'dotenv';

dotenv.config();

function patchAsyncDataSourceSetup() {
    const oldIsDataSource = InstanceChecker.isDataSource;
    InstanceChecker.isDataSource = function(obj: unknown): obj is DataSource {
        if (obj instanceof Promise) {
            return true;
        }
        return oldIsDataSource(obj);
    }
}
patchAsyncDataSourceSetup();

const buildDataSource = async () => { ... }

export default buildDataSource();

[DanielMaranhao]

Any news on async DataSource?

[Ginden]

@DanielMaranhao Yes, it was fixed 0.3.8.

[DanielMaranhao]

Thanks @Ginden, but still I don't understand exactly the proper way to setup the async DataSource, sorry if it's something kind of obvious. Do you think you could help me?

I'm using TypeORM with NestJS, so I imagined this setup would work

const buildDataSource = async () => {
  await ConfigModule.envVariablesLoaded;

  return new DataSource({
    type: 'postgres',
    url: process.env.DATABASE_URL,
    entities: ['dist/domain/**/*.entity.js'],
    migrations: ['dist/database/migrations/*.js'],
  });
};

export default buildDataSource();

When I execute a migration generation, it stops giving an error. However, it appears to simply run and do nothing.

PS C:\Workspaces\ws-nest\nest-lecture-project> yarn migration:generate src/database/migrations/create-user
yarn run v1.22.17
$ yarn typeorm migration:generate -d dist/database/data-source -p src/database/migrations/create-user
$ C:\Workspaces\ws-nest\nest-lecture-project\node_modules\.bin\typeorm migration:generate -d dist/database/data-source -p src/database/migrations/create-user
Done in 1.89s.

Would you be able to tell if I am in the right track? I'm also thinking about trying to use a static class, to be able to leverage a configuration namespace and partial registration through Nest, though I'm not sure if it's possible in this case.

[DanielMaranhao]

Hello again, sorry for the inconvenience. It was indeed a quite obvious solution. What I had to do was the following: even though I'm using Nest, this DataSource has no integration with it, and so this ConfigModule hook to await for the env to load would not work. I needed to install dotenv and dotenv expand manually, even though the ConfigModule already uses them under the hood. And so applying both of them manually, it finally worked. Even the factory became unnecessary.

import * as dotenv from 'dotenv';
import * as dotenvExpand from 'dotenv-expand';
import { DataSource } from 'typeorm';

dotenvExpand.expand(dotenv.config());

export default new DataSource({
  type: 'postgres',
  url: process.env.DATABASE_URL,
  entities: ['dist/domain/**/*.entity.js'],
  migrations: ['dist/database/migrations/*.js'],
});

I'm leaving this here just in case someone passes through a similar situation. Sorry once again due to the obvious question, and best wishes to all.

---CORRECTION---

It is not necessary to install dotenv and dotenv-expand again. You can just import them as described above, however autocompletion of the module won't work, probably because it's a "sub-dependency".

[Kinco-dev]

@DanielMaranhao Thank you very much, my hero !! Very nice of you to come back and give the answer that worked for you

[alper]

Why doesn't the TypeORM CLI use the datasource definition that's already in Nest?

[mattzuba]

Why doesn't the TypeORM CLI use the datasource definition that's already in Nest?

You can create a ormconfig.ts file like this that will allow that:

import { NestFactory } from '@nestjs/core';
import { AppModule } from './src/app/app.module';
import { DataSource } from 'typeorm';

export default NestFactory.create(AppModule)
  .then((app) => app.get(DataSource))
  .then((dataSource) => Promise.all([dataSource, dataSource.destroy()]))
  .then(([dataSource]) => dataSource);

[Borderliner]

@mattzuba Do you happen to know how to get this working with a database.providers.ts file and database.module.ts file as the following:

// database.providers.ts
import { ConfigService } from '@nestjs/config'
import { DataSource } from 'typeorm'
import Migrations from './database.migrations'

export const databaseProviders = [
    {
        inject: [ConfigService],
        provide: DataSource,
        useFactory: async (config: ConfigService) => {
            const databaseConfig = config.get('database')
            const dataSource = new DataSource({
                type: databaseConfig['driver'],
                host: databaseConfig['host'],
                port: databaseConfig['port'],
                username: databaseConfig['username'],
                password: databaseConfig['password'],
                database: databaseConfig['name'],
                entities: [
                    __dirname + '/../**/*.entity{.ts,.js}',
                ],
                synchronize: process.env.NODE_ENV === 'dev',
                migrations: [...Migrations],
                migrationsRun: true
            })
            try {
                if (!dataSource.isInitialized) {
                    await dataSource.initialize()
                }
            } catch (error) {
                console.error(error?.message)
            }
            return dataSource
        }
    }
]

// database.module.ts
import { Module } from '@nestjs/common'
import { databaseProviders } from './database.providers'
import { ConfigService } from '@nestjs/config'

@Module({
    providers: [...databaseProviders, ConfigService],
    exports: [...databaseProviders]
})
export class DatabaseModule { }

I have this script at package.json:
"migration:dev:run": "NODE_ENV=dev typeorm-ts-node-commonjs migration:run --dataSource ./ormconfig.ts",
I get this error:

CannotExecuteNotConnectedError: Cannot execute operation on "default" connection because connection is not yet established.
    at DataSource.destroy (/home/reza/Lab/Projects/smm_js/backend/node_modules/src/data-source/DataSource.ts:306:19)
    at /home/x/Lab/Projects/smm_js/backend/ormconfig.ts:7:63
    at async Function.loadDataSource (/home/x/Lab/Projects/smm_js/backend/node_modules/src/commands/CommandUtils.ts:47:39)
    at async Object.handler (/home/x/Lab/Projects/smm_js/backend/node_modules/src/commands/MigrationRunCommand.ts:42:26)