feat(ts-morph): use .d.ts files for ts-morph discovery

.eslintrc.js

@@ -10,7 +10,7 @@ module.exports = {
     'plugin:@typescript-eslint/eslint-recommended',
     'plugin:@typescript-eslint/recommended',
   ],
-  ignorePatterns: ['node_modules', 'dist', 'coverage', '**/*.js'],
+  ignorePatterns: ['node_modules', 'dist', 'coverage', '**/*.js', '**/*.d.ts'],
   parser: '@typescript-eslint/parser',
   parserOptions: {
     'project': 'tsconfig.json',

docs/docs/configuration.md

@@ -29,16 +29,21 @@ MikroORM.init({
 });
 ```
 
-By default, `TsMorphMetadataProvider` is used that analyses your entity source files. You can
-use `ReflectMetadataProvider` if you do not want the source file analyses to happen. 
-If you aim to use plain JavaScript instead of TypeScript, use the `JavaScriptMetadataProvider`.
+By default, `ReflectMetadataProvider` is used that leverages the `reflect-metadata`. 
+You can also use `TsMorphMetadataProvider` by installing `@mikro-orm/reflection`. 
+This provider will analyse your entity source files (or `.d.ts` type definition files). 
+If you aim to use plain JavaScript instead of TypeScript, use `EntitySchema` or 
+the `JavaScriptMetadataProvider`.
 
 > You can also implement your own metadata provider and use it instead. To do so, extend the 
 > `MetadataProvider` class.
 
 ```typescript
+import { MikroORM } from '@mikro-orm/core';
+import { TsMorphMetadataProvider } from '@mikro-orm/reflection';
+
 MikroORM.init({
-  metadataProvider: ReflectMetadataProvider,
+  metadataProvider: TsMorphMetadataProvider,
 });
 ```
 
@@ -50,9 +55,6 @@ MikroORM.init({
     warnWhenNoEntities: false, // by default, discovery throws when no entity is processed
     requireEntitiesArray: true, // force usage of `entities` instead of `entitiesDirs`
     alwaysAnalyseProperties: false, // do not analyse properties when not needed (with ts-morph)
-
-    // you can explicitly specify the path to your tsconfig.json (used only when `entitiesDirsTs` is not provided)
-    tsConfigPath: string,
   },
 });
 ```

docs/docs/metadata-cache.md

@@ -2,10 +2,17 @@
 title: Metadata Cache
 ---
 
+> In v4 you need to explicitly install `@mikro-orm/reflection` to use `TsMorphMetadataProvider`.
+
 Under the hood, `MikroORM` uses [`ts-morph`](https://github.com/dsherret/ts-morph) to read 
 TypeScript source files of all entities to be able to detect all types. Thanks to this, 
 defining the type is enough for runtime validation.
 
+If you use folder-based discovery (via `entitiesDirs`), you should specify paths to
+the compiled entities via `entitiesDirs` as well as paths to the TS source files of
+those entities via `entitiesDirsTs`. When you run the ORM via `ts-node`, the latter
+will be used automatically, or if you explicitly pass `tsNode: true` in the config.
+
 This process can be a bit slow, mainly because `ts-morph` will scan all your source files
 based on your `tsconfig.json`. You can speed up this process by whitelisting only the folders
 where your entities are via `entitiesDirsTs` option. 

docs/docs/metadata-providers.md

@@ -3,7 +3,7 @@ title: Metadata Providers
 ---
 
 As part of entity discovery process, MikroORM uses so called `MetadataProvider` to get necessary
-type information about your entities' properties. There are 3 built in metadata providers you can 
+type information about your entities' properties. There are 3 built-in metadata providers you can 
 use:
 
 > You can also implement custom metadata provider by extending abstract `MetadataProvider` class.
@@ -14,11 +14,10 @@ By default, MikroORM uses [`ts-morph`](https://github.com/dsherret/ts-morph) to
 TypeScript source files of all entities to be able to detect all types. Thanks to this, 
 defining the type is enough for runtime validation.
 
-This process can be a bit slow as well as memory consuming, mainly because `ts-morph` will
-scan all your source files based on your `tsconfig.json`. You can speed up this process by 
-whitelisting only the folders where your entities are via `entitiesDirsTs` option. 
-
-> You can specify the path to `tsconfig.json` manually via `discovery: { tsConfigPath: '...' }`.
+If you use folder-based discovery (via `entitiesDirs`), you should specify paths to
+the compiled entities via `entitiesDirs` as well as paths to the TS source files of
+those entities via `entitiesDirsTs`. When you run the ORM via `ts-node`, the latter
+will be used automatically, or if you explicitly pass `tsNode: true` in the config.
 
 After the discovery process ends, all [metadata will be cached](metadata-cache.md). By default, 
 `FileCacheAdapter` will be used to store the cache inside `./temp` folder in JSON files. 
@@ -46,7 +45,6 @@ Next step is to enable `emitDecoratorMetadata` flag in your `tsconfig.json`.
 ```typescript
 await MikroORM.init({
   metadataProvider: ReflectMetadataProvider,
-  cache: { enabled: false },
   // ...
 });
 ```
@@ -133,7 +131,6 @@ manually.
 ```typescript
 await MikroORM.init({
   metadataProvider: JavaScriptMetadataProvider,
-  cache: { enabled: false },
   // ...
 });
 ```

docs/docs/upgrading-v3-to-v4.md

@@ -96,3 +96,10 @@ You need to specify the platform type either via `type` option or provide the dr
 implementation via `driver` option. 
 
 Available platforms types: `[ 'mongo', 'mysql', 'mariadb', 'postgresql', 'sqlite' ]`
+
+## Removed configuration `discovery.tsConfigPath`
+
+Removed as it is no longer needed, it was used only for `TsMorphMetadataProvider`,
+when the `entitiesDirsTs` were not explicitly provided. In v4, this is no longer
+needed, as ts-morph discovery will use `d.ts` files instead, that should be located
+next to the compiled entities. 

packages/core/src/metadata/MetadataDiscovery.ts

@@ -82,6 +82,7 @@ export class MetadataDiscovery {
   private async discoverDirectory(basePath: string): Promise<void> {
     const files = await globby(Utils.normalizePath(basePath, '*'), { cwd: Utils.normalizePath(this.config.get('baseDir')) });
     this.logger.log('discovery', `- processing ${chalk.cyan(files.length)} files from directory ${chalk.cyan(basePath)}`);
+    const found: [ObjectConstructor, string][] = [];
 
     for (const filepath of files) {
       const filename = basename(filepath);
@@ -107,6 +108,10 @@ export class MetadataDiscovery {
       }
 
       this.metadata.set(name, Utils.copy(MetadataStorage.getMetadata(name, path)));
+      found.push([target, path]);
+    }
+
+    for (const [target, path] of found) {
       await this.discoverEntity(target, path);
     }
   }

packages/core/src/utils/Configuration.ts

@@ -26,7 +26,6 @@ export class Configuration<D extends IDatabaseDriver = IDatabaseDriver> {
       requireEntitiesArray: false,
       alwaysAnalyseProperties: true,
       disableDynamicFileAccess: false,
-      tsConfigPath: process.cwd() + '/tsconfig.json',
     },
     strict: false,
     // eslint-disable-next-line no-console
@@ -318,7 +317,6 @@ export interface MikroORMOptions<D extends IDatabaseDriver = IDatabaseDriver> ex
     requireEntitiesArray?: boolean;
     alwaysAnalyseProperties?: boolean;
     disableDynamicFileAccess?: boolean;
-    tsConfigPath?: string;
   };
   type?: keyof typeof Configuration.PLATFORMS;
   driver?: { new (config: Configuration): D };

packages/reflection/src/TsMorphMetadataProvider.ts

@@ -1,6 +1,6 @@
 import globby from 'globby';
 import { Project, PropertyDeclaration, SourceFile } from 'ts-morph';
-import { EntityMetadata, EntityProperty, MetadataProvider, Utils } from '@mikro-orm/core';
+import { EntityMetadata, EntityProperty, MetadataProvider, MetadataStorage, Utils } from '@mikro-orm/core';
 
 export class TsMorphMetadataProvider extends MetadataProvider {
 
@@ -19,9 +19,14 @@ export class TsMorphMetadataProvider extends MetadataProvider {
     await this.initProperties(meta);
   }
 
-  async getExistingSourceFile(meta: EntityMetadata): Promise<SourceFile> {
-    const path = meta.path.match(/\/[^/]+$/)![0].replace(/\.js$/, '.ts');
-    return this.getSourceFile(path)!;
+  async getExistingSourceFile(meta: EntityMetadata, ext?: string, validate = true): Promise<SourceFile> {
+    if (!ext) {
+      return await this.getExistingSourceFile(meta, '.d.ts', false) || await this.getExistingSourceFile(meta, '.ts');
+    }
+
+    const path = meta.path.match(/\/[^/]+$/)![0].replace(/\.js$/, ext);
+
+    return (await this.getSourceFile(path, validate))!;
   }
 
   protected async initProperties(meta: EntityMetadata): Promise<void> {
@@ -84,14 +89,14 @@ export class TsMorphMetadataProvider extends MetadataProvider {
     return { type, optional };
   }
 
-  private async getSourceFile(file: string): Promise<SourceFile> {
+  private async getSourceFile(file: string, validate: boolean): Promise<SourceFile | undefined> {
     if (!this.sources) {
       await this.initSourceFiles();
     }
 
     const source = this.sources.find(s => s.getFilePath().endsWith(file));
 
-    if (!source) {
+    if (!source && validate) {
       throw new Error(`Source file for entity ${file} not found, check your 'entitiesDirsTs' option. If you are using webpack, see https://bit.ly/35pPDNn`);
     }
 
@@ -119,7 +124,8 @@ export class TsMorphMetadataProvider extends MetadataProvider {
       const dirs = await this.validateDirectories(tsDirs);
       this.sources = this.project.addSourceFilesAtPaths(dirs);
     } else {
-      this.sources = this.project.addSourceFilesFromTsConfig(this.config.get('discovery').tsConfigPath!);
+      const dirs = Object.values(MetadataStorage.getMetadata()).map(m => m.path.replace(/\.js$/, '.d.ts'));
+      this.sources = this.project.addSourceFilesAtPaths(dirs);
     }
   }
 

tests/MetadataValidator.test.ts

@@ -1,4 +1,4 @@
-import { ReferenceType, MikroORM, MetadataStorage, MetadataValidator, ReflectMetadataProvider } from '@mikro-orm/core';
+import { ReferenceType, MikroORM, MetadataStorage, MetadataValidator } from '@mikro-orm/core';
 import { Author2, Book2, BookTag2, FooBar2, FooBaz2, Publisher2, Test2 } from './entities-sql';
 import { BASE_DIR } from './bootstrap';
 
@@ -114,7 +114,6 @@ describe('MetadataValidator', () => {
       dbName: `mikro_orm_test`,
       port: 3307,
       cache: { enabled: true },
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       type: 'mysql',
       baseDir: BASE_DIR,
     })).rejects.toThrowError(`Entity 'FooBar2' extends unknown base entity 'BaseEntity22', please make sure to provide it in 'entities' array when initializing the ORM`);
@@ -124,7 +123,6 @@ describe('MetadataValidator', () => {
       entities: [Author2, Book2, BookTag2, Publisher2, Test2],
       dbName: `mikro_orm_test`,
       port: 3307,
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       type: 'mysql',
       baseDir: BASE_DIR,
     })).rejects.toThrowError(`Entity 'Author2' extends unknown base entity 'BaseEntity2', please make sure to provide it in 'entities' array when initializing the ORM`);

tests/SchemaGenerator.test.ts

@@ -10,7 +10,6 @@ describe('SchemaGenerator', () => {
     const dbName = `mikro_orm_test_${Date.now()}`;
     const orm = await MikroORM.init({
       entities: [FooBar2, FooBaz2, BaseEntity22],
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       dbName,
       port: 3307,
       baseDir: BASE_DIR,
@@ -27,7 +26,6 @@ describe('SchemaGenerator', () => {
     const dbName = `mikro_orm_test_${Date.now()}`;
     const orm = await MikroORM.init({
       entities: [FooBar2, FooBaz2, BaseEntity22],
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       dbName,
       port: 3307,
       baseDir: BASE_DIR,
@@ -47,7 +45,6 @@ describe('SchemaGenerator', () => {
     const dbName = `mikro_orm_test_${Date.now()}`;
     const orm = await MikroORM.init({
       entities: [FooBar2, FooBaz2, BaseEntity22],
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       dbName,
       port: 3307,
       baseDir: BASE_DIR,
@@ -65,7 +62,6 @@ describe('SchemaGenerator', () => {
     const dbName = `mikro_orm_test_${Date.now()}`;
     const orm = await MikroORM.init({
       entities: [FooBar2, FooBaz2, BaseEntity22],
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       dbName,
       port: 3307,
       baseDir: BASE_DIR,
@@ -370,7 +366,6 @@ describe('SchemaGenerator', () => {
     const dbName = `mikro_orm_test_${Date.now()}`;
     const orm = await MikroORM.init({
       entities: [FooBar2, FooBaz2, BaseEntity22],
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       dbName,
       baseDir: BASE_DIR,
       type: 'postgresql',
@@ -386,7 +381,6 @@ describe('SchemaGenerator', () => {
     const dbName = `mikro_orm_test_${Date.now()}`;
     const orm = await MikroORM.init({
       entities: [FooBar2, FooBaz2, BaseEntity22],
-      discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
       dbName,
       baseDir: BASE_DIR,
       type: 'postgresql',

tests/bootstrap.ts

@@ -59,7 +59,6 @@ export async function initORMMySql<D extends MySqlDriver | MariaDbDriver = MySql
       Author2, Address2, Book2, BookTag2, Publisher2, Test2, FooBar2, FooBaz2, FooParam2, Configuration2, BaseEntity2, BaseEntity22,
       Car2, CarOwner2, User2, BaseUser2, Employee2, Manager2, CompanyOwner2, Sandwich,
     ],
-    discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
     clientUrl: `mysql://root@127.0.0.1:3306/mikro_orm_test`,
     port: type === 'mysql' ? 3307 : 3309,
     baseDir: BASE_DIR,
@@ -96,7 +95,6 @@ export async function initORMMySql<D extends MySqlDriver | MariaDbDriver = MySql
 export async function initORMPostgreSql() {
   const orm = await MikroORM.init<PostgreSqlDriver>({
     entities: [Author2, Address2, Book2, BookTag2, Publisher2, Test2, FooBar2, FooBaz2, FooParam2, Label2, Configuration2, BaseEntity2, BaseEntity22],
-    discovery: { tsConfigPath: BASE_DIR + '/tsconfig.test.json' },
     dbName: `mikro_orm_test`,
     baseDir: BASE_DIR,
     type: 'postgresql',

tests/reflection/TsMorphMetadataProvider.test.ts

@@ -11,14 +11,30 @@ export type IdentifiedReference<T, PK extends keyof T = 'id' & keyof T> = { [K i
 
 describe('TsMorphMetadataProvider', () => {
 
-  test('should load project file when no entitiesDirsTs provided', async () => {
+  test('should load TS files directly', async () => {
     const orm = await MikroORM.init({
       entities: [Author, Book, Publisher, BaseEntity, BaseEntity3, BookTagSchema, Test, FooBaz, FooBar],
       baseDir: __dirname,
       clientUrl: 'mongodb://localhost:27017,localhost:27018,localhost:27019/mikro-orm-test?replicaSet=rs0',
       type: 'mongo',
       cache: { enabled: false },
-      discovery: { tsConfigPath: __dirname + '/../tsconfig.json', alwaysAnalyseProperties: false },
+      discovery: { alwaysAnalyseProperties: false },
+      metadataProvider: TsMorphMetadataProvider,
+    });
+
+    expect(Object.keys(orm.getMetadata().getAll()).sort()).toEqual(['Author', 'Book', 'BookTag', 'FooBar', 'FooBaz', 'Publisher', 'Test']);
+    await orm.close();
+  });
+
+  test('should load entities based on .d.ts files', async () => {
+    const orm = await MikroORM.init({
+      entitiesDirs: ['./entities-compiled'],
+      tsNode: false,
+      baseDir: __dirname,
+      clientUrl: 'mongodb://localhost:27017,localhost:27018,localhost:27019/mikro-orm-test?replicaSet=rs0',
+      type: 'mongo',
+      cache: { enabled: false },
+      discovery: { alwaysAnalyseProperties: false },
       metadataProvider: TsMorphMetadataProvider,
     });
 

tests/reflection/entities-compiled/Author.d.ts

@@ -0,0 +1,19 @@
+import { Book } from './Book';
+import { BaseEntity } from './BaseEntity';
+import { Collection } from '../TsMorphMetadataProvider.test';
+export declare class Author extends BaseEntity {
+    name: string;
+    email: string;
+    age?: number;
+    termsAccepted: boolean;
+    optional?: boolean;
+    identities?: string[];
+    born?: Date;
+    books: Collection<Book>;
+    friends: Collection<Author>;
+    favouriteBook: Book;
+    favouriteAuthor: Author;
+    version: number;
+    versionAsString: string;
+    constructor(name: string, email: string);
+}