feat(core): add support for multiple schemas (including UoW) (#2296)

Entity instances now hold schema name (as part of `WrappedEntity`). Managed entities will have the schema from `FindOptions` or metadata. Methods that create new entity instances like `em.create()` or `em.getReference()` now have an options parameter to allow setting the schema. We can also use `wrap(entity).setSchema()`.

Entities can now specify `@Entity({ schema: '*' })`, that way they will be ignored in `SchemaGenerator` unless `schema` option is specified.
    
- if we have schema specified on entity level, it only exists in that schema
- if we have * schema on entity, it can exist in any schema, always controlled by the parameter
- no schema on entity - default schema or from global orm config

Closes #2074

BREAKING CHANGE:
`em.getReference()` now has options parameter.

docs/docs/entity-manager-api.md

@@ -203,9 +203,9 @@ Shortcut for `wrap(entity).assign(data, { em })`
 
 ----
 
-#### `getReference(entityName: EntityName<T>, id: Primary<T>, wrapped?: boolean, convertCustomTypes?: boolean): T | Reference<T>`
+#### `getReference(entityName: EntityName<T>, id: Primary<T>, options?: GetReferenceOptions): T | Reference<T>`
 
-Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded
+Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded.
 
 ----
 

docs/docs/entity-references.md

@@ -170,13 +170,13 @@ author!: IdentifiedReference<Author>;
 
 When you define the property as `Reference` wrapper, you will need to assign the `Reference`
 to it instead of the entity. You can create it via `Reference.create()` factory, or use `wrapped`
-parameter of `em.getReference()`:
+option of `em.getReference()`:
 
 ```typescript
 const book = await orm.em.findOne(Book, 1);
 const repo = orm.em.getRepository(Author);
 
-book.author = repo.getReference(2, true);
+book.author = repo.getReference(2, { wrapped: true });
 
 // same as:
 book.author = Reference.create(repo.getReference(2));

docs/docs/multiple-schemas.md

@@ -6,14 +6,16 @@ In MySQL and PostgreSQL is is possible to define your entities in multiple schem
 terminology, it is called database, but from implementation point of view, it is a schema. 
 
 > To use multiple schemas, your connection needs to have access to all of them (multiple 
-> connections are not supported).
+> connections are not supported in a single MikroORM instance).
 
-All you need to do is simply define the table name including schema name in `collection` option:
+All you need to do is simply define the schema name via `schema` options, or 
+table name including schema name in `tableName` option:
 
 ```typescript
-@Entity({ tableName: 'first_schema.foo' })
+@Entity({ schema: 'first_schema' })
 export class Foo { ... }
 
+// or alternatively we can specify it inside custom table name
 @Entity({ tableName: 'second_schema.bar' })
 export class Bar { ... }
 ```
@@ -35,3 +37,44 @@ To create entity in specific schema, you will need to use `QueryBuilder`:
 const qb = em.createQueryBuilder(User);
 await qb.insert({ email: 'foo@bar.com' }).withSchema('client-123');
 ```
+
+## Wildcard Schema
+
+Since v5, MikroORM also supports defining entities that can exist in multiple
+schemas. To do that, we just specify wildcard schema:
+
+```ts
+@Entity({ schema: '*' })
+export class Book {
+
+  @PrimaryKey()
+  id!: number;
+
+  @Property({ nullable: true })
+  name?: string;
+
+  @ManyToOne(() => Author, { nullable: true, onDelete: 'cascade' })
+  author?: Author;
+
+  @ManyToOne(() => Book, { nullable: true })
+  basedOn?: Book;
+
+}
+```
+
+Entities like this will be by default ignored when using `SchemaGenerator`, 
+as we need to specify which schema to use. For that we need to use the `schema`
+option of the `createSchema/updateSchema/dropSchema` methods or the `--schema` 
+CLI parameter.
+
+On runtime, the wildcard schema will be replaced with either `FindOptions.schema`,
+or with the `schema` option from the ORM config.
+
+### Note about migrations
+
+Currently, this is not supported via migrations, they will always ignore
+wildcard schema entities, and `SchemaGenerator` needs to be used explicitly.
+Given the dynamic nature of such entities, it makes sense to only sync the 
+schema dynamically, e.g. in an API endpoint. We could still use the ORM 
+migrations, but we need to add the dynamic schema queries manually to migration
+files. It makes sense to use the `safe` mode for such queries.

docs/docs/upgrading-v4-to-v5.md

@@ -26,16 +26,22 @@ List of such methods:
 - `em.findOne()`
 - `em.findOneOrFail()`
 - `em.findAndCount()`
+- `em.getReference()`
 - `em.merge()`
 - `em.fork()`
 - `em.begin()`
+- `em.assign()`
+- `em.create()`
 - `repo.find()`
 - `repo.findOne()`
 - `repo.findOneOrFail()`
 - `repo.findAndCount()`
 - `repo.findAll()`
+- `repo.getReference()`
 - `repo.merge()`
 - `collection.init()`
+- `repo.create()`
+- `repo.assign()`
 
 This also applies to the methods on `IDatabaseDriver` interface.
 

packages/cli/src/commands/SchemaCommandFactory.ts

@@ -1,6 +1,6 @@
 import type { Arguments, Argv, CommandModule } from 'yargs';
 import c from 'ansi-colors';
-import type { Dictionary, MikroORM } from '@mikro-orm/core';
+import type { MikroORM } from '@mikro-orm/core';
 import type { AbstractSqlDriver } from '@mikro-orm/knex';
 import { SchemaGenerator } from '@mikro-orm/knex';
 import { CLIHelper } from '../CLIHelper';
@@ -38,6 +38,7 @@ export class SchemaCommandFactory {
       type: 'boolean',
       desc: 'Runs queries',
     });
+
     if (command !== 'fresh') {
       args.option('d', {
         alias: 'dump',
@@ -50,6 +51,11 @@ export class SchemaCommandFactory {
       });
     }
 
+    args.option('schema', {
+      type: 'string',
+      desc: 'Set the current schema for wildcard schema entities',
+    });
+
     if (command === 'create' || command === 'fresh') {
       args.option('seed', {
         type: 'string',
@@ -91,15 +97,15 @@ export class SchemaCommandFactory {
 
     const orm = await CLIHelper.getORM() as MikroORM<AbstractSqlDriver>;
     const generator = new SchemaGenerator(orm.em);
-    const params = SchemaCommandFactory.getOrderedParams(args, method);
+    const params = { wrap: !args.fkChecks, ...args };
 
     if (args.dump) {
       const m = `get${method.substr(0, 1).toUpperCase()}${method.substr(1)}SchemaSQL` as 'getCreateSchemaSQL' | 'getUpdateSchemaSQL' | 'getDropSchemaSQL';
       const dump = await generator[m](params);
       CLIHelper.dump(dump, orm.config);
     } else if (method === 'fresh') {
-      await generator.dropSchema(SchemaCommandFactory.getOrderedParams(args, 'drop'));
-      await generator.createSchema(SchemaCommandFactory.getOrderedParams(args, 'create'));
+      await generator.dropSchema(params);
+      await generator.createSchema(params);
     } else {
       const m = method + 'Schema';
       await generator[m](params);
@@ -114,26 +120,18 @@ export class SchemaCommandFactory {
     await orm.close(true);
   }
 
-  private static getOrderedParams(args: Arguments<Options>, method: SchemaMethod): Dictionary {
-    const ret: Dictionary = { wrap: !args.fkChecks };
-
-    if (method === 'update') {
-      ret.safe = args.safe;
-      ret.dropTables = args.dropTables;
-    }
-
-    if (method === 'drop') {
-      ret.dropMigrationsTable = args.dropMigrationsTable;
-
-      if (!args.dump) {
-        ret.dropDb = args.dropDb;
-      }
-    }
-
-    return ret;
-  }
-
 }
 
 type SchemaMethod = 'create' | 'update' | 'drop' | 'fresh';
-export type Options = { dump: boolean; run: boolean; fkChecks: boolean; dropMigrationsTable: boolean; dropDb: boolean; dropTables: boolean; safe: boolean; seed: string };
+
+export type Options = {
+  dump: boolean;
+  run: boolean;
+  fkChecks: boolean;
+  dropMigrationsTable: boolean;
+  dropDb: boolean;
+  dropTables: boolean;
+  safe: boolean;
+  seed: string;
+  schema: string;
+};

packages/core/src/EntityManager.ts

@@ -5,7 +5,7 @@ import { QueryHelper, TransactionContext, Utils } from './utils';
 import type { AssignOptions, EntityLoaderOptions, EntityRepository, IdentifiedReference } from './entity';
 import { EntityAssigner, EntityFactory, EntityLoader, EntityValidator, Reference } from './entity';
 import { UnitOfWork } from './unit-of-work';
-import type { CountOptions, DeleteOptions, EntityManagerType, FindOneOptions, FindOneOrFailOptions, FindOptions, IDatabaseDriver, UpdateOptions } from './drivers';
+import type { CountOptions, DeleteOptions, EntityManagerType, FindOneOptions, FindOneOrFailOptions, FindOptions, IDatabaseDriver, InsertOptions, LockOptions, UpdateOptions, GetReferenceOptions } from './drivers';
 import type { AnyEntity, AutoPath, Dictionary, EntityData, EntityDictionary, EntityDTO, EntityMetadata, EntityName, FilterDef, FilterQuery, GetRepository, Loaded, New, Populate, PopulateOptions, Primary } from './typings';
 import type { IsolationLevel, LoadStrategy } from './enums';
 import { LockMode, ReferenceType, SCALAR_TYPES } from './enums';
@@ -289,7 +289,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
     where = await this.processWhere(entityName, where, options, 'read');
     this.validator.validateEmptyWhere(where);
     this.checkLockRequirements(options.lockMode, meta);
-    let entity = this.getUnitOfWork().tryGetById<T>(entityName, where);
+    let entity = this.getUnitOfWork().tryGetById<T>(entityName, where, options.schema);
     const isOptimisticLocking = !Utils.isDefined(options.lockMode) || options.lockMode === LockMode.OPTIMISTIC;
 
     if (entity && !this.shouldRefresh<T>(meta, entity, options) && isOptimisticLocking) {
@@ -383,9 +383,9 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
   /**
    * Runs your callback wrapped inside a database transaction.
    */
-  async lock(entity: AnyEntity, lockMode: LockMode, options: { lockVersion?: number | Date; lockTableAliases?: string[] } | number | Date = {}): Promise<void> {
-    options = Utils.isPlainObject(options) ? options as Dictionary : { lockVersion: options };
-    await this.getUnitOfWork().lock(entity, lockMode, options.lockVersion, options.lockTableAliases);
+  async lock<T>(entity: T, lockMode: LockMode, options: LockOptions | number | Date = {}): Promise<void> {
+    options = Utils.isPlainObject(options) ? options as LockOptions : { lockVersion: options };
+    await this.getUnitOfWork().lock(entity, { lockMode, ...options });
   }
 
   /**
@@ -401,7 +401,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
   /**
    * Fires native insert query. Calling this has no side effects on the context (identity map).
    */
-  async nativeInsert<T extends AnyEntity<T>>(entityNameOrEntity: EntityName<T> | T, data?: EntityData<T>): Promise<Primary<T>> {
+  async nativeInsert<T extends AnyEntity<T>>(entityNameOrEntity: EntityName<T> | T, data?: EntityData<T>, options: InsertOptions<T> = {}): Promise<Primary<T>> {
     let entityName;
 
     if (data === undefined) {
@@ -413,7 +413,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
 
     data = QueryHelper.processObjectParams(data) as EntityData<T>;
     this.validator.validateParams(data, 'insert data');
-    const res = await this.driver.nativeInsert(entityName, data, { ctx: this.transactionContext });
+    const res = await this.driver.nativeInsert(entityName, data, { ctx: this.transactionContext, ...options });
 
     return res.insertId as Primary<T>;
   }
@@ -427,7 +427,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
     where = await this.processWhere(entityName, where, options, 'update');
     this.validator.validateParams(data, 'update data');
     this.validator.validateParams(where, 'update condition');
-    const res = await this.driver.nativeUpdate(entityName, where, data, { ctx: this.transactionContext });
+    const res = await this.driver.nativeUpdate(entityName, where, data, { ctx: this.transactionContext, ...options });
 
     return res.affectedRows;
   }
@@ -439,15 +439,15 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
     entityName = Utils.className(entityName);
     where = await this.processWhere(entityName, where, options, 'delete');
     this.validator.validateParams(where, 'delete condition');
-    const res = await this.driver.nativeDelete(entityName, where, { ctx: this.transactionContext });
+    const res = await this.driver.nativeDelete(entityName, where, { ctx: this.transactionContext, ...options });
 
     return res.affectedRows;
   }
 
   /**
    * Maps raw database result to an entity and merges it to this EntityManager.
    */
-  map<T extends AnyEntity<T>>(entityName: EntityName<T>, result: EntityDictionary<T>): T {
+  map<T extends AnyEntity<T>>(entityName: EntityName<T>, result: EntityDictionary<T>, options: { schema?: string } = {}): T {
     entityName = Utils.className(entityName);
     const meta = this.metadata.get(entityName);
     const data = this.driver.mapResult(result, meta) as Dictionary;
@@ -460,7 +460,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
       }
     });
 
-    return this.merge<T>(entityName, data as EntityData<T>, { convertCustomTypes: true, refresh: true });
+    return this.merge<T>(entityName, data as EntityData<T>, { convertCustomTypes: true, refresh: true, ...options });
   }
 
   /**
@@ -486,7 +486,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
 
     entityName = Utils.className(entityName as string);
     this.validator.validatePrimaryKey(data as EntityData<T>, this.metadata.get(entityName));
-    let entity = this.getUnitOfWork().tryGetById<T>(entityName, data as FilterQuery<T>, false);
+    let entity = this.getUnitOfWork().tryGetById<T>(entityName, data as FilterQuery<T>, options.schema, false);
 
     if (entity && entity.__helper!.__initialized && !options.refresh) {
       return entity;
@@ -510,7 +510,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
    * the whole `data` parameter will be passed. This means we can also define `constructor(data: Partial<T>)` and
    * `em.create()` will pass the data into it (unless we have a property named `data` too).
    */
-  create<T extends AnyEntity<T>, P extends string = never>(entityName: EntityName<T>, data: EntityData<T>, options: { managed?: boolean } = {}): New<T, P> {
+  create<T extends AnyEntity<T>, P extends string = never>(entityName: EntityName<T>, data: EntityData<T>, options: { managed?: boolean; schema?: string } = {}): New<T, P> {
     return this.getEntityFactory().create<T, P>(entityName, data, { ...options, newEntity: !options.managed });
   }
 
@@ -524,7 +524,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
   /**
    * Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded
    */
-  getReference<T extends AnyEntity<T>, PK extends keyof T>(entityName: EntityName<T>, id: Primary<T>, wrapped: true, convertCustomTypes?: boolean): IdentifiedReference<T, PK>;
+  getReference<T extends AnyEntity<T>, PK extends keyof T>(entityName: EntityName<T>, id: Primary<T>, options: Omit<GetReferenceOptions, 'wrapped'> & { wrapped: true }): IdentifiedReference<T, PK>;
 
   /**
    * Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded
@@ -534,17 +534,18 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
   /**
    * Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded
    */
-  getReference<T extends AnyEntity<T>>(entityName: EntityName<T>, id: Primary<T>, wrapped: false, convertCustomTypes?: boolean): T;
+  getReference<T extends AnyEntity<T>>(entityName: EntityName<T>, id: Primary<T>, options: Omit<GetReferenceOptions, 'wrapped'> & { wrapped: false }): T;
 
   /**
    * Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded
    */
-  getReference<T extends AnyEntity<T>>(entityName: EntityName<T>, id: Primary<T>, wrapped?: boolean, convertCustomTypes?: boolean): T | Reference<T>;
+  getReference<T extends AnyEntity<T>>(entityName: EntityName<T>, id: Primary<T>, options?: GetReferenceOptions): T | Reference<T>;
 
   /**
    * Gets a reference to the entity identified by the given type and identifier without actually loading it, if the entity is not yet loaded
    */
-  getReference<T extends AnyEntity<T>>(entityName: EntityName<T>, id: Primary<T>, wrapped = false, convertCustomTypes = false): T | Reference<T> {
+  getReference<T extends AnyEntity<T>>(entityName: EntityName<T>, id: Primary<T>, options: GetReferenceOptions = {}): T | Reference<T> {
+    options.convertCustomTypes ??= false;
     const meta = this.metadata.get(Utils.className(entityName));
 
     if (Utils.isPrimaryKey(id)) {
@@ -555,9 +556,9 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
       id = [id] as Primary<T>;
     }
 
-    const entity = this.getEntityFactory().createReference<T>(entityName, id, { merge: true, convertCustomTypes });
+    const entity = this.getEntityFactory().createReference<T>(entityName, id, { merge: true, ...options });
 
-    if (wrapped) {
+    if (options.wrapped) {
       return Reference.create(entity);
     }
 
@@ -956,6 +957,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
 export interface MergeOptions {
   refresh?: boolean;
   convertCustomTypes?: boolean;
+  schema?: string;
 }
 
 interface ForkOptions {