feat(sql): use joined strategy as default for SQL drivers (#4958)

This builds on the foundation of #4957, which resolved the biggest
difference of the strategies. As part of the switch, few other smaller
bugs were also fixed, as most of the test suite is now using the joined
strategy too, so it uncovered some bugs that were fixed only with the
select-in.

To keep the old behaviour, you can override the default loading strategy
in your ORM config.

docs/docs/loading-strategies.md

@@ -3,105 +3,123 @@ title: Relationship Loading Strategies
 sidebar_label: Loading Strategies
 ---
 
-> `JOINED` loading strategy is SQL only feature.
+MikroORM supports two loading strategies:
 
-Controls how relationships get loaded when querying. By default, populated relationships are loaded via the `select-in` strategy. This strategy issues one additional `SELECT` statement per relation being loaded.
+- `select-in` which uses separate queries for each relation - it you populate two relations, you will get three queries - one for the root entity, and one for each populated relation.
+- `joined` which uses a single query and joins the relations instead.
 
-The loading strategy can be specified both at mapping time and when loading entities.
+> `joined` strategy is supported **only in SQL drivers** and is the **default** for those since v6.
 
-For example, given the following entities:
+## Configuring the strategy
+
+The loading strategy can be specified both at mapping time and when loading entities, as well as globally in your ORM config.
+
+Given the following entities:
 
 ```ts
-import { Entity, LoadStrategy, OneToMany, ManyToOne } from '@mikro-orm/core';
+import { Entity, LoadStrategy, OneToMany, ManyToOne, PrimaryKey } from '@mikro-orm/core';
 
 @Entity()
 export class Author {
+
+  @PrimaryKey()
+  id!: number;
+
   @OneToMany(() => Book, b => b.author)
   books = new Collection<Book>(this);
+
 }
 
 @Entity()
 export class Book {
+
+  @PrimaryKey()
+  id!: number;
+
   @ManyToOne()
   author: Author;
+
 }
 ```
 
-The following will issue two SQL statements. One to load the author and another to load all the books belonging to that author:
+With the default `joined` strategy, this will issue a single query both the `Author` entity and its `books` relation.
 
 ```ts
-const author = await orm.em.findOne(Author, 1, { populate: ['books'] });
+const author = await orm.em.findOne(Author, 1, {
+  populate: ['books'],
+});
+```
+
+To override the strategy, you can use the `strategy` option:
+
+```ts
+const author = await orm.em.findOne(Author, 1, {
+  populate: ['books'],
+  strategy: 'select-in',
+});
 ```
 
-If we update the `Author.books` mapping to the following:
+This will issue two SQL statements, one to load the author and another to load all the books belonging to that author:
+
+Alternatively, you can control the strategy in your entity definition.
 
 ```ts
 import { Entity, LoadStrategy, OneToMany } from '@mikro-orm/core';
 
 @Entity()
 export class Author {
+
+  // ...
+
   @OneToMany({
     entity: () => Book,
     mappedBy: b => b.author,
-    strategy: LoadStrategy.JOINED,
+    strategy: 'select-in', // force select-in strategy for this relation
   })
   books = new Collection<Book>(this);
-}
-```
-
-The following will issue **one** SQL statement:
 
-```ts
-const author = await orm.em.findOne(Author, 1, { populate: ['books'] });
+}
 ```
 
-You can also specify the load strategy as needed. This will override whatever strategy is declared in the mapping. This also works for nested populates:
-
-```ts
-const author = await orm.em.findOne(Author, 1, {
-  populate: ['books.publisher'],
-  strategy: LoadStrategy.JOINED
-});
-```
+The strategy defined on property level will always take a precedence.
 
 ## Changing the loading strategy globally
 
 You can use `loadStrategy` option in the ORM config:
 
 ```ts
 MikroORM.init({
-  loadStrategy: LoadStrategy.JOINED,
+  // ...
+  populate: ['books'],
+  loadStrategy: 'select-in', // 'joined' is the default for SQL drivers
 });
 ```
 
 This value will be used as the default, specifying the loading strategy on property level has precedence, as well as specifying it in the `FindOptions`.
 
-## Population where condition
-
-> This applies only to SELECT_IN strategy, as JOINED strategy implies the inference.
+## Population `where` condition
 
-In v4, when we used populate hints in `em.find()` and similar methods, the query for our entity would be analysed and parts of it extracted and used for the population. Following example would find all authors that have books with given IDs, and populate their books collection, again using this PK condition, resulting in only such books being in those collections.
+The where condition is by default applied only to the root entity. This can be controlled via `populateWhere` option. It accepts one of `all` (default), `infer` (use same condition as the `where` query) or an explicit filter query.
 
 ```ts
-// this would end up with `Author.books` collections having only books of PK 1, 2, 3
-const a = await em.find(Author, { books: [1, 2, 3] }, { populate: ['books'] });
+await em.find(Author, { ... }, {
+  populate: ['books'],
+  populateWhere: 'infer', // defaults to `all`
+
+  // or specify custom query, will be used via `join on` conditions
+  // populateWhere: { age: { $gte: 18 } },
+});
 ```
 
-Following this example, if we wanted to load all books, we would need a separate `em.populate()` call:
+> `populateWhere` can be also set globally, the default is `all`.
 
-```ts
-const a = await em.find(Author, { books: [1, 2, 3] });
-await em.populate(a, ['books']);
-```
+## Population `order by` clause
 
-This behaviour changed and is now configurable both globally and locally, via `populateWhere` option. Globally we can specify one of `PopulateHint.ALL` and `PopulateHint.INFER`, the former being the default in v5, the latter being the default behaviour in v4. Locally (via `FindOptions`) we can also specify custom where condition that will be passed to `em.populate()` call.
+Similarly to the `populateWhere`, you can also control the `order by` clause used for the populate queries. The default behaviour is to use the same ordering as for the root entity, and you can use `populateOrderBy` option to add a different ordering:
 
 ```ts
 await em.find(Author, { ... }, {
-  // defaults to PopulateHint.ALL in v5
-  populateWhere: PopulateHint.INFER, // revert to v4 behaviour
-
-  // or we can specify custom condition for the population:
-  // populateWhere: { ... },
+  populate: ['books'],
+  populateOrderBy: { books: { publishedAt: 'desc' } },
 });
 ```

packages/core/src/EntityManager.ts

@@ -404,7 +404,7 @@ export class EntityManager<D extends IDatabaseDriver = IDatabaseDriver> {
       for (const hint of (options.populate as unknown as PopulateOptions<Entity>[])) {
         const field = hint.field.split(':')[0] as EntityKey<Entity>;
         const prop = meta.properties[field];
-        const joined = (options.strategy || hint.strategy || prop.strategy || this.config.get('loadStrategy')) === LoadStrategy.JOINED && prop.kind !== ReferenceKind.SCALAR;
+        const joined = (prop.strategy || options.strategy || hint.strategy || this.config.get('loadStrategy')) === LoadStrategy.JOINED && prop.kind !== ReferenceKind.SCALAR;
 
         if (!joined) {
           continue;

packages/core/src/entity/ArrayCollection.ts

@@ -39,7 +39,7 @@ export class ArrayCollection<T extends object, O extends object> {
     const meta = this.property.targetMeta!;
     const args = [...meta.toJsonParams.map(() => undefined)];
 
-    return this.getItems().map(item => wrap(item as TT).toJSON(...args));
+    return this.map(item => wrap(item as TT).toJSON(...args));
   }
 
   toJSON(): EntityDTO<T>[] {

packages/core/src/entity/EntityFactory.ts

@@ -210,7 +210,6 @@ export class EntityFactory {
     }
 
     const pks = Utils.getOrderedPrimaryKeys<T>(id, meta);
-
     const exists = this.unitOfWork.getById<T>(entityName, pks as Primary<T>, schema);
 
     if (exists) {
@@ -305,7 +304,7 @@ export class EntityFactory {
       return this.unitOfWork.getById<T>(meta.name!, data[meta.primaryKeys[0]] as Primary<T>, schema);
     }
 
-    if (meta.primaryKeys.some(pk => data[pk] == null)) {
+    if (!Array.isArray(data) && meta.primaryKeys.some(pk => data[pk] == null)) {
       return undefined;
     }
 

packages/core/src/hydration/ObjectHydrator.ts

@@ -137,6 +137,7 @@ export class ObjectHydrator extends Hydrator {
     const hydrateToOne = (prop: EntityProperty, dataKey: string, entityKey: string) => {
       const ret: string[] = [];
 
+      const method = type === 'reference' ? 'createReference' : 'create';
       const nullVal = this.config.get('forceUndefined') ? 'undefined' : 'null';
       ret.push(`  if (data${dataKey} === null) {\n    entity${entityKey} = ${nullVal};`);
       ret.push(`  } else if (typeof data${dataKey} !== 'undefined') {`);
@@ -151,9 +152,9 @@ export class ObjectHydrator extends Hydrator {
       ret.push(`    } else if (data${dataKey} && typeof data${dataKey} === 'object') {`);
 
       if (prop.ref) {
-        ret.push(`      entity${entityKey} = Reference.create(factory.create('${prop.type}', data${dataKey}, { initialized: true, merge: true, newEntity, convertCustomTypes, schema }));`);
+        ret.push(`      entity${entityKey} = Reference.create(factory.${method}('${prop.type}', data${dataKey}, { initialized: true, merge: true, newEntity, convertCustomTypes, schema }));`);
       } else {
-        ret.push(`      entity${entityKey} = factory.create('${prop.type}', data${dataKey}, { initialized: true, merge: true, newEntity, convertCustomTypes, schema });`);
+        ret.push(`      entity${entityKey} = factory.${method}('${prop.type}', data${dataKey}, { initialized: true, merge: true, newEntity, convertCustomTypes, schema });`);
       }
 
       ret.push(`    }`);

packages/core/src/utils/Configuration.ts

@@ -70,7 +70,7 @@ export class Configuration<D extends IDatabaseDriver = IDatabaseDriver> {
     baseDir: process.cwd(),
     hydrator: ObjectHydrator,
     flushMode: FlushMode.AUTO,
-    loadStrategy: LoadStrategy.SELECT_IN,
+    loadStrategy: LoadStrategy.JOINED,
     dataloader: Dataloader.OFF,
     populateWhere: PopulateHint.ALL,
     connect: true,

packages/core/src/utils/Utils.ts

@@ -632,6 +632,28 @@ export class Utils {
     return cond;
   }
 
+  /**
+   * Maps nested FKs from `[1, 2, 3]` to `[1, [2, 3]]`.
+   */
+  static mapFlatCompositePrimaryKey(fk: Primary<any>[], prop: EntityProperty, fieldNames = prop.fieldNames, idx = 0): Primary<any> | Primary<any>[] {
+    if (!prop.targetMeta) {
+      return fk[idx++];
+    }
+
+    const parts: Primary<any>[] = [];
+
+    for (const pk of prop.targetMeta.getPrimaryProps()) {
+      parts.push(this.mapFlatCompositePrimaryKey(fk, pk, fieldNames, idx));
+      idx += pk.fieldNames.length;
+    }
+
+    if (parts.length < 2) {
+      return parts[0];
+    }
+
+    return parts;
+  }
+
   static getPrimaryKeyCondFromArray<T extends object>(pks: Primary<T>[], meta: EntityMetadata<T>): Record<string, Primary<T>> {
     return meta.getPrimaryProps().reduce((o, pk, idx) => {
       if (Array.isArray(pks[idx]) && pk.targetMeta) {