Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maximum call stack size exceeded - ArrayType + TsMorphMetadataProvider + findAndCount - clone #3720

Closed
ml1nk opened this issue Nov 10, 2022 · 6 comments
Labels
bug Something isn't working

Comments

@ml1nk
Copy link
Contributor

ml1nk commented Nov 10, 2022

Describe the bug
When using findAndCount some properties of qb are cloned:

packages/knex/src/query/QueryBuilder.ts

    // clone array/object properties
    const properties = [
      'flags', '_populate', '_populateWhere', '_populateMap', '_joins', '_joinedProps', '_aliases', '_cond', '_data', '_orderBy',
      '_schema', '_indexHint', '_cache', 'subQueries', 'lockMode', 'lockTables',
    ];
    properties.forEach(prop => (qb as any)[prop] = Utils.copy(this[prop]));

Inside some of these properties is metadata (_aliases.c0.metadata for example) which also gets cloned (intentional?). In some cases it becomes a cycle until the stack size is exceeded. I tried to figure out when it occurs:

  • TsMorphMetadataProvider is used
  • ArrayType (any other customType?) is used by the entity
  • The cache is disabled, invalid or deleted

Even if you activate the cache and restart orm by closing and init it again, it stils fails to clone metadata/customType.

Stack trace

 FAIL  tests/issues/GHXXXX.test.ts (164.932 s)
  GHXXXX
    ✕ call stack size exceeded (160270 ms)

  ● GHXXXX › call stack size exceeded

    DriverException: Maximum call stack size exceeded

       6 |   /* istanbul ignore next */
       7 |   convertException(exception: Error & Dictionary): DriverException {
    >  8 |     return new DriverException(exception);
         |            ^
       9 |   }
      10 |
      11 | }

      at PostgreSqlExceptionConverter.convertException (packages/core/src/platforms/ExceptionConverter.ts:8:12)
      at PostgreSqlExceptionConverter.convertException (packages/postgresql/src/PostgreSqlExceptionConverter.ts:48:18)
      at PostgreSqlDriver.convertException (packages/core/src/drivers/DatabaseDriver.ts:262:50)
      at packages/core/src/drivers/DatabaseDriver.ts:267:18
      previous RangeError: Maximum call stack size exceeded
      at _clone (packages/core/src/utils/clone.ts:13:18)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at _clone (packages/core/src/utils/clone.ts:104:27)
      at packages/core/src/utils/clone.ts:81:28
          at Map.forEach (<anonymous>)
      at _clone (packages/core/src/utils/clone.ts:79:14)
      at _clone (packages/core/src/utils/clone.ts:104:27)

To Reproduce

import { ArrayType, Entity, MikroORM, PrimaryKey, Property } from '@mikro-orm/core';
import type { PostgreSqlDriver } from '@mikro-orm/postgresql';
import { TsMorphMetadataProvider } from '@mikro-orm/reflection';


@Entity()
export class A {

  @PrimaryKey()
  id!: number;

  @Property({ type: ArrayType })
  types!: string[];

}

describe.only('GHXXXX', () => {

  let orm: MikroORM<PostgreSqlDriver>;

  beforeAll(async () => {
    orm = await MikroORM.init({
      cache: {
        enabled: false,
      },
      entities: [A],
      dbName: 'my-db-name',
      metadataProvider: TsMorphMetadataProvider,
      type: 'postgresql',
    });

    await orm.schema.refreshDatabase();
  });

  afterAll(() => orm.close(true));

  test('call stack size exceeded', async () => {

      await orm.em.findAndCount(A, 5);

  });
});

Additional context
Not from the test above but our productive code (TreeNodeEntity contains an ArrayType)

_aliases.c0.metadata.indexes.node.targetMeta.properties.types.customType.platform.config.cache...

image
image

Versions

Dependency Version
mikro-orm 5.5.3
pg 8.8.0
@B4nan B4nan added the bug Something isn't working label Nov 10, 2022
@B4nan B4nan closed this as completed in c3b4c20 Nov 10, 2022
@ml1nk
Copy link
Contributor Author

ml1nk commented Nov 10, 2022

I tried your change but there is still a call stack size exceeded when _joins is used in the same constellation.

On our codebase:
_joins.prop.name.targetMeta

Repro:

import { ArrayType, Entity, MikroORM, OneToOne, PrimaryKey, Property, Ref } from '@mikro-orm/core';
import type { PostgreSqlDriver } from '@mikro-orm/postgresql';
import { TsMorphMetadataProvider } from '@mikro-orm/reflection';


@Entity()
export class A {

  @PrimaryKey()
  id!: number;

  @Property({ type: ArrayType })
  types!: string[];

  @OneToOne()
  other!: Ref<B>;

}

@Entity()
export class B {

  @PrimaryKey()
  id!: number;

  @OneToOne({ mappedBy: 'other' })
  other!: Ref<A>;

}

describe.only('GHXXXX', () => {

  let orm: MikroORM<PostgreSqlDriver>;

  beforeAll(async () => {
    orm = await MikroORM.init({
      cache: {
        enabled: false,
      },
      entities: [A, B],
      dbName: 'my-db-name',
      metadataProvider: TsMorphMetadataProvider,
      type: 'postgresql',
    });

    await orm.schema.refreshDatabase();
  });

  afterAll(() => orm.close(true));

  test('no stackoverflow should occur', async () => {
      await orm.em.createQueryBuilder(A).leftJoin('other', 'other').getCount();
  });
});

@B4nan
Copy link
Member

B4nan commented Nov 10, 2022

Oh right, I know what caused it, the platform on the custom type, otherwise, it was fine to clone the metadata instances.

@B4nan B4nan reopened this Nov 10, 2022
@ml1nk
Copy link
Contributor Author

ml1nk commented Nov 10, 2022

It was fine to clone the metadata, but ist it needed? Even without cycle the metadata should be relativ constant and is rather large for a deep clone or am i missing something?

@B4nan
Copy link
Member

B4nan commented Nov 10, 2022

Yeah, but that's much more complex refactoring, there are potentially many places like this. Feel free to send a PR, I will go with the simple way 🤷

edit: But maybe there is easy way to go about this, as the clone implementation is now inlined so we can tweak it a bit to just pass the entity metadata instance.

@ml1nk
Copy link
Contributor Author

ml1nk commented Nov 10, 2022

That's more than enough for me, i lost hours here. I will try to create one, if i ever see it in the perf-tab.

@B4nan B4nan closed this as completed in 9e05104 Nov 10, 2022
@ml1nk
Copy link
Contributor Author

ml1nk commented Nov 11, 2022

It's working fine now. Thank you very much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants