ORM attempts to update multiple, unwanted properties on a populated find or on a populated remove

[Jcalcaldev]

I'll do my best to explain what's going on.
For a full reproduction, see below.

On a freshly seeded database, I'm running only this code:

const row = await em.findOne(Cosmetic, { itemType: ItemType.Cosmetic }, { populate: ['*'] });

console.log('row: ', row);

This finds the row, logs it correctly, and THEN throws an error:

 TypeError: Cannot read properties of undefined (reading 'fieldNames')

      at ../../node_modules/@mikro-orm/knex/AbstractSqlDriver.js:521:18
          at Set.forEach (<anonymous>)
      at SqliteDriver.nativeUpdateMany (../../node_modules/@mikro-orm/knex/AbstractSqlDriver.js:519:14)
      at ChangeSetPersister.persistManagedEntitiesBatch (../../node_modules/@mikro-orm/core/unit-of-work/ChangeSetPersister.js:186:39)
      at async ChangeSetPersister.persistManagedEntities (../../node_modules/@mikro-orm/core/unit-of-work/ChangeSetPersister.js:159:13)

Upon further digging, I went to AbstractSqlDriver and logged the keys on the nativeUpdateMany function. This is what I got:

 KEYS:  Set(3) { 'type', 'updatedAt', 'itemType' }

So I see that the ORM is trying to update type and itemType, which are two enum properties of my tables which should never change, they are explicit discriminator columns from single table inheritance entities.

This only happens, as far as I know, if I populate with ['*']. It does not happen with ALL entities in my repo, but with around half of them.

If I log the changeSets, this is what I get:

Change Sets:  [
      ChangeSet<Character> {
        entity: Character {
          user: [User],
          _boundToUser: null,
          _boundToSaveSlot: null,
          _boundToCharacter: null,
          type: 'Character',
          createdAt: 2024-02-27T16:35:50.245Z,
          updatedAt: 2024-02-27T16:35:50.501Z,
          deletedAt: null,
          firstName: 'firstName',
          lastName: null,
          affinity: 1,
          saveSlot: [SaveSlot],
          id: '04eb2408-56a1-4936-981d-a3c37746ffcb',
          itemSlots: [Collection<ItemSlot>],
          characterBoundAssets: [Collection<Asset>]
        },
        type: 'update',
        payload: { type: 'Character', updatedAt: 1709051750501 },
        primaryKey: undefined,
        name: 'Character',
        rootName: 'Asset',
        collection: 'asset',
        schema: undefined,
        originalEntity: {
          id: '04eb2408-56a1-4936-981d-a3c37746ffcb',
          user: '40f85adf-2df2-4670-a99f-980f45045ae4',
          createdAt: 2024-02-27T16:35:50.245Z,
          updatedAt: 2024-02-27T16:35:50.245Z,
          firstName: 'firstName',
          affinity: 1
        }
      },
      ChangeSet<HouseDecoration> {
        entity: HouseDecoration {
          user: [User],
          _boundToUser: null,
          _boundToSaveSlot: null,
          _boundToCharacter: null,
          type: 'Item',
          createdAt: 2024-02-27T16:35:50.247Z,
          updatedAt: 2024-02-27T16:35:50.502Z,
          deletedAt: null,
          kind: 'HOUSE_DECORATION.random',
          itemType: 'HouseDecoration',
          houseDecorationSlot: [HouseDecorationSlot],
          id: '4c03e5d3-0f3b-47c9-95f1-14b572ad2cdb'
        },
        type: 'update',
        payload: { itemType: 'HouseDecoration', updatedAt: 1709051750502 },
        primaryKey: undefined,
        name: 'HouseDecoration',
        rootName: 'Asset',
        collection: 'asset',
        schema: undefined,
        originalEntity: {
          id: '4c03e5d3-0f3b-47c9-95f1-14b572ad2cdb',
          user: '40f85adf-2df2-4670-a99f-980f45045ae4',
          type: 'Item',
          createdAt: 2024-02-27T16:35:50.247Z,
          updatedAt: 2024-02-27T16:35:50.247Z,
          kind: 'HOUSE_DECORATION.random'
        }
      },
      ChangeSet<HouseFloorSkin> {
        entity: HouseFloorSkin {
          user: [User],
          _boundToUser: null,
          _boundToSaveSlot: null,
          _boundToCharacter: null,
          type: 'Item',
          createdAt: 2024-02-27T16:35:50.247Z,
          updatedAt: 2024-02-27T16:35:50.503Z,
          deletedAt: null,
          kind: 'HOUSE_FLOOR_SKIN.random',
          itemType: 'HouseFloorSkin',
          floorSkinSlot: [HouseSkinSlot],
          id: 'f6f08aaa-68dd-4e37-98f0-3656f9394868'
        },
        type: 'update',
        payload: { itemType: 'HouseFloorSkin', updatedAt: 1709051750503 },
        primaryKey: undefined,
        name: 'HouseFloorSkin',
        rootName: 'Asset',
        collection: 'asset',
        schema: undefined,
        originalEntity: {
          id: 'f6f08aaa-68dd-4e37-98f0-3656f9394868',
          user: '40f85adf-2df2-4670-a99f-980f45045ae4',
          type: 'Item',
          createdAt: 2024-02-27T16:35:50.247Z,
          updatedAt: 2024-02-27T16:35:50.247Z,
          kind: 'HOUSE_FLOOR_SKIN.random'
        }
      },
      ChangeSet<HouseWallSkin> {
        entity: HouseWallSkin {
          user: [User],
          _boundToUser: null,
          _boundToSaveSlot: null,
          _boundToCharacter: null,
          type: 'Item',
          createdAt: 2024-02-27T16:35:50.247Z,
          updatedAt: 2024-02-27T16:35:50.503Z,
          deletedAt: null,
          kind: 'HOUSE_WALL_SKIN.random',
          itemType: 'HouseWallSkin',
          wallSkinSlot: [HouseSkinSlot],
          id: '507c3678-166d-42c8-91b7-9c7d1a3eed34'
        },
        type: 'update',
        payload: { itemType: 'HouseWallSkin', updatedAt: 1709051750503 },
        primaryKey: undefined,
        name: 'HouseWallSkin',
        rootName: 'Asset',
        collection: 'asset',
        schema: undefined,
        originalEntity: {
          id: '507c3678-166d-42c8-91b7-9c7d1a3eed34',
          user: '40f85adf-2df2-4670-a99f-980f45045ae4',
          type: 'Item',
          createdAt: 2024-02-27T16:35:50.247Z,
          updatedAt: 2024-02-27T16:35:50.247Z,
          kind: 'HOUSE_WALL_SKIN.random'
        }
      }
    ]

I think this will be easier to understand in this minimal reproduction test:
https://gist.github.com/Jcalcaldev/7d5937e2ee56e5f54d717eb4fb644019

Keep in mind, in the test, the error is not fired upon a find, but upon a removal. I found this is another way to reproduce the same error by coincidence.

I also noticed that if I remove the em.clear() line on the beforeEach block, the test goes through as intended.

The entities' structure is a bit bloated, but I cleaned it the best I could, while preserving everything that I think might be important.

[B4nan]

That repro is really huge, first step should be to simplify it, now you have 700 loc, get to 100 and things will surely make more sense (at least to me). The way it is now is rather undebuggable :)

[Jcalcaldev]

https://miro.com/app/board/uXjVNnLtp6I=/?share_link_id=395779795428

Here's a diagram for easier visualization. I'm trying to reduce it but the test passes every time I take something away.

Also here's another gist, mildly unbloated.
https://gist.github.com/Jcalcaldev/b683c3f5765a66b1b9deb57a4be6cbee

edit: On this gist, removing uppercase letters from floorSkin and floorSkinSlot no longer makes the test pass.

[B4nan]

What should this be reproducing? I see a failed insert query due to SQLITE_CONSTRAINT: NOT NULL constraint failed: asset.type which comes from the before hooks, not from the test itself.

[B4nan]

A few notes about your repro:

• you have some class constructors that doesnt do anything (like the one in House entity) but you try to put values in there. Those are of course ignored.
• no point in adding unique: true to 1:1 decorator options - that's the point of it, 1:1 is a m:1 with unique: true
• specifying inversedBy on the m:1 and 1:1 properties in optional, it has no effect, only mappedBy is required on the inverse sides, owning side does not have to point to the inverse

And the most important one, which actually might be significant - you're trying to have two levels of STI. Your Asset entity is the root, but then you extend it in the Item entity and try to redefine the discriminator. This is surely not tested, as it is not expected usage, I would even call this unsupported, as the root entity is important and you are trying to have two at the same time.

[B4nan]

What do you expect from the schema, do you want all the entities that extend from Asset to use the same db table, including the ones that extend Item entity? As that could be done, but only with a single discriminator for all the entities. Or alternatively, if you want to have a separate item table for those extending Item, just drop the extends clause from it (and the discr value). A combination of those two is not possible, you can't have multiple discriminators, on multiple levels (speaking of the current implementation, I guess it could be supported in the future).

[Jcalcaldev]

Ok, so after hitting my head against the wall for a while I caved in and removed the double inheritance. It seems to be working now. I assume since it's not supported it threw the ORM nuts.

Regarding you getting SQLITE_CONSTRAINT: NOT NULL constraint failed: asset.type that's weird, I'm getting the fieldNames error i posted at the beggining.

Just in case you feel like digging in, version is 6.1.5, macOS Sonoma 14.2, typescript 5.2.2.

I have a feeling that the issue manifests itself as issues with the explicit discriminator column enums. The ORM tried to update them in the changesets, or sometimes errors out or not depending if there are uppercases. Maybe related to how I'm setting them on the children's constructors?

Thanks for the notes about the repro, they're very helpful.

If we can get multiple levels of STI in the future that would be amazing.

Anyways, I think it's safe to close for now. Thanks for the help!

If for some reason you decide to dig in and figure out that the problem was actually related to 2-level STI or something else, please let me know.

Edit: Just in case, using 2 levels of STI seemed to work correctly at least on the database level.