Error updating relation during schema apply

[dstoyanoff]

Preflight Checklist

• I have completed all Troubleshooting Steps.
• I'm on the latest version of Directus.
• There's no other issue that already describes my problem.

Describe the Bug

Hey guys,
I have found a weird error when trying to apply a snapshot. What's worth mentioning is that I am working on top of an existing database that already has some migrations applied in it. I have created a snapshot out of it and now trying to apply it back.

The error that I was getting is this one:

/Users/danielstoyanoff/dev/directus/api/src/services/relations.ts:449
                const m2oFieldDBType = this.schema.collections[relation.collection!].fields[relation.field!].dbType;
                                                                       ^
TypeError: Cannot read properties of undefined (reading 'fields')
    at RelationsService.alterType (/Users/danielstoyanoff/dev/directus/api/src/services/relations.ts:449:72)
    at TableBuilder._fn (/Users/danielstoyanoff/dev/directus/api/src/services/relations.ts:243:11)
    at TableBuilder.toSQL (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/schema/tablebuilder.js:44:14)
    at SchemaCompiler_PG.alterTable (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/schema/compiler.js:101:25)
    at SchemaCompiler_PG.toSQL (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/schema/compiler.js:63:26)
    at SchemaBuilder.toSQL (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/schema/builder.js:35:45)
    at ensureConnectionCallback (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/execution/internal/ensure-connection-callback.js:4:30)
    at Runner.ensureConnection (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/execution/runner.js:272:20)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Runner.run (/Users/danielstoyanoff/dev/directus/node_modules/knex/lib/execution/runner.js:30:19)

Unfortunately, reproduction is hard to achieve, but here is some debug data I was able to get:
Since the issue happens in the alterType function, I've checked what's passed as a relation there and it seems to be an incorrect one, as I am only getting a partial data, which causes the exception:

{
  schema: {
    constraint_name: 'footer_section_items_translations_footer_section___kxled_foreig'
  }
}

Going a step backwards, I can see that the relation diff looks like this:

  {
    collection: 'footer_section_items_translations',
    field: 'footer_section_items_id',
    related_collection: 'footer_section_items',
    diff: [{
       kind: 'E',
       path: [ 'schema', 'constraint_name' ],
       lhs: 'footer_section_items_translations_footer_section___griuu_foreig',
       rhs: 'footer_section_items_translations_footer_section___kxled_foreig'
    }]
  }

Here are some details of the schema.
Relation inside the snapshot:

    {
      "collection": "footer_section_items_translations",
      "field": "footer_section_items_id",
      "related_collection": "footer_section_items",
      "schema": {
        "table": "footer_section_items_translations",
        "column": "footer_section_items_id",
        "foreign_key_table": "footer_section_items",
        "foreign_key_column": "id",
        "foreign_key_schema": "public",
        "constraint_name": "footer_section_items_translations_footer_section___kxled_foreig",
        "on_update": "NO ACTION",
        "on_delete": "SET NULL"
      },
      "meta": {
        "many_collection": "footer_section_items_translations",
        "many_field": "footer_section_items_id",
        "one_collection": "footer_section_items",
        "one_field": "translations",
        "one_collection_field": null,
        "one_allowed_collections": null,
        "junction_field": "language_code",
        "sort_field": null,
        "one_deselect_action": "nullify"
      }
    },

Relation record in the database:

[
  {
    "many_collection": "footer_section_items_translations",
    "many_field": "footer_section_items_id",
    "one_collection": "footer_section_items",
    "one_field": "translations",
    "one_collection_field": null,
    "one_allowed_collections": null,
    "junction_field": "languages_code",
    "sort_field": null,
    "one_deselect_action": "nullify"
  }
]

I was able to fix this by changing the reduce's default value to the relation itself inside of apply-snapshot, but not sure if that's the proper fix.

Btw, it's worth adding more trace logs in the snapshots process as it takes forever to investigate.

Let me know if I can provide other information.

[rijkvanzanten]

Btw, it's worth adding more trace logs in the snapshots process as it takes forever to investigate.

LOG_LEVEL="trace" should already show all the SQL queries that are performed. I'll gladly take more suggestions for what other things would be useful to log in there 👍🏻

[rijkvanzanten]

It apparently had some trouble reading the fields from that "footer_section_items_translations" in the schema information 🤔 Does that collection still exist in the database? Could you try manually clearing the schema cache by sending a POST to /utils/cache/clear (using an admin user) to check if it might've been the schema cache being outdated?

[dstoyanoff]

Trace logs the queries, but doesn't log what causes the queries. In this case the error happened while preparing the query. I think that it would be useful to log what record/diff is being edited (that's why it's a trace log after all, it's ok to be a bit noisy).

I've tried to clear the cache, but it doesn't help. And I don't think the cache has effect here, since the snapshot is applied immediately after creating the docker container, so there shouldn't be any cache written at all yet.

Let me give you more details on the change I made to patch it locally, so you can understand it better. This is inside apply-snapshot.ts:

The problem here is that this reduce statement only adds the schema diff and doesn't add the rest of the relation information (collection, field, related_collection). This is immediately passed to the alterType function in RelationsService, which throws as expected, since relation.collection is undefined.

Hope it's clear now and can guide you to the problem.

Note that this only happens on a dirty database, as the issue is in the update operation. In case of creation, there is an if statement in RelationsService.createOne that guards from calling alterType and the issue is not happening there:

While on the update operation, this check is done on the existingRelation, which has the related_collection field, unlike for the incoming relation

[rijkvanzanten]

Ahhhhhhh yes yes that does make sense. Thanks!

[dstoyanoff]

Lovely! I think this is the last bug that prevents me from going on staging with the snapshot feature, can't wait!