Add migrations for new permissions structure

[rijkvanzanten]

Scope

What's changed:

• Adds new directus_policies table that's a 1-1 copy of the former directus_roles
• Removes the access control fields from directus_roles
• Updates directus_permissions to:
  • Be attached to policies instead of roles
  • Split up permissions/fields/validation/presets into individual rows based on a new type column

Potential Risks / Drawbacks

• This will most likely not work during zero-downtime deployments, as we're changing the data model for permissions heavily. The old version can't run on the new structure and vice versa.

Review Notes / Questions

• Wondering if there's a cleverer way to replace the previous permissions than a full truncate + insert
• Todo's left are:
  • Heavy testing

[paescuj]

Getting the following error at the moment (same with other DB vendors):

[Error: INSERT INTO "_knex_temp_alter254" SELECT * FROM "directus_permissions"; - SQLITE_CONSTRAINT: NOT NULL constraint failed: _knex_temp_alter254.policy] {
  errno: 19,
  code: 'SQLITE_CONSTRAINT'
}

[rijkvanzanten]

@paescuj Oh no here come the database specific bugs :| _knex_temp_alter254.policy sounds like the temporary copy of the table that knex generates during alter statements

[paescuj]

@paescuj Oh no here come the database specific bugs :| _knex_temp_alter254.policy sounds like the temporary copy of the table that knex generates during alter statements

Oh yeah, how we love those 😅
Fortunately, it doesn't seem to be one of that kind in this case 😌

It happens for permissions assigned to the "Public" role (a.k.a. null role), which conflicts with

directus/api/src/database/migrations/20240328A-permissions-policies.ts

Line 60 in f1b4257

table.uuid('policy').notNullable().alter();

Not sure how we want to handle the "Public" role in the new policy system, so depending on that we either need to transform null values to something else in the migration or need to remove the NOT NULL constraint.

Example permission causing the issue (before ALTER statement):

  {
    id: 30,
    role: null,
    collection: 'test',
    action: 'create',
    permissions: null,
    validation: '{"_and":[{"id":{"_eq":"1"}}]}',
    presets: '{"id":1}',
    fields: 'id',
    policy: null
  }

[rijkvanzanten]

Please use Rebase and merge instead of Squash and merge below ⭐

[DanielBiegler]

After running the migration I'm stuck when trying to open the app. Still gotta figure out why exactly but thought to comment here just in case. Will update when I find more.

api dev: [20:50:24.361] ERROR: select `ip_access` from `directus_roles` where `id` is null limit 1 - SQLITE_ERROR: no such column: ip_access
api dev:     err: {
api dev:       "type": "Error",
api dev:       "message": "select `ip_access` from `directus_roles` where `id` is null limit 1 - SQLITE_ERROR: no such column: ip_access",
api dev:       "stack":
api dev:           Error: select `ip_access` from `directus_roles` where `id` is null limit 1 - SQLITE_ERROR: no such column: ip_access
api dev:       "errno": 1,
api dev:       "code": "SQLITE_ERROR",
api dev:       "extensions": {
api dev:         "stack":
api dev:             Error: select `ip_access` from `directus_roles` where `id` is null limit 1 - SQLITE_ERROR: no such column: ip_access
api dev:       }
api dev:     }

[rijkvanzanten]

@DanielBiegler That's fully expected! This PR is only the migrations for the database, but doesn't update anything else yet in the API or app 🙂

That's also why we're rebasing this into the auditus branch rather than the main branch 👍🏻

Just doing it in smaller PRs on top of auditus to make the review cycle easier as we can review each step of the way in isolation, which should hopefully mean that the final review of the whole project branch is going to go way smoother as every individual piece has already been reviewed 🚀

[DanielBiegler]

That's fully expected!

Oh whoops sry @rijkvanzanten . I assumed it would work because I saw the merge commits from auditus into this branch by you and Pascal and thought those would include some fixes but alrighty then. 🤔

[rijkvanzanten]

All good! Those updates are all to keep auditus up to date with the upstream main.