Syntax to specify indexes and unique constraints (ie.isIndexed, isUnique) across multiple fields

[molomby]

Most field types support the isIndexed and isUnique config options. Behind the scenes these options creates an index (with or without an additional unique constraint) on the field at the database level. There's currently no way (within Keystone) to configure indexes that include more than one field; app developers are left to create the required indexes manually.

Any syntax developed here should allow for:

• Multiple indexes per list
• Multiple fields per index, in a specific order (the order that fields are referenced in an index dictates the index behaviour so must be explicit)
• Multiple indexes per field (It's completely possible for a system to require multiple indexes for the exact same set of fields covering, for example, different directions)
• Direction per field, per index (ie. asc or desc)
• Possibly a key or handle (from which a constraint name can be derived)

Partial indexes and some basic full-text stuff would also be super great.

Syntax

Maybe something like...

keystone.createList('Article', {
   fields: {
      slug: { type: Slug, from: 'title' },
      author: { type: Relationship, ref: 'User' },
      site: { type: Relationship, ref: 'Site' },
      status: {
         type: Select,
         options: ['draft', 'published', 'archived'],
         defaultValue: 'draft',
      },
      publishedAt: { type: DateTimeUtc },
      articelOfTheDayOn: { type: Date },
      title: { type: Text },
      subtitle: { type: Text },
      content: { type: Content, /* ... */ },
      tags: { type: Text },
   },

   // Some ideas/example syntaxes for multi-field indexes
   indexes: {

      // Article slugs must be unique to a site
      // This is often also how an article is retrieved
      siteLookup: {
         isUnique: true,
         fields: ['site', 'slug'],
      },

      // We often want to retrieve the recent articles published in a site
      // Specifying the publishedAt decending order here makes our query faster
      recentlyPublished: {
         isUnique: false,
         fields: [
            'site',
            'status',
            { field: 'publishedAt', direction: 'desc' }
         ],
      },

      // You could specify single-field indexes here too
      // An alternative to using the field-level `isIndexed` option
      slug: {
         fields: ['slug'],
      },

      // Maybe we want some shorthand syntaxes too?
      authorList: 'author, title asc',

      // Supporting partial indexes would be awesome
      // Eg. the articelOfTheDayOn field stores the day on which an article was highlighted on a site
      // Most articles have a null value (because they were never an "article of the day")
      // For each site, only one article should ever be "article of the day" for any day value
      // (Without something like this Mongo has problems because `undefined === undefined`)
      articelOfTheDay: {
         isUnique: true,
         where: {
            articelOfTheDayOn_is_not: null,
         }
         fields: [
            { field: 'site' },
            { field: 'articelOfTheDayOn' }
         ]
      },

      // Not sure if we want to get into full-text stuff here..?
      // See https://github.com/keystonejs/keystone/issues/319
      content: {
         type: 'full-text',
         fields: [
            { field: 'title', weight: 10 },
            { field: 'subtitle', weight: 8 },
            { field: 'content', weight: 1 },
            { field: 'tags', weight: 5 },
         ],

         // Maybe this also gives us a way to configure the 'list search'..?
         // See https://github.com/keystonejs/keystone/issues/343
         isSearchFilter: true,
      },
   }
});

This still only exposes a small subset of the indexing options afforded by most DBs. Anything requiring non-default collations, other methods (eg. hash), fill factor, etc. would require manual creation.

Upserts

In addition to being generally handy, knowing about multi-field indexes might allow us to (better) support upsert operations. There are some notes on this in the issue #182.

Implementation Notes

Currently, the isIndexed and isUnique options are handled by individual field types. From one perspective, this makes a lot of sense -- only fields types know enough about what they store and how they store it to make decisions about what "being indexed" means, and some field types (eg. Checkbox) don't support them at all. Once you get into multi-field indexes though, some of the responsibility needs to picked up by the database adapter though. I'm not clear on what kind of refactoring this would require.

Related

This stuff is probably related to: #319, #343, #1654 and #182.

[VinayaSathyanarayana]

I am supportive of multi-field indexes

[stale]

It looks like there hasn't been any activity here in over 6 months. Sorry about that! We've flagged this issue for special attention. It wil be manually reviewed by maintainers, not automatically closed. If you have any additional information please leave us a comment. It really helps! Thank you for you contribution. :)

[gautamsi]

anyone looking at this there is possibility to have multi field indexes for mongodb. see this #3025

[stale]

It looks like there hasn't been any activity here in over 6 months. Sorry about that! We've flagged this issue for special attention. It wil be manually reviewed by maintainers, not automatically closed. If you have any additional information please leave us a comment. It really helps! Thank you for you contribution. :)

[PranavBhatia]

Are multi-field indexes available now?
I have a pivot table for a many-to-many relationship, however, I want to make a UNIQUE INDEX on the referenced id's
Can I do that with KeystoneJS?

[g012]

This has become critical for me. I have a many-many relationship, and I wanted to add a 'refcount' column to it. So as suggested in prisma doc, I made the table explicit (but in Keystone then).

I then needed to use upsert : if create fails, I need to increase 'refcount'. I can do all that in prisma, as long as I have a primary key @id[a, b]. But since I can't have this or a unique key on 2 columns, I can't use Prisma's upsert, because it works only on unique field selectors.

Therefore, I made this in raw SQL, with Postgres "ON CONFLICT ...", after adding the unique key on [a, b] manually. But now, each time I start Keystone, Prisma fails to start because I have this unique key in the table but not in Prisma's schema. So I can't auto-restart the server, etc.

I'm only left with doing manual migrations so Prisma doesn't delete my table, and I have to remove every reference to this table from Keystone and lose the GraphQL query ability.

It is really CRITICAL for me to have this feature, multi-columns primary keys and unique indices. It is already well supported by Prisma.

Thanks.