Support Meilisearch v1.0.0+

[rub1e]

Last week, Meilisearch v1.0.0 was released. As a result of changes to the way they handle primary key inference, anyone following the Medusa docs for using Meilisearch will be unable to use the Meilisearch plugin.

The problem - meilisearch

If multiple attributes ending with id are detected, Meilisearch will ask you to specify the primary key instead of choosing the first found one.

If your documents have multiple keys like id and collection_id, Meilisearch won't index them by default, because it won't choose a primary key. And it won't return an error, because it only ever returns 202 for accepting a task.

Since this applies to the Medusa seed data, a new user following the docs won't have their documents indexed, and they won't get any feedback as to why. (Yes, I spent more than the weekend figuring that out 😞 )

The problem - medusa-plugin-meilisearch

The obvious solution is to just set the primary key at the point of index creation. The problem with that is, as far I can tell, the SearchService's createIndex method isn't actually called

From what I can see, the way it works is that the first call to Meilisearch is from the updateSettings method and that's what creates the index. But since you can't set the primary key this way, the index is always created without a primary key. Which means the subsequent call to addDocuments will fail.

(Temporary) Solution

The fix for this is very easy - I just create the index and set the primary key in updateSettings

async updateSettings(indexName, settings) {
    await this.client_.createIndex(indexName, {
      primaryKey: "id",
    });
    return await this.client_.index(indexName).updateSettings(settings);
  }

This means that the call to createIndex is always made, but Meilisearch just ignores it so it's not the end of the world in the short term. Of course this does mean the primary key is hardcoded; plus there's a race condition as you have to rely on the index being created by the time addDocuments is called; so it's a hack, not really a pushable fix.

A more permanent solution would involve changing the way the SearchService module is used, and calling createIndex first. Then the user could set their own primary key in the config.

(Another thing I'd like to do is be able to set my own transformDocuments function in config)

My partner and I would happily devote some time to this, as we intend to use Meilisearch with Medusa going forward, but I thought it would be best to start this discussion first. I know Medusa is in very active development so perhaps there are already ideas floating around for the future of the search service and plugins. Looking forward to hearing people's thoughts on this. 😃

[rub1e]

I haven't published my fix, but someone on Discord was asking about this so I just popped the files in a repo in case it's useful to anyone

https://github.com/rub1e/medusa-meilisearch-temp-hack

[Charlio99]

Thanks for sharing this @rub1e, indeed, reverting the server to v0.30.5 solves the problem.

[tiltroom]

All it takes is a POST request and v1+ will work just fine.
Did anyone try to read the documentation of Meili?
https://docs.meilisearch.com/learn/core_concepts/primary_key.html#setting-the-primary-key

I think Meili wrote something about this in the changelog too

[Horib]

Could you clarify what you mean?
I seem to have the same problem as everyone else, Redis is running, Meilisearch server going on a specific key and medusa server going. when i call the "localhost:9000/store/products/search" endpoint with body of { "q": "" } I get no hits.

I have looked at the documentation of Meili, and a simple post request does not work

[tiltroom]

Sure thing. You can use cURL or Postman for this and you need to run this:

curl --location 'meili-host/indexes'
--header 'Authorization: Bearer meili-key'
--header 'Content-Type: application/json'
--data '{
"uid": "products",
"primaryKey": "id"
}'

Replace meili-host with MEILISEARCH_HOST from your env and meili-key with MEILISEARCH_API_KEY from your env too.

[olivermrbl]

Thanks @rub1e - highly appreciate the elaborate description of your issues as well as the input on how to resolve them.

I've had a look at Meilisearch v1.0.0, and I believe we can come up with a solution fairly quickly that fixes the issues and meets your additional requirements.

Note, you'll find that the proposed changes are very similar to what you've already mentioned. Therefore, I might repeat some things, but I just want to flesh it out in full to not have the context for the changes live in a different message.

Changes

Primary key inference

Meilisearch has improved their primary key inference mechanism such that it tries to detect the key upon receiving documents for the first time. It does this by searching for a single field that ends with id. Though, because we have multiple properties that end with id, Meilisearch fails to infer the primary key.

Proposal
Change the shape of the index options exposed in the plugin options in medusa-config.js to allow you to specify the primary key used for your documents in Meilisearch.

Important: This will be a breaking change, as we will move the provider-specific index settings to a nested property indexSettings.

In medusa-config.js, you'll be able to do the following:

    settings?: {
      products: {
        indexSettings: {
          searchableAttributes: [ ... ]
        }
        primaryKey: "some_other_id"
      }
    }

The primaryKey will default to id.

The primary key is applied in the method updateSettings. We check to see if the index with the provided primary key already exists. If not, we create it before we add the documents.

Custom document transformer

Currently, users wishing to store documents in a specific shape will not be able to use our Meilisearch plugin as is. It would be a big improvement to be able to store Medusa entities in a shape that fits businesses' exact requirements.

Proposal
Expose a custom document transformer in medusa-config.js overriding the default one provided out-of-the-box.

In medusa-config.js, you'll be able to do the following:

    settings?: {
      products: {
        indexSettings: {
          searchableAttributes: [ ... ]
        }
        transformer: (document: Product) {
          return { id: document.id }
        }
      }
    }

With the above configuration, Meilisearch will only store the id of Medusa products.

Let me know what you think about these proposals @rub1e :)

Also, I took the liberty of creating a draft PR with a first take on the implementation.

[olivermrbl]

We've reviewed the proposal internally and decided to proceed with these changes @rub1e. I will move this discussion to our roadmap and mark it as completed :)

[rub1e]

I'm so sorry I missed the notification for this! We'll be looking at this and all the other amazing v1.8 updates in the coming days, very exciting, well done to you and the team 🚀