Integration Guide for AWS

[wperron]

Over at @denoland, we want to start using Meili to power the search results for the module registry at https://deno.land/x . The backend is mostly built on AWS, and we would like to keep using AWS to host a Meili server.

My first thought was to use the official Meili Docker image and use aws Fargate with an EFS mount, though we also thought about provisioning an EC2 instance with an EBS volume.

Do you have any material or suggestions on the most efficient and cost effective way to run Meilisearch on AWS?

CC: @lucacasonato

[tpayet]

Hey @wperron
That is great news! Meili server works like a database meaning that it needs persistent storage. Either solutions would work fine, just keep in mind that it might need as much RAM as persistent storage so I would suggest using the cheapest solution when it comes to memory. You can read more about storage here: https://docs.meilisearch.com/resources/about_storage.html#about-storage

We also have a tutorial to setup MeiliSearch behind NGINX if you go with the EC2 + EBS solution : https://docs.meilisearch.com/running-production/

let me know if you have any other questions, would be glad to guide you through your journey with MeiliSearch!

[wperron]

Hi @tpayet thanks for the response! thanks for the link about the storage and memory considerations, this is very useful information indeed.

I guess what I'm also curious about is how well does Meili handle multi-instance setup? are there any precautions or special settings we need to be aware of? or is the suggestion to simply stick to a single instance setup?

[tpayet]

You cannot have multiple instances of MeiliSearch running on the same state aka the data.ms folder where all MeiliSearch data are stored.
However you are free to have multiple completely separate instances of MeiliSearch running under the same load balancer as long as you can ensure that all instances have the same state if you really want to have redundancy. You'll be glad to know that we are working on high availability feature with MeiliSearch that will allow you to run clusters of MeiliSearch

Edit: We just released snapshots that you can set up to write on your persistent storage so you will be able to start up any instance of MeiliSearch in a few seconds without the hassle of re-indexing all your data.

[wperron]

For more info, my current proposal uses an Application Load Balancer to front ECS tasks. The task definition mounts /data.ms to an EFS volume (essentially an NFS mount). Will that be problematic with more than one instance?

[tpayet]

As long as you are able to ensure every instances of MeiliSearch have the same state, there won't be any problem!

[wperron]

@tpayet do you have an ETA for the HA clustered mode?

[tpayet]

Hey! sorry for the delay, I did not have my computer last week. We do not have a specific ETA, but I would say that we will have HA in the 2-3 months.

[wperron]

Just for information: we're likely going to go with a simple EC2 deployment instead of Fargate (denoland/deno_registry2#146) that's loosely based on the Digital Ocean scripts you already have.

I used Packer to build a custom AMI; it would be nice to have an official public AMI available, I think it would make it easier for people to adopt meili 🙂