Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meilisearch deployment on Fargate #122

Closed
wants to merge 7 commits into from

Conversation

wperron
Copy link
Contributor

@wperron wperron commented Sep 10, 2020

This PR introduces the necessary AWS resources to deploy Meilisearch on Fargate. I personally like Fargate because while the straight compute-hour cost is higher that EC2, it removes a lot of complexity from the management side of things; logging and metrics monitoring is baked-in with Container Insights, the ECS service takes care of restarting failed instances automatically without needing an Autoscaling Group, using a Docker image means we don't have to deal with managing server configurations or ssh into a server. I'm not religiously attached to Fargate, we can discuss the pros/cons and I always move to a more traditional EC2 deployment if need be.

Here's a rough price estimate:

  • Fargate: $21.62 with a single instance at 0.5vcpu, 2Gb memory
  • EFS: < $0.30 (less than 1Gb total)
  • ALB: $16.74/mo flat rate + LCU-hours depending on new http connections per hour

Note that the Load Balancer isn't stricly necessary, we could also use a simple dns-based round-robin and get the job done just fine; ALB simply makes it easier to integrate with Security Groups and AWS Certificate Manager.

Checklist:

  • perform some tests to make sure the EFS volume mount works as expected
  • perform some tests with more than one instance (load & resiliency)
  • perform tests with meili version upgrades
  • configure Meilisearch master key to control access
  • tweak LB to only allow external traffic to the /search endpoint
  • adjust resource allocation based on the amount of data we estimate we'll need
  • get some metrics on the current usage of the search endpoint to estimate the LB costs
  • Implement a backup strategy

Fixes #69

@wperron
Copy link
Contributor Author

wperron commented Sep 10, 2020

The EFS volume works great; dropping and recreating a task attaches to the volume. Also interesting point; dropping the ecs service actually waits for Meili to empty its internal queue before terminating cleanly, I'll try this out by dropping a task directly instead of the overarching service just to make sure.

@wperron
Copy link
Contributor Author

wperron commented Sep 14, 2020

Running with 3 instances, I get some weird artifacts. I think it's related to how Meili updates its internal state, but I would create a "modules" index and then curl /indexes/modules , and depending on the instance hit, I would get a mix of 404s and 200s for at least 5 minutes. Search results hits would also vary depending on the instance hit by the request.

I also did the same test locally just to be sure it wasn't because of the EFS mount, and I get the same behavior locally.

For now I think running with a single instance is a better option, we can revisit the multi-instance setup when the cluster mode lands in meili.

@wperron
Copy link
Contributor Author

wperron commented Sep 17, 2020

After consideration, I don't think Fargate is the right choice right now due to both the low amount of data we have and how much harder Fargate has been so far; EC2 just seems like a much simpler option.

@wperron wperron closed this Sep 17, 2020
@wperron wperron deleted the poc/meilisearch branch September 17, 2020 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve module search
1 participant