A search backend for Wagtail using MeiliSearch.
Caution
This package is still in development and until version 1.0.0, I will not maintain a DeprecationWarning pattern. I built the integration with meilisearch about 2 years ago for a project and decided to make it a public package to improve it and integrate more features.
Tip
If you need support or require help with a Wagtail project, you can hire me 😊
en - https://softquantum.com/resources/wagtailmeili-integrating-a-blazing-fast-search-engine-with-wagtail
Wagtailmeili requires the following:
- Python >= 3.11
- Wagtail >= 5.2
Install Meilisearch python client e.g., using pip
pip install meilisearch
Add wagtailmeili
to your INSTALLED_APPS
INSTALLED_APPS = [
# ...
"wagtailmeili",
# ...
]
add the search backend 'meilisearch' to your WAGTAILSEARCH_BACKENDS
Caution
Leave the 'default' backend for the admin as you don't want to depend only on what was indexed in meilisearch Different use cases to consider so still work in progress.
import os
WAGTAILSEARCH_BACKENDS = {
"meilisearch": {
"BACKEND": "wagtailmeili.backend",
"HOST": os.environ.get("MEILISEARCH_HOST", "http://127.0.0.1"),
"PORT": os.environ.get("MEILISEARCH_PORT", "7700"),
"MASTER_KEY": os.environ.get("MEILISEARCH_MASTER_KEY", "your-master-key"),
# "STOP_WORDS": ...
# "RANKING_RULES: ...
# "SKIP_MODELS": ...
# "SKIP_MODELS_BY_FIELD_VALUE": ...
},
"default": {
"BACKEND": "wagtail.search.backends.database",
}
}
- STOP_WORDS: see defaults in settings.py
- RANKING_RULES: see defaults in settings.py
- SKIP_MODELS: "skip_models" is a list of models that you want to skip from indexing no matter the model setup.
WAGTAILSEARCH_BACKENDS = {
"meilisearch": {
"SKIP_MODELS": ["app_label.Model1", "app_label.Model2",],
# ...
}
}
- SKIP_MODELS_BY_FIELD_VALUE: A convenient way to skip instances based on attributes
WAGTAILSEARCH_BACKENDS = {
"meilisearch": {
# add this to not index pages that are not published for example
"SKIP_MODELS_BY_FIELD_VALUE": {
"wagtailmeili_testapp.MoviePage": {
"field": "live",
"value": False,
},
},
# ...
}
}
In any model you will be doing a search on with Meilisearch, add the page or model manager.
It will use the correct backend when doing something like MySuperPage.objects.search()
.
from wagtail.models import Page
from django.db import models
from wagtailmeili.manager import MeilisearchPageManager, MeilisearchModelManager
class MySuperPage(Page):
"""A Super Page to do incredible things indexed in Meilisearch."""
objects = MeilisearchPageManager()
class MySuperModel(models.Model):
"""A Super Model to do incredible things indexed in Meilisearch."""
objects = MeilisearchModelManager()
To declare sortable attributes or add ranking rules for the model, just add, for example:
from wagtail.models import Page
class MySuperPage(Page):
"""A Super Page to do incredible things indexed in Meilisearch."""
sortable_attributes = [
"published_last_timestamp",
# ...
]
ranking_rules = [
"published_last_timestamp:desc",
# ...
]
{% load meilisearch %}
{% get_matches_position result %}
Wagtailmeili automatically handles cleanup of stale documents from your MeiliSearch indexes to ensure search results remain accurate and up-to-date.
Real-time Cleanup: When items are deleted or unpublished, they are automatically removed from the search index via Django signals.
Rebuild Cleanup: When running python manage.py update_index
, stale documents are automatically detected and removed.
Smart Detection: The system automatically handles both regular models and Page models with live
field detection.
Use the management command for manual cleanup operations:
# Clean all indexes (dry-run mode)
python manage.py cleanup_search_index --dry-run
# Clean all indexes
python manage.py cleanup_search_index
# Clean specific model
python manage.py cleanup_search_index --model wagtailmeili_testapp.MoviePage
# Verbose output
python manage.py cleanup_search_index --verbosity 2
You can also perform cleanup operations programmatically:
from wagtail.search.backends import get_search_backend
# Get the MeiliSearch backend
backend = get_search_backend('meilisearch')
index = backend.get_index_for_model(MyModel)
# Remove specific items
index.delete_item(item_id)
# Bulk remove multiple items
index.bulk_delete_items([id1, id2, id3])
# Clean up stale documents
live_ids = MyModel.objects.filter(live=True).values_list('pk', flat=True)
index.cleanup_stale_documents(live_ids)
The cleanup system respects your existing configuration:
- SKIP_MODELS: Models in this list won't be cleaned up
- SKIP_MODELS_BY_FIELD_VALUE: Field-based skipping is honored during cleanup
- Page Models: Only live pages (
live=True
) are considered valid for Page models
-
Adding tests(v0.3.3) -
Cleaning up index if pages are unpublished or models deleted(v0.4.0) - Exploring Meilisearch and bringing more of its features for Wagtail
- Getting a leaner implementation (looking at Autocomplete and rebuilder)
- Giving more love to the Sample project with a frontend
- official documentation
The Wagtail Movie Database (WMDB) is a sample project for testing purposes. To run the project, follow these steps:
- start the local meilisearch instance
meilisearch --master-key=<your masterKey>
- copy the directory wagtail_moviedb wherever you want
- create a virtualenv and activate it (instructions on linux/macOS)
python -m venv .venv
source .venv/bin/activate
- install the dependencies
pip install -r requirements.txt
- add an .env file
MEILISEARCH_MASTER_KEY="your masterKey"
- apply migrations
python manage.py migrate
- Create a superuser (optional)
python manage.py createsuperuser
- load movies & start local web server
python manage.py load_movies
python manage.py runserver
- visit your meilisearch local instance: https://127.0.0.1:7700, give it your master-key. You should see some movies loaded.
- update index (optional):
python manage.py update_index
This package uses flit for both local development and publishing.
- Install flit (if not already available):
# Via pip
pip install flit
# Via pyenv (if you have a Python version with flit pre-installed)
# flit may already be available in your pyenv Python installation
- Install the package locally for development:
# Using pip
pip install "pip>=21.3"
pip install -e '.[dev]' -U
# Using flit
flit install -s
For more information on installing and using flit, see the official flit documentation.
Welcome to all contributions and reviews!
- Install Meilisearch locally following their documentation
- Start Meilisearch instance in your favorite terminal
meilisearch --master-key correctmasterkey
To make changes to this project, first clone this repository:
git clone git@github.com:softquantum/wagtailmeili.git
cd wagtailmeili
With your preferred virtualenv activated, install testing dependencies:
pip install "pip>=21.3"
pip install -e '.[dev]' -U
You can run tests as shown below:
pytest
or with tox
tox
- ✅ This project is to experiment with my dev experience and improve my skills.
- ✅ It is, since v0.3, developed in an augmented development setup (JetBrains Pycharm, Claude Code with custom commands and configs)
- ✅ I commit to have a test suite that makes sense (reviews are welcome)
- ✅ The project is used in production in real projects: no shortcuts for quality standards, so if you find a bug please report it.
- ✅ It is an open source project so you can hire me for support ☕️
This project is released under the 3-Clause BSD License.