fast, scalable & resilient replication daemon for elasticsearch
Please note this is still Work In Progress Do not use for prod use at all
- Unmatched performance **
- Scalable to the skies
- Support for MongoDB by default
- Support for any data, if u can put them into our NSQ
** Initial load can process up to 1 Milion messagges per second, Queue consumption performance depends on your hardware
Local Development is powered by Docker. Please see Makefile to get you up & running quickly. For local development be sure to set APP_ENV=dev in .env.
ENV Types
dist: .env file with all options you can configure, use for custom configuration
local: jump start for local development
Tip: You can just do
cp .env.local .envto use preconfigured settings
To learn more about local environment please refer to this wiki
Please refer to Makefile. It has all of the commands available to you to run the project in different modes.
By default daemon is handling replication from DB to NSQ so consumer can process data to elasticsearch. You can fill up NSQ by yourself if you need neccessary data manipulation on your side. Don't forget to turn off the producers in config afterwards.
Our roadmap could be found here. ERD | Notion
If you feel like you want to help me, please, pick an issue from notion which is not assigned and is in to-do column and create new issue here on github, tagging me for more information.
This project has a MIT license.