Mysql Sync to Elastic Search

[prakritidev]

Need Help for MYSQL Sync to ElasticSearch.

[prakritidev]

@kevivforever I want update on this. Give me time slot so that we can work on this together.

[prakritidev]

I found this thing reliable than other methods. Let me know what do you think.

The best way is by making a script to synchronize yourself. Please do not use rivers, they are slow and non-reliable.
Add a "modified_at" column to your tables, a trigger to update this column every time you modify/insert data and make your script do queries like "SELECT * FROM XXX where modified_at >= '2015-10-01 00:00:00"
Take the data, transform into what's useful to you and next time you only synchronize what changed. This way you can easily handle relationships customizing your script. You now can make N queries after something is modified gathering all the relationships, by adding a queue you can scale to many machines doing the ETL (Extract, Transform, Load) operation into ES.

[prakritidev]

@kevivforever Links that we will be following for this project.

https://sysadminci.wordpress.com/2016/01/06/import-mysql-data-in-elasticsearch-server/
https://logz.io/blog/docker-logging/
https://medium.com/@siddontang/how-to-sync-your-mysql-data-to-elasticsearch-ddae009243c1
https://www.elastic.co/blog/found-keeping-elasticsearch-in-sync