Integration with ElasticSearch

[rth]

E-discovery tools are typically used in combination with a search engine and this issue aims therefore to evaluate the possibility of integrating FreeDiscovery with ElasticSearch.

Multiple aspects of this question are possible,

• API design: it might be worthwhile to make the FreeDiscovery REST API follow as much as possible the philosophy and principles of the ElasticSearch API which would make easier for users to use both
• ElasticSearch integration within FreeDiscovery: FreeDiscovery methods that perform nearest-neighbor search in the semantic space (including LSI + 1-NN categorization) could be made to take as optional input an elastic search index, in order to combine search in the semantic space with regular search.
• FreeDiscovery integration within ElasticSearch: ElasticSearch has a system of plugins aiming to enhance the base functionality, and it might be possible to add a FreeDiscovery plugin (though the specifics remain to be determined).

This is a long term discussion... Also might be related to Solr integration discussion #118 .

[sany2k8]

Is their any update with Integration with ElasticSearch or Solr? I need to find near duplicate json documents of my Elasticsearch Index. Please let me know how can I do that? If it is not possible now then let me know how can I find
NDD now if I store all the data sets of my elasticsearch indices inside a folder e.g 20_newsgroups keeps text documents inside a sub-folder with e.g 1.txt file.