Current status of this project?

[utsengar]

Hello,

I am not sure if this is the right place to ask this question but I couldn’t find a mailing list or contact info.

I have been playing with this library and looks really good, it just works. I are planning to use elastic search to index about 45M documents. So, I will use this with a pig script to throw data from cassandra to elastic search.

So, what can we expect a stable release and what is the current status of this project for production use?

[utsengar]

Also wanted to let you know, I am getting a performance of 25,000 documents (with 13 columns)/min. There were 6 mappers allocated to the job and I am running pig 0.9.1

I get a better performance by accessing the cassandra data via a river, details: https://groups.google.com/forum/?fromgroups=#!topic/hector-users/OzAN3ji_gec

[costin]

@utkarsh2012 Thanks for the kind words. You can use the issue list or use the ElasticSearch group to post question (will update the home page to better reflect this).

Since it's a young project there are still a couple of things to iron out first before thinking about an official production release. However we do work on pushing out nightly builds as well as milestones.

Note that production wise, the data is stored within ES - meaning while this project can improve the read/write to/from ES, once within ES, your data is handled by the ES production policy (i.e. if you use a stable version, things should be okay).

Time-wise, I don't want to give any ETAs yet since there are still some critical design points that I'd like to get right first before doing estimates - otherwise I'm just guessing which is bad for everyone involved :)

[costin]

@utkarsh2012 by the way, regarding performance there's a lot when can improve on. Currently the code uses only 1 mapper/reducer task since the main aim is to handle the pig/hive integration properly. This will be changed shortly to be:

1. pluggable
2. use proper parallel querying/insertion
3. use an improved json/object serialization

This should improve performance a lot.

[utsengar]

Sounds good. Will be closely watching the development of this project :)