data being indexed on elasticsearch does not get pushed to mongodb

[forste]

I initialized the river as documented in the wiki. Replica set is set up, but I only have a single replication server.

When I insert data in mongodb it gets pushed to elasticsearch:

mongo
PRIMARY> use DBNAME
PRIMARY> entry = {    "user" : "phil",
...     "post_date" : "2009-11-15T14:12:12",
...     "message" : "trying out Elastic Search"}
{
    "user" : "phil",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}

db.contacts.insert(entry)

$ curl -XGET 'http://localhost:9200/contacts/_search?pretty=true&size=5000' -d '
> { 
>     "query" : { 
>         "matchAll" : {} 
>     } 
> }'
{
  "took" : 62,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "contacts",
      "_type" : "contact",
      "_id" : "1",
      "_score" : 1.0, "_source" : {
    "user" : "phil",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}
    } ]
  }

However, when I index data to elasticsearch issuing a PUT request, the data does not show up in mongodb (data from above has been cleared before executing the following sample)

$ curl -XPUT 'http://localhost:9200/contacts/contact/1' -d '{
>     "user" : "kimchy",
>     "post_date" : "2009-11-15T14:12:12",
>     "message" : "trying out Elastic Search"
> }'
{"ok":true,"_index":"contacts","_type":"contact","_id":"1","_version":1}forste@machine:~/opt$ 
$ url -XGET 'http://localhost:9200/contacts/_search?pretty=true&size=5000' -d '
{                     
    "query" : {                         
        "matchAll" : {} 
    } 
}'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "contacts",
      "_type" : "contact",
      "_id" : "1",
      "_score" : 1.0, "_source" : {
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}
    } ]
  }
}

$ mongo
PRIMARY> use DBNAME
PRIMARY> db.contacts.find()
PRIMARY> 

Does the river work both ways?

[richardwilly98]

Hi,

The river only import the data from MongoDB in ElasticSearch.

Thanks,
Richard.

On Thu, Aug 23, 2012 at 10:40 AM, Stephan notifications@github.com wrote:

I initialized the river as documented in the wiki. Replica set is set up,
but I only have a single replication server.

When I insert data in mongodb it gets pushed to elasticsearch:

mongo
use DBNAME
PRIMARY> entry = { "user" : "phil",
... "post_date" : "2009-11-15T14:12:12",
... "message" : "trying out Elastic Search"}
{
"user" : "phil",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search"
}

db.contacts.insert(entry)

$ curl -XGET 'http://localhost:9200/contacts/_search?pretty=true&size=5000' -d '

{
"query" : {
"matchAll" : {}
}
}'
{
"took" : 62,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "contacts",
"_type" : "contact",
"_id" : "1",
"_score" : 1.0, "_source" : {
"user" : "phil",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search"
}
} ]
}

However, when I index data to elasticsearch via PUSH, the data does not
show up in mongodb

$ curl -XPUT 'http://localhost:9200/contacts/contact/1' -d '{

"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search"

}'
{"ok":true,"_index":"contacts","_type":"contact","_id":"1","_version":1}forste@machine:~/opt$
$ url -XGET 'http://localhost:9200/contacts/_search?pretty=true&size=5000' -d '
{
"query" : {
"matchAll" : {}
}
}'
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "contacts",
"_type" : "contact",
"_id" : "1",
"_score" : 1.0, "_source" : {
"user" : "kimchy",
"post_date" : "2009-11-15T14:12:12",
"message" : "trying out Elastic Search"
}
} ]
}
}

PRIMARY> db.contacts.find()
PRIMARY>

Does the river work both ways?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/32.

[karansingh1000]

Hi Richard,

Your plugin/river is superb.

I needed some help.
How do i make your river index multiple collections of MongoDB at the same time.

I want the river to index two collections at the same time instead of only one.
Is there any way to do that...

Thanks
Jaskaran

[richardwilly98]

Hi,

In the current version one river setting will only monitor one MongoDB collection so you could create one river for each MongoDB collection to monitor.

curl -XPUT "localhost:9200/_river/mongodb/_meta" ....
curl -XPUT "localhost:9200/_river/mongodb2/_meta" ....

Thanks,
Richard.

[karansingh1000]

Hi Richard,

Thanks a lot for your help.
It works now :-)
Great Thank you very much....

Thanks
Jaskaran

[nishtha-singh]

Hi Richard,
I followed your blog and had set up mongodb river plugin for elastic search.
I followed all the steps in sequence and in addition to them I had successfully setup replica sets for Mongodb and inserted data into the collection person of database testmongo but I'm getting the same output again and again which is
{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
I am specifying the version as well
Mongodb - 2.4.9
ElasticSearch 1.0.1
elastic search mapper attachments 1.6.0
river mongodb plugin 1.6.1
Also I tried several solutions but none of them worked for me. Any suggestion would be of great help.
Thanks,
Nishtha

[richardwilly98]

@nishtha-singh with ES 1.0.X you should use the river release 2.0.0:

bin/plugin --install com.github.richardwilly98.elasticsearch/elasticsearch-river-mongodb/2.0.0