facing issue dumping data from mongo to elasticsearch without transformation

[mandeepm91]

Here are the contents of my tes/config.yaml file and tes/application.js file

config.yaml

# api:
#   interval: 60s
#   uri: "http://requestb.in/13gerls1"
#   key: "48593282-b38d-4bf5-af58-f7327271e73d"
#   pid: "something-static"
nodes:
  localmongo:
    type: mongo
    uri: mongodb://localhost/foo
  es:
    type: elasticsearch
    uri: http://localhost:9200/

application.js

pipeline = Source({name:"localmongo", namespace:"foo.bar"}).save({name:"es", namespace:"foo.bar"}); 

However it fails while inserting to elasticsearch with the error:

failed to parse [_id]

However, it used to work fine earlier ( a month ago ). Also, if I apply a log transform to see the documents, I can see _id is stored differently:

{"_id":{"$oid":"54df376881ee34db87b27377"},"firstName":"Robert","lastName":"Baratheon"}

In the transformation, changing

doc._id = doc._id['$oid']; 

does solve the problem though. Just want to confirm if there has been a change in the way mongo _id is handled or I am doing something wrong ?

[nstott]

There was indeed a change related to how _id's are stored,
I'm working on a solution to this
see:
#62

[jipperinbham]

@mandeepm91 we've made some changes to how the _id is used/accessed, please take a look at this blogpost for the full details