Cannot read property 'status' of undefined

[jovimon]

Hello,

I have Elasticsearch v1.1.2 debian package.

Just installed the Exporter (version 1.3.3) and first test was to export an index to file and to copy it to a new index in the cluster.

Export task completes OK, but copying to new index fails:

$ nodejs exporter.js -f /home/user/Desktop/asdf-export -b localhost -q 12345 -j logstash-asdf-2014.08.14 -x false
Elasticsearch Exporter - Version 1.3.3
Reading mapping from meta file /home/user/Desktop/asdf-export.meta
Waiting for mapping on target host to be ready, queue length 100
Creating index mapping in target ElasticSearch instance
Waiting for mapping on target host to be ready, queue length 200
Waiting for mapping on target host to be ready, queue length 300
Waiting for mapping on target host to be ready, queue length 400
Waiting for mapping on target host to be ready, queue length 500
Waiting for mapping on target host to be ready, queue length 600
Waiting for mapping on target host to be ready, queue length 700
Waiting for mapping on target host to be ready, queue length 800
Waiting for mapping on target host to be ready, queue length 900
Waiting for mapping on target host to be ready, queue length 1000
Waiting for mapping on target host to be ready, queue length 1100
Waiting for mapping on target host to be ready, queue length 1200
Waiting for mapping on target host to be ready, queue length 1300
Waiting for mapping on target host to be ready, queue length 1400
Waiting for mapping on target host to be ready, queue length 1500
Waiting for mapping on target host to be ready, queue length 1600
Mapping is now ready. Starting with 1600 queued hits.
Processed 100 of 78973 entries (0%)
Processed 200 of 78973 entries (0%)
Processed 300 of 78973 entries (0%)
Processed 400 of 78973 entries (1%)
Processed 500 of 78973 entries (1%)
Processed 600 of 78973 entries (1%)
Processed 700 of 78973 entries (1%)
Processed 800 of 78973 entries (1%)
Processed 900 of 78973 entries (1%)
Processed 1000 of 78973 entries (1%)
Processed 1100 of 78973 entries (1%)
Processed 1200 of 78973 entries (2%)
Processed 1300 of 78973 entries (2%)
Caught exception in Main process: TypeError: Cannot read property 'status' of undefined
TypeError: Cannot read property 'status' of undefined
    at IncomingMessage.<anonymous> (/home/user/node_modules/elasticsearch-exporter/drivers/es.js:584:34)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)
Number of calls:    76
Fetched Entries:    7600 documents
Processed Entries:  1300 documents
Source DB Size:     78973 documents

Script shows only 1300 processed documents but elasticsearch's kopf plugin says index has 5k+ documents.

Hope you can help.

Thanks in advance.

[mallocator]

Try the latest version with the fix I just added. I don't know what the actual cause for the error is and it's also really hard to reproduce such errors, so I'm just hoping that this is now fixed.

[jovimon]

Thanks. It works now ;)

Now it continues after an error and I can see the problem, that in my case was like:

{"create":{"_index":"logstash-2014.08.14","_type":"type","_id":"LVELGyJ_Qqa0hkWBj0KIxA","status":409,"error":"DocumentAlreadyExistsException[[logstash-2014.08.14][3] [type][LVELGyJ_Qqa0hkWBj0KIxA]: document already exists]"}}

[mallocator]

Did the document exist before, or is this an error from the exporter?

[jovimon]

I believe it's from the exporter.

I did as follows:
1.- Export an index to file.
2.- Import the file I just created to a new index.

[mallocator]

Can you check the file if the entry with the problem shows up twice?

In any case, I'm surprised this throws an error at all, since an existing document should simply be ignored or overwritten (depending on mode).

[jovimon]

Most of them appear only once, but I checked all of them and got 40 duplicates in the exported file. Index size is about 7600 items.

IMHO, the app seems to be in a loophole trying to import again items it had already imported, because it's been running for 30min and it's still throwing errors.

[mallocator]

It might still be running because somewhere it doesn't count these errors as documents, but that should be visible in the output. Is it stuck somewhere at 99% or does it just repeatedly throw the same error?

[jovimon]

It keeps throwing the same error (409 DocumentAlreadyExistsException) with different document IDs at a rate of about 20 errors per second or so.

Just before the errors begin to flood the console I can see some "Processed XX of YY entries (ZZ%)" but I believe last one I saw this morning was about 70%.

[mallocator]

When you so the script it should tell you the document stats. Can you post
those too?

I'll be honest, I have no idea what's wrong right now. This is the first
time I've heard of such behavior.
On Aug 18, 2014 9:56 AM, "jovimon" notifications@github.com wrote:

It keeps throwing the same error (409 DocumentAlreadyExistsException) with
different document IDs at a rate of about 20 errors per second or so.

Just before the errors begin to flood the console I can see some
"Processed XX of YY entries (ZZ%)" but I believe last one I saw this
morning was about 70%.

—
Reply to this email directly or view it on GitHub
#65 (comment)
.