"More than one type" error

[peio]

Dear Moshe,
I am using elasticsearch_loader with the elastic version 7.1.1 on elastic cloud. When I add my own mapping to the index and then use the loader the following error is displayed:

"Rejecting mapping update to [myindex] as the final mapping would have more than 1 type: [_doc, json]', u'type': u'illegal_argument_exception'"

Tracing the issue I believe this change is the cause: https://www.elastic.co/blog/index-type-parent-child-join-now-future-in-elasticsearch

I was able to go around it by commenting this line:

elasticsearch_loader/elasticsearch_loader/iter.py

Line 18 in 510fdbe

'_type': config['type'],

But I believe there might be better solution, which you might be able to propose.

Thank you for the helpful software!
Peio

[moshe]

Hi @peio,
Did you add the --type flag?
from the error, I can see that you already have json type in your index so add --type=json (es7 requires only one type per index and the default if _doc)

[peio]

Hi @moshe
Yes, the command I execute is:

elasticsearch_loader.exe --index bgnews --http-auth elastic:XXXXX --es-host https://XXXXX.us-east-1.aws.found.io:9243 --type json json 24chasa-2019-05-31.json --lines

I have also noted difference on the behavior of the command parsing library on Windows 10 (Powershell) and Fedora Linux. It is separate issue, related to the parsing of the wildcard (*.json) but might be indication where the cause is.

Thank you,
Peio

[moshe]

can you try to change the type to _doc?
elasticsearch_loader.exe --index bgnews --http-auth elastic:XXXXX --es-host https://XXXXX.us-east-1.aws.found.io:9243 --type _doc json 24chasa-2019-05-31.json --lines

[peio]

Oh, --type _doc makes much more sense. It will take me a while to do a setup, because I am loading with a patched elasticsearch_loader on a prod setup, but will do a test setup as soon as I can.

[peio]

Yep, confirmed. --type _doc is the right argument.
Only thing I can do is an documentation improvement.

Thank you!

[moshe]

Actually, it is documented in the --help message:

  --type TEXT                     Docs type. TYPES WILL BE DEPRECATED IN APIS
                                  IN ELASTICSEARCH 7, AND COMPLETELY REMOVED
                                  IN 8.  [required]

I will write a better help message + warning if using es7 with type argument.

Thanks @peio