Writing index names dynamically with nonlowercase fields causes error

[rbraley]

rdd.saveToES("index_{camelCaseField}")

yields the following error:
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Invalid index [index_{camelCaseField}] - needs to be lowercase

The field in my documents is not lowercase and cannot be changed. I "fixed" this by removing
+// Assert.isTrue(StringUtils.isLowerCase(index), String.format("Invalid index [%s] - needs to be lowercase", index));
in mr/src/main/java/org/elasticsearch/hadoop/rest/Resource.java
it has been working for me and it seems an unnecessary restriction on the source json documents anyhow.

I would like a more long term solution than forking elasticsearch-hadoop :)

[Highbrainer]

Hi,

any comment on this? Is this really an unncessary restriction or is there an hidden reason for that?

Thanks in advance!

[costin]

@rbraley @Highbrainer Maybe I'm missing something here but Elasticsearch doesn't allow indexes with uppercase.
Ignore es-hadoop and try:

curl -XPUT 'http://localhost:9200/index_Foo'

[2015-06-21 23:54:40,571][DEBUG][action.admin.indices.create] [Lady Killer] [index_Foo] failed to create
org.elasticsearch.indices.InvalidIndexNameException: [index_Foo] Invalid index name [index_Foo], must be lowercase

the connector does early validation so one doesn't start a job only to have things failing...

Unless I'm missing something, I'm closing this down as won't fix. If you think this isn't the case, please comment.

[Highbrainer]

@costin : My bad - my problem is with non lowercase Fields as stated in the title - eventhough @rbraley 's text deals with index names...

Something like

PUT localhost:9200/myindex/mytype/123 -d '{"fieldWithUpperCase" : "some value"}'

[costin]

@Highbrainer Have you tried using uppercase/camelcase fields and have ran into an error? If so, please open a new issue. This is supported and tested and part of the test suite.

[costin]

I think I finally understand what the issue is about - the field name is camelcase and is used dynamically but the value is not. I'll submit a fix shortly.

[rbraley]

Hi costin this fix only works if there is a single { } pattern in the index portion. if we have an index_{fieldOne}_{fieldTwo} it will error out.