Dynamic index writer doesn't work with dataframes (spark sql)

[AlexeyRaga]

Hi, I am trying to use dynamic field writer feature:

conf.set("es.index.auto.create", "true")
....


docFrame.select("*", "doc.start").saveToEs("test2/{start}")

And here is the exception I get:

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot find match for test2/{start}

I tracked it down a bit and I see that it uses ScalaMapFieldExtractor trying to extract the value from the row, which may of may not be a problem (unfortunately it is hard to track because the source package on maven doesn't contain anything under serialization.

[costin]

trying to extract the value from the row, which may of may not be a problem

What do you documents look like? Typically this exception occurs if the RDD / DataFrame does not provide the field mentioned in the template.

[costin]

No follow-up, closing the issue