You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I am generating HashMap[String,String] and then send to Elastic Search with this line of code. rddAsMap.saveAsHadoopDataset(jobConf)
I also have type mapping defined on elastic search for my index and type properly.
But somehow elastic search kept all fields as string. Why is the mapping not kicking in?
Thanks.
Tian
The text was updated successfully, but these errors were encountered:
I don't follow. Can you maybe post some snippets of code of what happens vs what your expectations are? You can verify the mapping of a document directly in Elasticsearch through its REST API. Most likely the mapping doesn't apply since maps are often mapped dynamically - you should be able to check that at ES level.
Costin, sorry my fault, it turns out that the mapping indeed worked, it is just I am looking at the _source data which is indeed string. But I have another question about your package. Currently we generate HashMap[String,String] in order to call saveAsHadoopDataset. Is it possible to generate more generic HashMap[String, Object] and allow me to pass proper type of data directly to Elastic Search w/o ES to do the mapping? Thanks.
@tzhang101 I'm not sure what sure what you mean - you can send whatever object (whether it's a map or not) to es-hadoop. You haven't posted any code (typically it makes a big difference) so I can only infer that you are configuring OutputFormat to accept a Map[String, String] - simply configure it differently. Plus you don't have to use saveAsHadoopDataset()- simply use the native es-spark Java/Spark API.
Both approaches are explained in the docs so please review them.
In the future, for questions, please use the mailing list or the IRC (as mentioned in the resources) - the issue tracker is not the appropriate place. Thank you.
Hi, I am generating HashMap[String,String] and then send to Elastic Search with this line of code. rddAsMap.saveAsHadoopDataset(jobConf)
I also have type mapping defined on elastic search for my index and type properly.
But somehow elastic search kept all fields as string. Why is the mapping not kicking in?
Thanks.
Tian
The text was updated successfully, but these errors were encountered: