You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Am trying to fetch data from elastic server into spark 2.4 environment using "org.elasticsearch.spark.sql" connector, but even though i got all the configs right, there is no data received on the spark engine
Using the below config to connect to Elastic from spark
Issue description
On analyzing with fiddler found that the request created had a search_type=scan param in the query string, which i guess is no longer supported
The only reason the search_type parameter gets added to the request is if it is running on a version earlier than 5.0, and there is currently an existing issue where more recent versions of ES-Hadoop reading from 2.x versions of Elasticsearch are not returning results. See #1277
What kind an issue is this?
Am trying to fetch data from elastic server into spark 2.4 environment using "org.elasticsearch.spark.sql" connector, but even though i got all the configs right, there is no data received on the spark engine
Using the below config to connect to Elastic from spark
Issue description
On analyzing with fiddler found that the request created had a search_type=scan param in the query string, which i guess is no longer supported
http://NodeURL/index/type/_search?search_type=scan&scroll=5m&size=50&preference=_shards%3A3%3B_local
On opening this on the browser didn't receive any data, but the same query without the search_type param worked as expected.
Am wondering when the above param is no longer supported, why is it being appended in the request url by the elasticsearch-spark connector
Version Info
spark - 2.4.0
scala - 2.11
elasticsearch-spark-20_2.11 (6.6.0)
OS: : Win 10
JVM : jdk1.8.0_181
Can someone please help me figure out a way to fetch data
The text was updated successfully, but these errors were encountered: