use Pysqark to read data from InfluxDB

[imadmoussa1]

I am using Pyspark and interested to read data directly with as RDD or Dataframe. is it possible ?
Now I am querying the data using DataFrameClient and convert it with createDataFrame, but the process is very slow. Specially loading a huge amount of data from influx to analyse.

[bednar]

Hi @imadmoussa1,

the client has capability to query data into DataFrame. The limitation will be amount of your data, because the client has to read all data into memory and transform this data into DataFrame. Unfortunately there is no option how to stream the InfluxDB's response into DataFrame directly.

Here is an example how to query data into DataFrame: https://github.com/influxdata/influxdb-client-python/blob/master/examples/query.py#L107

Regards