pyspark.ss.rst

Structured Streaming

Core Classes

pyspark.sql.streaming

DataStreamReader DataStreamWriter ForeachBatchFunction StreamingQuery StreamingQueryException StreamingQueryManager

Input and Output

pyspark.sql.streaming

DataStreamReader.csv DataStreamReader.format DataStreamReader.json DataStreamReader.load DataStreamReader.option DataStreamReader.options DataStreamReader.orc DataStreamReader.parquet DataStreamReader.schema DataStreamReader.text DataStreamWriter.foreach DataStreamWriter.foreachBatch DataStreamWriter.format DataStreamWriter.option DataStreamWriter.options DataStreamWriter.outputMode DataStreamWriter.partitionBy DataStreamWriter.queryName DataStreamWriter.start DataStreamWriter.trigger

Query Management

pyspark.sql.streaming

StreamingQuery.awaitTermination StreamingQuery.exception StreamingQuery.explain StreamingQuery.id StreamingQuery.isActive StreamingQuery.lastProgress StreamingQuery.name StreamingQuery.processAllAvailable StreamingQuery.recentProgress StreamingQuery.runId StreamingQuery.status StreamingQuery.stop StreamingQueryManager.active StreamingQueryManager.awaitAnyTermination StreamingQueryManager.get StreamingQueryManager.resetTerminated