index.rst

Welcome to Spark Python API Docs!

Contents:

pyspark pyspark.sql pyspark.streaming pyspark.ml pyspark.mllib

Core classes:

pyspark.SparkContext

Main entry point for Spark functionality.

pyspark.RDD

A Resilient Distributed Dataset (RDD), the basic abstraction in Spark.

pyspark.sql.SQLContext

Main entry point for DataFrame and SQL functionality.

pyspark.sql.DataFrame

A distributed collection of data grouped into named columns.

Indices and tables

• search