Skip to content

rvm-xx/sparklingpandas

 
 

Repository files navigation

SparklingPandas

SparklingPandas aims to make it easy to use the distributed computing power of PySpark to scale your data anlysis with Pandas.

Documentation

None (right now). You can find some slides in the slides/ directory

Requirements

The primary requirement of SparklingPandas is that you have a recent (v1.0-SNAPSHOT currently) version of Spark installed - http://spark.apache.org

Using

Make sure you have the SPARK_HOME enviroment variable set correctly, as SparklingPandas uses this for including the PySpark libraries

Other than that you can install SparklingPandas with pip and just import it. The primary unit of SparklingPandas is a PRDD (Pandas Resillent Distributed Data Set)

State

This is in early development and should not be considered usable.

Support

Check out our Google group at https://groups.google.com/forum/#!forum/sparklingpandas

About

Pandas On PySpark(POPS)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 98.8%
  • Shell 1.2%