Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 

Unit Testing with PySpark

A minimal unit testing setup with PySpark fixtures.

For CDSW users, you must ensure that the PYSPARK_PYTHON environment variable in the CDSW points to the same version of python as the driver (env var PYSPARK_DRIVER_PYTHON.)

For instance, you may want to set a project-level environment variable to something like.

PYSPARK_PYTHON=/usr/local/bin/python3.6

About

A very tiny example of unit testing with PySpark

Resources

Releases

No releases published

Packages

No packages published

Languages