$ pip install spooq
$ python setup.py sdist bdist_wheel
The output is stored as dist/Spooq-<VERSION_NUMBER>-py3-none-any.whl
and Spooq-<VERSION_NUMBER>.tar.gz.
$ python setup.py bdist_egg
The output is stored as dist/Spooq-<VERSION_NUMBER>-py3.8.egg
$ zip_file_name="Spooq-$(grep "__version__" spooq/_version.py | cut -d " " -f 3 | tr -d \").zip"
$ zip -r $zip_file_name spooq
The output is stored as Spooq-<VERSION_NUMBER>.zip
.
For Submitting or Launching Spark:
$ pyspark --py-files Spooq-<VERSION_NUMBER>.egg
The library has still to be imported in the pyspark application!
Within Running Spark Session:
>>> sc.addFile("Spooq-<VERSION_NUMBER>.egg")
>>> import spooq
$ cd spooq
$ python setup.py install
$ pip install git+https://github.com/Breaka84/Spooq@master
Please refer to dev_setup
.