Semantic analysis on Spark

To run the Spark application, assuming Spark binaries are in your path:

$ spark-submit --packages com.databricks:spark-csv_2.11:1.4.0 app.py

To test the pipelines and datasource files:

$ spark-submit --packages com.databricks:spark-csv_2.11:1.4.0 datasources.py

$ spark-submit --packages com.databricks:spark-csv_2.11:1.4.0 pipelines.py

Name		Name	Last commit message	Last commit date
Latest commit History 28 Commits
.gitignore		.gitignore
README.md		README.md
app.py		app.py
datasources.py		datasources.py
exploration.ipynb		exploration.ipynb
pipelines.py		pipelines.py
requirements.txt		requirements.txt
test.py		test.py
transformers.py		transformers.py

Provide feedback