Skip to content
/ pyspark_projects Public

A collection of small projects exploring PySpark features and functionality including packages and modules, algorithms, and general data science techniques.

0 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Bcromas/pyspark_projects

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

26 Commits
images
images
 
 
.gitignore
.gitignore
 
 
Clustering.ipynb
Clustering.ipynb
 
 
Linear_Regression.ipynb
Linear_Regression.ipynb
 
 
Logistic_Regression.ipynb
Logistic_Regression.ipynb
 
 
README.md
README.md
 
 
Random_Forest.ipynb
Random_Forest.ipynb
 
 
Streaming.ipynb
Streaming.ipynb
 
 
requirements.txt
requirements.txt
 
 
run_streaming_things.ipynb
run_streaming_things.ipynb
 
 
streaming_things.py
streaming_things.py
 
 

Repository files navigation

PySpark Projects

A collection of small projects exploring PySpark features and functionality including:

Packages and modules

readStream/writeStream, Pipeline, OneHotEncoder, StringIndexer, StandardScaler, VectorAssembler

Algorithms

RandomForestClassifier, KMeans, LinearRegressionm, ridge and LASSO regressions, LogisticRegression

Techniques

feature extraction, evaluating the colinearity of features, calculating AUC, extracting feature importances, pre-processing, and EDA.

About

A collection of small projects exploring PySpark features and functionality including packages and modules, algorithms, and general data science techniques.

Topics

pyspark spark-streaming spark-sql spark-mllib spark-dataframes

Resources

Readme
Activity

Stars

0 stars

Watchers

2 watching

Forks

0 forks
Report repository

Languages