DAT SF 12

Instructor:	Alessandro Gagliardi
EiRs:	Ramesh Sampath
	Otto Stegmaier
	Alex Chao
Classes:	6:30pm-9:30pm, Tuesday and Thursdays
	January 15 – March 31
Office Hours:	Alex Chao, 5:30 - 6:30 before class at GA
	Otto Stegmaier, 9:30 - 10:00 after class at GA
	Ramesh Sampath, 4:00 - 6:00 Saturdays remote
	Can also set by appointment

Homework is to be submitted by posting it to your own github repo. Then post the URL and folder where the homework lives at here.

Tentative Course Outline

Intro to Data Science, Relational Databases & SQL
- Awesome Public Datasets
Getting started with IPython & Git
- How to Use Git and GitHub
- SourceTree
- What’s new in IPython
APIs and semi-structured data
- Mining Twitter
IPython.parallel & StarCluster
- Parallel Computing with IPython
- AWS Provisining
Hadoop Distributed File System and Spark
- AMP Camp 5 Exercises
- Announcing Spark 1.3
Intro to ML: k-Nearest Neighbor Classification
- Nearest Neighbor Methods (PDF Slides)
- k-Nearest Neighbor Classification Algorithm (YouTube Video)
- K Nearest Neighbors (Coursera Video)
- KNN for Humans
- Intuitive Classification using KNN and Python
- Nearest Neighbors Classification (scikit-learn documentation)
- Should I normalize/standardize/rescale the data?
Clustering: Hierarchical and K-Means
- A Tutorial on Clustering Algorithms (web tutorial)
- Hierarchical Clustering in Action
- K-means (scikit-learn documentation)
- Clustering: k-means (PDF Slides)
- Clustering: Hierarchical Clustering (PDF Slides)
Probability, A/B Tests & Statistical Significance
- Probability and Statistics (Khan Academy Course)
- What’s a good value for R-squared?
- Visualizing Distributions of Data
Multiple Linear Regression and ANOVA
- Plotting Linear Models using Seaborn
- Why ANOVA and Linear Regression are the Same Analysis
- Linear Regresion and ANOVA (Examples in R)
- Multiple (Linear) Regression (Examples in R)
Logistic Regression and Generlized Linear Models
- Logistic Regression in Python
- Explaining Odds Ratios
- GLM for Poisson Data
- Logistic Regression (scikit-learn documentation)
- sklearn logistic regression with unbalanced classes
Project Elevator Pitches
- See Student Project Repos below
Naïve Bayes, Cross Validation, ROC, AUC & Midterm Review - Part I
- Bayes' Theorem with Lego
- Probabilistic Programming and Bayesian Methods for Hackers
- Doing Naive Bayes Classification
- Receiver operating characteristic (wikipedia article)
- Receiver Operating Characteristic (ROC) (scikit-learn documentation)
Naïve Bayes, Cross Validation, ROC, AUC - Part II
Principal Components Analysis
- PCA and other Decomposition Examples in scikit-learn
- Principal Component Analysis Explained Visually
- Should you apply PCA to your data?
Decision Trees and Forests
- Decision Tree Learning (Wikipedia article)
- Decision Trees
- How to construct a tree
- Information gain
Support Vector Machines
- Support Vector Machines (scikit-learn documentation)
- A User's Guide to Support Vector Machines
Scaling Out
- numpy.memmap
- Feature Hashing
- HashingVectorizer
- Vectorizing a large text corpus with the hashing trick
- Pyrallel - Parallel Data Analytics in Python
- Online Passive-Aggressive Algorithms
Recommendation Systems
- Machine Learning & Recommender Systems @ Netflix Scale
Visualization
- Chart Suggestions—A Thought-Starter
- Seaborn: statistical data visualization
- Welcome to Bokeh
- ggplot for python
- plotly
Final Project Presentations (12 min. each)
Final Project Presentations (12 min. each)
Future Directions

Project Schedule

Date	Due	Returned
1/22	Preliminary Project Proposals Due (3-4 sentences)
1/27	Homework 1
1/29		EiR Feedback on Project Proposals
2/3		EiR Feedback on Homework 1
2/5	Formal Proposals (including data and methods chosen)
2/10	Homework 2 Assigned
2/12		EiR Feedback on Formal Proposals
2/17	Homework 2 Due
2/19	Project Elevator Pitch in class (4 minutes each)	Project Live on Github
2/24		Homework 3 Assigned
2/26	Peer Feedback of Projects	Peer Feedback on Project
3/3		Midterm Assessment Posted
3/12	Midterm Assessment Due
3/17	At least one working model
3/24-26	Final Presentations (12 minutes each)	Midterm Graded

Student Project Repos:

Name		Name	Last commit message	Last commit date
Latest commit History 152 Commits
.ipynb_checkpoints		.ipynb_checkpoints
Homework		Homework
Mid-Term-Submissions		Mid-Term-Submissions
Project		Project
Solutions		Solutions
assets		assets
data		data
datasets/lfw_home		datasets/lfw_home
.gitignore		.gitignore
01 - Intro to Data.ipynb		01 - Intro to Data.ipynb
02 - SQL & Python.ipynb		02 - SQL & Python.ipynb
02 - Solutions.ipynb		02 - Solutions.ipynb
03 - Twitter Mongo Lab.ipynb		03 - Twitter Mongo Lab.ipynb
04 - Big Data I - AWS.ipynb		04 - Big Data I - AWS.ipynb
04 - StarCluster Lab.ipynb		04 - StarCluster Lab.ipynb
05 - Big Data II - MapReduce.ipynb		05 - Big Data II - MapReduce.ipynb
05 - Hadoop Lab.ipynb		05 - Hadoop Lab.ipynb
05 - Spark Lab.ipynb		05 - Spark Lab.ipynb
06 - ML & kNN.ipynb		06 - ML & kNN.ipynb
06 - Neighbors Lab.ipynb		06 - Neighbors Lab.ipynb
07 - Clustering Lab-Soln.ipynb		07 - Clustering Lab-Soln.ipynb
07 - Clustering Lab.ipynb		07 - Clustering Lab.ipynb
07 - Clustering.ipynb		07 - Clustering.ipynb
08-Stats.ipynb		08-Stats.ipynb
08-Stats_Lab.ipynb		08-Stats_Lab.ipynb
09-MultiLineaRegression.ipynb		09-MultiLineaRegression.ipynb
09-RegressionLab.ipynb		09-RegressionLab.ipynb
10-GLM.ipynb		10-GLM.ipynb
10-LogitLab-Solutions.ipynb		10-LogitLab-Solutions.ipynb
10-LogitLab.ipynb		10-LogitLab.ipynb
12-Bayes.ipynb		12-Bayes.ipynb
12-TextMining_solutions.ipynb		12-TextMining_solutions.ipynb
13-TextMining.ipynb		13-TextMining.ipynb
14-DimensionalityReduction.ipynb		14-DimensionalityReduction.ipynb
14-PCA_lab-Solution.ipynb		14-PCA_lab-Solution.ipynb
14-PCA_lab.ipynb		14-PCA_lab.ipynb
15-BoostedTrees-Lab-Solution.ipynb		15-BoostedTrees-Lab-Solution.ipynb
15-BoostedTrees.ipynb		15-BoostedTrees.ipynb
15-DecisionTrees_Forests.ipynb		15-DecisionTrees_Forests.ipynb
16-SupportVectorMachines.ipynb		16-SupportVectorMachines.ipynb
16-svm_lab.ipynb		16-svm_lab.ipynb
16-svm_solution.ipynb		16-svm_solution.ipynb
17-ScalingOut.ipynb		17-ScalingOut.ipynb
18-recommendation-systems-movie-lens.ipynb		18-recommendation-systems-movie-lens.ipynb
18-recommendation-systems.ipynb		18-recommendation-systems.ipynb
19-DataViz.ipynb		19-DataViz.ipynb
22-NextSteps.ipynb		22-NextSteps.ipynb
Gini and Entropy.ipynb		Gini and Entropy.ipynb
Next Steps After DAT 12.pdf		Next Steps After DAT 12.pdf
Pandas for SQL Developers.ipynb		Pandas for SQL Developers.ipynb
README.md		README.md
dummy_api_cred.yml		dummy_api_cred.yml
enron.db		enron.db
fetch_data.py		fetch_data.py
index.html		index.html
sample.ipynb		sample.ipynb
save-pdf-plot.ipynb		save-pdf-plot.ipynb
sim_functions.png		sim_functions.png
svm_gui.py		svm_gui.py
trans-aa.csv		trans-aa.csv

coolcalbeans/DAT_SF_12

Folders and files

Latest commit

History

Repository files navigation

DAT SF 12

Tentative Course Outline

Project Schedule

Student Project Repos:

About

Resources

Stars

Watchers

Forks

Languages