Statistical Computing and Introduction to Data Science

STAT5206 - Fall 2023

Teaching Team

Yongchan Kwon (yk3012 (at) columbia (dot) edu)

• Office Hours: By appointment.

Fangyi Chen (fc2630 (at) columbia (dot) edu)

• Office hours: Every Friday 3 pm - 4 pm

Logistics

Class time: F 10:10am-12:40pm, Location: 207 Mathematics Building

Timeline (Last update Nov 13th)

Date	Topic	Reference	Due
Week 1	- Introduction - Python 101-1 (Variable)	- PCN Chapter 1-10 - PPSC Chapter 1, 2 - PDSH Chapter 2
Week 2	- Python 101-2 (Function, Package, Loop, if/else, File I/O)	- PCN Chapter 1-10
Week 3	- Numpy	- PCN Chapter 11	- HW1 (Due Sep 27th 11:59 PM)
Week 4	- N-gram - OLS
Week 5	- Pandas (DataFrame, Grouping, Merge, Timestamp)	- PCN Chapter 12	- HW2 (Due Oct 13th 11:59 PM)
Week 6	- COVID-19 - (AB testing)	- PCN Chapter 16
Week 7	- Visualization (matplotlib, seaborn) - Requests - (Twitter) - (NYTimes) - Review	- PCN Chapter 17	- HW3 (Due Oct 22nd 7:00 PM)
Week 8	Midterm (Oct. 27th at 6:10 PM - 8:40 PM)
Week 9	- Midterm review
Week 10	- regular expression	- PCN Chapter 13
Week 11	- SQL - (Internet speed)	- PCN Chapter 20	- HW 4 (Due Nov 19th 11:59 PM)
Week 12	University Holiday NO CLASS
Week 13	- Linear Model (Feature Engineering, Data Splitting, Cross-validation) - Medical Insurance	- PCN Chapter 18 - PDSH Chapter 5	- HW 5 (Due Dec 4th 4:00 PM)
Week 14	Final (TBU)

Learning outcomes

This course is designed for beginners who do NOT have any experience in Python programming. We will cover following topics:

• Basic concepts for Python programming
  • Data types, Functions, Control flow (loop/if-else)
• Data acquistion
  • Data collection using API and SQL query
• Data wrangling for statistical analysis
  • from hierarchical structured data to tabular data
• Data visualization
  • scatterplot, histogram, boxplot, ...
• Basic statistical analysis
  • linear model, simple optimization

Prerequisites

• An introductory statistics class
  • Basic probability distributions (e.g. Gaussian, binomial distributions and their likelihoods)
  • Basic hypothesis testing (e.g. t-test)
  • Summary statistics
  • Histograms, boxplots, etc
• Multivariate calculus
  • Derivatives and functions
• Matrix operations and inverses of matrices
• You should be at least co-enrolled in a modeling class like regression

References

• Google!
• Basics only - Programming with Python by Software Carpentry (PPSC)
• Python concept notes (PCN)
• Python Data Science Handbook (PDSH)
• LearningPython.org (LP)
• Data, Models, and Decisions: The Fundamentals of Management Science

Grading

- Homeworks (40%)

• Late homeworks will receive 0 credit
• No make-up homeworks will be granted even if you registered late to the class
• Please read these important things related to submitting homeworks on Ed

- Exams (60%)

• Final (60%)

Exam accomodations

In order to receive disability-related academic accommodations for this course, students must first be registered with their school Disability Services (DS) office. Detailed information is available online for both the Columbia and Barnard registration processes.

Refer to the appropriate website for information regarding deadlines, disability documentation requirements, and drop-in hours (Columbia)/intake session (Barnard).

For this course, students are not required to have testing forms or accommodation letters signed by faculty. However, students must do the following:

· The Instructor section of the form has already been completed and does not need to be signed by the professor.

· The student must complete the Student section of the form and submit the form to Disability Services.

· Master forms are available in the Disability Services office or online: https://health.columbia.edu/services/testing-accommodations

Expectations

• Take chances!
  • Break the code in lecture
• Give feedback in office hours or e-mail, don't waste your time if you think a topic is not helpful
• Participate and ask questions, this is not easy!
  • In class: forecast what should be done, compare with what is happening, then summarize the difference.
  • Online: describe what you observe, describe what you expect, communicate clearly.
  • To each other: summarize the conversation to ensure you're listening and think constructively before criticizing.
• THE MOST IMPORTANT Academic honesty: https://www.cs.columbia.edu/education/honesty/

Acknowledgement

A lot of these materials are based off the materials from Prof. Wayne Tai Lee's STAT5206 homepage.