Lab: Titanic EDA

This week was all about Pandas and plotting. At this point you should be chomping at the bit to get your hands dirty on a real-world dataset.

For this lab, we're going to take a look at the Titanic manifest. We'll be exploring this data to see what we can learn regarding the survival rates of different groups of people.

Prework

Fork and clone this repo.

Step 1: Reading the data

1. Go to https://www.kaggle.com/c/titanic/data
2. If you scroll down the page a bit, you'll see a data dictionary explaining each of the columns. Take a minute to familiarize yourself with how the csv is structured.
3. Download the train.csv file into this lab folder.
4. Open the starter_code notebook

Step 2: Cleaning the data

1. Create a bar chart showing how many missing values are in each column

• Bonus : Theres a good library for visualizing missing values called Missingno.
  • Install Instructions
  • Usage Documentation

2. Which column has the most NaN values? How many cells in that column are empty?
3. Delete all rows where Embarked is empty
4. Fill all empty cabins with ¯\(ツ)/¯

Note: NaN, empty, and missing are synonymous.

Step 3: Feature extraction

1. There are two columns that pertain to how many family members are on the boat for a given person. Create a new column called FamilyCount which will be the sum of those two columns.
2. Reverends have a special title in their name. Create a column called IsReverend: 1 if they're a preacher, 0 if they're not.
3. In order to feed our training data into a classification algorithm, we need to convert our categories into 1's and 0's using pd.get_dummies

• Familiarize yourself with the pd.get_dummies documentation
• Create 3 columns: Embarked_C, Embarked_Q and Embarked_S. These columns will have 1's and 0's that correspond to the C, Q and S values in the Embarked column
• Do the same thing for Sex
• BONUS: Extract the title from everyone's name and create dummy columns

Step 4: Exploratory analysis

df.groupby() may be very useful.

1. What was the survival rate overall?
2. Which gender fared the worst? What was their survival rate?
3. What was the survival rate for each Pclass?
4. Did any reverends survive? How many?
5. What is the survival rate for cabins marked ¯\(ツ)/¯
6. What is the survival rate for people whose Age is empty?
7. What is the survival rate for each port of embarkation?
8. What is the survival rate for children (under 12) in each Pclass?
9. Did the captain of the ship survive? Is he on the list?
10. Of all the people that died, who had the most expensive ticket? How much did it cost?
11. Does having family on the boat help or hurt your chances of survival?

Step 5: Plotting

Using Matplotlib and Seaborn, create several charts showing the survival rates of different groups of people. It's fine if a handful of charts are basic (Gender, Age, etc), but what we're really looking for is something beneath the surface.