-
Notifications
You must be signed in to change notification settings - Fork 0
/
lin_reg_exercise_solutions.py
215 lines (125 loc) · 7.11 KB
/
lin_reg_exercise_solutions.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
# coding: utf-8
# ___
#
# <a href='http://www.pieriandata.com'> <img src='../Pierian_Data_Logo.png' /></a>
# ___
# # Linear Regression - Project Exercise
#
# Congratulations! You just got some contract work with an Ecommerce company based in New York City that sells clothing online but they also have in-store style and clothing advice sessions. Customers come in to the store, have sessions/meetings with a personal stylist, then they can go home and order either on a mobile app or website for the clothes they want.
#
# The company is trying to decide whether to focus their efforts on their mobile app experience or their website. They've hired you on contract to help them figure it out! Let's get started!
#
# Just follow the steps below to analyze the customer data (it's fake, don't worry I didn't give you real credit card numbers or emails).
# ## Imports
# ** Import pandas, numpy, matplotlib,and seaborn. Then set %matplotlib inline
# (You'll import sklearn as you need it.)**
# In[275]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
# ## Get the Data
#
# We'll work with the Ecommerce Customers csv file from the company. It has Customer info, suchas Email, Address, and their color Avatar. Then it also has numerical value columns:
#
# * Avg. Session Length: Average session of in-store style advice sessions.
# * Time on App: Average time spent on App in minutes
# * Time on Website: Average time spent on Website in minutes
# * Length of Membership: How many years the customer has been a member.
#
# ** Read in the Ecommerce Customers csv file as a DataFrame called customers.**
customers = pd.read_csv("Ecommerce Customers")
# **Check the head of customers, and check out its info() and describe() methods.**
customers.head()
customers.describe()
customers.info()
sns.set_palette("GnBu_d")
sns.set_style('whitegrid')
# More time on site, more money spent.
sns.jointplot(x='Time on Website',y='Yearly Amount Spent',data=customers)
# ** Do the same but with the Time on App column instead. **
sns.jointplot(x='Time on App',y='Yearly Amount Spent',data=customers)
# ** Use jointplot to create a 2D hex bin plot comparing Time on App and Length of Membership.**
sns.jointplot(x='Time on App',y='Length of Membership',kind='hex',data=customers)
# **Let's explore these types of relationships across the entire data set. Use [pairplot](https://stanford.edu/~mwaskom/software/seaborn/tutorial/axis_grids.html#plotting-pairwise-relationships-with-pairgrid-and-pairplot) to recreate the plot below.(Don't worry about the the colors)**
sns.pairplot(customers)
# **Based off this plot what looks to be the most correlated feature with Yearly Amount Spent?**
# Length of Membership
# **Create a linear model plot (using seaborn's lmplot) of Yearly Amount Spent vs. Length of Membership. **
sns.lmplot(x='Length of Membership',y='Yearly Amount Spent',data=customers)
# ## Training and Testing Data
#
# Now that we've explored the data a bit, let's go ahead and split the data into training and testing sets.
# ** Set a variable X equal to the numerical features of the customers and a variable y equal to the "Yearly Amount Spent" column. **
y = customers['Yearly Amount Spent']
# In[288]:
X = customers[['Avg. Session Length', 'Time on App','Time on Website', 'Length of Membership']]
# ** Use model_selection.train_test_split from sklearn to split the data into training and testing sets. Set test_size=0.3 and random_state=101**
from sklearn.model_selection import train_test_split
# In[290]:
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)
# ## Training the Model
#
# Now its time to train our model on our training data!
#
# ** Import LinearRegression from sklearn.linear_model **
# In[291]:
from sklearn.linear_model import LinearRegression
# **Create an instance of a LinearRegression() model named lm.**
# In[292]:
lm = LinearRegression()
# ** Train/fit lm on the training data.**
# In[293]:
lm.fit(X_train,y_train)
# **Print out the coefficients of the model**
# In[294]:
# The coefficients
print('Coefficients: \n', lm.coef_)
# ## Predicting Test Data
# Now that we have fit our model, let's evaluate its performance by predicting off the test values!
#
# ** Use lm.predict() to predict off the X_test set of the data.**
# In[295]:
predictions = lm.predict( X_test)
# ** Create a scatterplot of the real test values versus the predicted values. **
# In[296]:
plt.scatter(y_test,predictions)
plt.xlabel('Y Test')
plt.ylabel('Predicted Y')
# ## Evaluating the Model
#
# Let's evaluate our model performance by calculating the residual sum of squares and the explained variance score (R^2).
#
# ** Calculate the Mean Absolute Error, Mean Squared Error, and the Root Mean Squared Error. Refer to the lecture or to Wikipedia for the formulas**
# calculate these metrics by hand!
from sklearn import metrics
print('MAE:', metrics.mean_absolute_error(y_test, predictions))
print('MSE:', metrics.mean_squared_error(y_test, predictions))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, predictions)))
# ## Residuals
#
# You should have gotten a very good model with a good fit. Let's quickly explore the residuals to make sure everything was okay with our data.
#
# **Plot a histogram of the residuals and make sure it looks normally distributed. Use either seaborn distplot, or just plt.hist().**
sns.distplot((y_test-predictions),bins=50);
# ## Conclusion
# We still want to figure out the answer to the original question, do we focus our effors on mobile app or website development? Or maybe that doesn't even really matter, and Membership Time is what is really important. Let's see if we can interpret the coefficients at all to get an idea.
#
# ** Recreate the dataframe below. **
coeffecients = pd.DataFrame(lm.coef_,X.columns)
coeffecients.columns = ['Coeffecient']
coeffecients
# ** How can you interpret these coefficients? **
# Interpreting the coefficients:
#
# - Holding all other features fixed, a 1 unit increase in **Avg. Session Length** is associated with an **increase of 25.98 total dollars spent**.
# - Holding all other features fixed, a 1 unit increase in **Time on App** is associated with an **increase of 38.59 total dollars spent**.
# - Holding all other features fixed, a 1 unit increase in **Time on Website** is associated with an **increase of 0.19 total dollars spent**.
# - Holding all other features fixed, a 1 unit increase in **Length of Membership** is associated with an **increase of 61.27 total dollars spent**.
# **Do you think the company should focus more on their mobile app or on their website?**
#
# This is tricky, there are two ways to think about this: Develop the Website to catch up to the performance of the mobile app, or develop the app more since that is what is working better. This sort of answer really depends on the other factors going on at the company, you would probably want to explore the relationship between Length of Membership and the App or the Website before coming to a conclusion!
#
# ## Great Job!
#
# Congrats on your contract work! The company loved the insights! Let's move on.