Skip to content

Commit e945ff4

Browse files
committed
update
1 parent 1a8267a commit e945ff4

5 files changed

Lines changed: 657 additions & 273 deletions

File tree

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
:::: {.content-visible when-profile="fr"}
2+
3+
::: {.exercise}
4+
## Exercice 1 (optionnel): construire la base de données
5+
6+
__Cet exercice est OPTIONNEL__
7+
8+
1. Télécharger et importer le shapefile [depuis ce lien](https://www2.census.gov/geo/tiger/GENZ2019/shp/cb_2019_02_sldl_500k.zip)
9+
2. Exclure les Etats suivants : "02", "69", "66", "78", "60", "72", "15"
10+
3. Importer les résultats des élections depuis [ce lien](https://raw.githubusercontent.com/tonmcg/US_County_Level_Election_Results_08-20/master/2020_US_County_Level_Presidential_Results.csv
11+
)
12+
4. Importer les bases disponibles sur le site de l'USDA en faisant attention à renommer les variables de code FIPS de manière identique
13+
dans les 4 bases
14+
5. *Merger* ces 4 bases dans une base unique de caractéristiques socioéconomiques
15+
6. *Merger* aux données électorales à partir du code FIPS
16+
7. *Merger* au shapefile à partir du code FIPS. Faire attention aux 0 à gauche dans certains codes. Il est
17+
recommandé d'utiliser la méthode `str.lstrip` pour les retirer
18+
8. Importer les données des élections 2000 à 2016 à partir du [MIT Election Lab](https://electionlab.mit.edu/data)?
19+
Les données peuvent être directement requêtées depuis l'url
20+
<https://dataverse.harvard.edu/api/access/datafile/3641280?gbrecs=false>
21+
9. Créer une variable `share` comptabilisant la part des votes pour chaque candidat.
22+
Ne garder que les colonnes `"year", "FIPS", "party", "candidatevotes", "share"`
23+
10. Faire une conversion `long` to `wide` avec la méthode `pivot_table` pour garder une ligne
24+
par comté x année avec en colonnes les résultats de chaque candidat dans cet état.
25+
11. Merger à partir du code FIPS au reste de la base.
26+
27+
:::
28+
29+
Si vous ne faites pas l'exercice 1, pensez à charger les données en executant la fonction `get_data.py` :
30+
31+
:::
32+
33+
:::: {.content-visible when-profile="en"}
34+
35+
::: {.exercise}
36+
## Exercise 1 (Optional): Build the Database
37+
38+
__This exercise is OPTIONAL__
39+
40+
1. Download and import the shapefile [from this link](https://www2.census.gov/geo/tiger/GENZ2019/shp/cb_2019_02_sldl_500k.zip)
41+
2. Exclude the following states: "02", "69", "66", "78", "60", "72", "15"
42+
3. Import election results from [this link](https://raw.githubusercontent.com/tonmcg/US_County_Level_Election_Results_08-20/master/2020_US_County_Level_Presidential_Results.csv)
43+
4. Import the datasets available on the USDA site, ensuring that FIPS code variables are named consistently
44+
across all 4 datasets
45+
5. *Merge* these 4 datasets into a single socioeconomic characteristics dataset
46+
6. *Merge* with the election data using the FIPS code
47+
7. *Merge* with the shapefile using the FIPS code. Be mindful of leading zeros in some codes. It is
48+
recommended to use the `str.lstrip` method to remove them
49+
8. Import election data from 2000 to 2016 from the [MIT Election Lab](https://electionlab.mit.edu/data).
50+
The data can be directly queried from this URL:
51+
<https://dataverse.harvard.edu/api/access/datafile/3641280?gbrecs=false>
52+
9. Create a `share` variable to account for each candidate's vote share.
53+
Keep only the columns `"year", "FIPS", "party", "candidatevotes", "share"`
54+
10. Perform a `long` to `wide` conversion using the `pivot_table` method to keep one row
55+
per county x year with each candidate's results in columns for that state.
56+
11. Merge with the rest of the dataset using the FIPS code.
57+
58+
:::
59+
60+
::::
61+
62+
```{python}
63+
#| echo: true
64+
#| output: false
65+
import requests
66+
67+
url = 'https://raw.githubusercontent.com/linogaliana/python-datascientist/main/content/modelisation/get_data.py'
68+
r = requests.get(url, allow_redirects=True)
69+
open('getdata.py', 'wb').write(r.content)
70+
71+
import getdata
72+
votes = getdata.create_votes_dataframes()
73+
```
74+
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
:::: {.content-visible when-profile="fr"}
2+
3+
::: {.exercise}
4+
## Exercice 2 (optionnel) : Regarder les corrélations entre les variables
5+
6+
__Cet exercice est OPTIONNEL__
7+
8+
1. Créer un DataFrame `df2` plus petit avec les variables `winner`, `votes_gop`, `Unemployment_rate_2019`,
9+
`Median_Household_Income_2019`,
10+
`Percent of adults with less than a high school diploma, 2015-19`,
11+
`Percent of adults with a bachelor's degree or higher, 2015-19`
12+
2. Représenter grâce à un graphique la matrice de corrélation. Vous pouvez utiliser le _package_ `seaborn` et sa fonction `heatmap`.
13+
3. Représenter une matrice de nuages de points des variables de la base `df2` avec `pd.plotting.scatter_matrix`
14+
4. (optionnel) Refaire ces figures avec `Plotly` qui offre également la possibilité de faire une matrice de corrélation.
15+
16+
:::
17+
::::
18+
19+
:::: {.content-visible when-profile="en"}
20+
21+
::: {.exercise}
22+
## Exercise 2 (Optional): Examining Correlations Between Variables
23+
24+
__This exercise is OPTIONAL__
25+
26+
1. Create a smaller DataFrame `df2` with the variables `winner`, `votes_gop`, `Unemployment_rate_2019`,
27+
`Median_Household_Income_2019`,
28+
`Percent of adults with less than a high school diploma, 2015-19`,
29+
`Percent of adults with a bachelor's degree or higher, 2015-19`.
30+
2. Use a graph to represent the correlation matrix. You can use the `seaborn` package and its `heatmap` function.
31+
3. Display a scatter plot matrix of the variables in `df2` using `pd.plotting.scatter_matrix`.
32+
4. (Optional) Recreate these figures using `Plotly`, which also offers the option to create a correlation matrix.
33+
34+
:::
35+
36+
::::
37+
38+
39+
```{python}
40+
#| output: false
41+
#| echo: true
42+
43+
# 1. Créer le data.frame df2.
44+
df2 = votes.set_index("GEOID").loc[
45+
: ,
46+
[
47+
"winner", "votes_gop",
48+
'Unemployment_rate_2019', 'Median_Household_Income_2019',
49+
'Percent of adults with less than a high school diploma, 2015-19',
50+
"Percent of adults with a bachelor's degree or higher, 2015-19"
51+
]
52+
]
53+
```
54+
55+
56+
::: {.content-visible when-profile="fr"}
57+
La matrice construite avec `seaborn` (question 2) aura l'aspect suivant :
58+
:::
59+
60+
::: {.content-visible when-profile="en"}
61+
Matrix created with `seaborn` (question 2) will look as follows:
62+
:::
63+
64+
```{python}
65+
import numpy as np
66+
import matplotlib.pyplot as plt
67+
import seaborn as sns
68+
69+
corr = df2.drop("winner", axis = 1).corr()
70+
71+
mask = np.zeros_like(corr, dtype=bool)
72+
mask[np.triu_indices_from(mask)] = True
73+
74+
# Set up the matplotlib figure
75+
fig = plt.figure()
76+
77+
# Generate a custom diverging colormap
78+
cmap = sns.diverging_palette(220, 10, as_cmap=True)
79+
80+
# Draw the heatmap with the mask and correct aspect ratio
81+
sns.heatmap(
82+
corr,
83+
mask=mask, # Mask upper triangular matrix
84+
cmap=cmap,
85+
annot=True
86+
vmax=.3,
87+
vmin=-.3,
88+
center=0, # The center value of the legend. With divergent cmap, where white is
89+
square=True, # Force cells to be square
90+
linewidths=.5, # Width of lines that divide cells
91+
cbar_kws={"shrink": .5} # Extra kwargs for the legend; in this case, shrink by 50%
92+
)
93+
94+
plt.show(fig)
95+
```
96+
97+
98+
::: {.content-visible when-profile="fr"}
99+
Alors que celle construite directement avec `corr` de `Pandas`
100+
ressemblera plutôt à ce tableau :
101+
:::
102+
103+
::: {.content-visible when-profile="en"}
104+
105+
Whereas the one constructed directly with `corr` from `Pandas` will look more like this table:
106+
:::
107+
108+
109+
110+
```{python}
111+
#| output: false
112+
#| echo: true
113+
# Construction directement avec pandas également possible
114+
g2 = df2.drop("winner", axis = 1).corr().style.background_gradient(cmap='coolwarm').format('{:.2f}')
115+
```
116+
117+
```{python}
118+
g2
119+
```
120+
121+
::: {.content-visible when-profile="fr"}
122+
Le nuage de point obtenu à l'issue de la question 3 ressemblera à :
123+
:::
124+
125+
::: {.content-visible when-profile="en"}
126+
The scatter plot obtained after question 3 will look like:
127+
:::
128+
129+
130+
```{python}
131+
#| echo: true
132+
# 3. Matrice de nuages de points
133+
pd.plotting.scatter_matrix(df2)
134+
```
135+
136+
::: {.content-visible when-profile="fr"}
137+
Le résultat de la question 4 devrait, quant à lui,
138+
ressembler au graphique suivant :
139+
:::
140+
141+
::: {.content-visible when-profile="en"}
142+
The result of question 4, on the other hand, should look like the following chart:
143+
:::
144+
145+
146+
```{python}
147+
#| echo: true
148+
# 4. Matrice de corrélation avec plotly
149+
import plotly
150+
import plotly.express as px
151+
htmlsnip2 = px.scatter_matrix(df2)
152+
htmlsnip2.update_traces(diagonal_visible=False)
153+
htmlsnip2.show()
154+
```

0 commit comments

Comments
 (0)