02_python/226@lhm.ipynb

{"cells": [{"cell_type": "markdown", "id": "db7f08c1", "metadata": {}, "source": ["## Open Government Data, provided by **Landeshauptstadt M\u00fcnchen**\n", "*Autogenerated Python starter code for data set with identifier* **226@lhm**"]}, {"cell_type": "markdown", "id": "164c4327", "metadata": {}, "source": ["## Dataset\n", "# **Daten der Raddauerz\u00e4hlstellen M\u00fcnchen April 2018**"]}, {"cell_type": "markdown", "id": "cad813bb-c986-4bb4-b52b-f78bb608086f", "metadata": {}, "source": ["## Data set links\n", "\n", "[Direct data shop link for dataset](https://opendata.muenchen.de/dataset/daten-der-raddauerzaehlstellen-muenchen-april-2018)"]}, {"cell_type": "markdown", "id": "9d4813e9", "metadata": {}, "source": ["## Metadata\n", "- **Issued** `2018-09-10T09:40:57`\n- **Modified** `2021-09-15T07:14:07`\n- **Startdate** `None`\n- **Enddate** `None`\n- **Theme** `['http://publications.europa.eu/resource/authority/data-theme/TECH']`\n- **Keyword** `['Fahrrad', 'Raddauerz\u00e4hlstellen', 'Raddauerz\u00e4hlstellen 2018']`\n- **Publisher** `['mobilitaetsreferat']`\n- **Landingpage** `https://opendata.muenchen.de/dataset/daten-der-raddauerzaehlstellen-muenchen-april-2018`\n"]}, {"cell_type": "markdown", "id": "8a857d65", "metadata": {"jp-MarkdownHeadingCollapsed": true, "tags": []}, "source": ["## Imports and helper functions"]}, {"cell_type": "code", "execution_count": null, "id": "93b39602-1c1e-46d2-ae70-1716b1481e9b", "metadata": {"tags": []}, "outputs": [], "source": ["%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "plt.style.use('ggplot')\n", "\n", "params = {\n", "    'text.color': (0.25, 0.25, 0.25),\n", "    'figure.figsize': [18, 6],\n", "   }\n", "\n", "plt.rcParams.update(params)\n", "\n", "import pandas as pd "]}, {"cell_type": "code", "execution_count": null, "id": "aa6611d7-e1c0-40a7-b0ff-601a8ef8439b", "metadata": {"tags": []}, "outputs": [], "source": ["# helper function for reading datasets with proper separator\n", "def get_dataset(url):\n", "    if url[-3:] != \"csv\":\n", "        print(\"The data set URL has no proper 'csv' extension. Reading the dataset might not have worked as expected.\\nPlease check the dataset link and adjust pandas' read_csv() parameters accordingly.\")\n", "    data = pd.read_csv(url, sep=\",\", on_bad_lines='warn', encoding_errors='ignore', low_memory=False)\n", "    # if dataframe only has one column or less the data is not comma separated, use \";\" instead\n", "    if data.shape[1] <= 1:\n", "        data = pd.read_csv(url, sep=';', on_bad_lines='warn', encoding_errors='ignore', low_memory=False)\n", "        if data.shape[1] <= 1:\n", "            print(\"The data wasn't imported properly. Very likely the correct separator couldn't be found.\\nPlease check the dataset manually and adjust the code.\")\n", "    return data"]}, {"cell_type": "markdown", "id": "02ce518f", "metadata": {}, "source": ["## Load data\n", "\n", "- The dataset has **`2` distribution(s)** in CSV format.\n", "- All available CSV distributions are listed below and can be read into a pandas dataframe."]}, {"cell_type": "code", "execution_count": null, "id": "0", "metadata": {"tags": []}, "outputs": [], "source": "# Distribution 0\n# Ktzhdistid               : 2261\n# Title                    : 15 Minuten-Werte - Daten der Raddauerz\u00e4hlstellen M\u00fcnchen April 2018\n# Issued                   : 2018-09-10T11:41:44\n# Modified                 : 2018-09-10T09:41:44\n# Rights                   : dl-by-de/2.0\n\ndf = get_dataset('https://opendata.muenchen.de/dataset/4f982f60-73e1-4b9f-95cd-cdf83635c3c7/resource/725b8a9b-78c8-4f93-b5a9-3f1a0bd766d8/download/rad20180415min.csv')\n\n# Distribution 1\n# Ktzhdistid               : 2262\n# Title                    : Tageswerte und Wetter - Daten der Raddauerz\u00e4hlstellen M\u00fcnchen April 2018\n# Issued                   : 2018-09-10T11:42:19\n# Modified                 : 2018-09-10T09:42:19\n# Rights                   : dl-by-de/2.0\n\ndf = get_dataset('https://opendata.muenchen.de/dataset/4f982f60-73e1-4b9f-95cd-cdf83635c3c7/resource/68e6a9e0-2b8c-4e03-9e19-c0b6e09405ae/download/rad201804tage.csv')\n\n"}, {"cell_type": "markdown", "id": "4ce1f78f", "metadata": {}, "source": ["## Analyze data"]}, {"cell_type": "code", "execution_count": null, "id": "3e3dab86", "metadata": {}, "outputs": [], "source": ["# drop columns that have no values\n", "df.dropna(how='all', axis=1, inplace=True)"]}, {"cell_type": "code", "execution_count": null, "id": "841bd8d2", "metadata": {}, "outputs": [], "source": ["print(f'The dataset has {df.shape[0]:,.0f} rows (observations) and {df.shape[1]:,.0f} columns (variables).')\n", "print(f'There seem to be {df.duplicated().sum()} exact duplicates in the data.')"]}, {"cell_type": "code", "execution_count": null, "id": "75e73c96", "metadata": {}, "outputs": [], "source": ["df.info(memory_usage='deep', verbose=True)"]}, {"cell_type": "code", "execution_count": null, "id": "02f3df4d", "metadata": {}, "outputs": [], "source": ["df.head()"]}, {"cell_type": "code", "execution_count": null, "id": "a0d7d898", "metadata": {}, "outputs": [], "source": ["# display a small random sample transposed in order to see all variables\n", "df.sample(3).T"]}, {"cell_type": "code", "execution_count": null, "id": "786806dd", "metadata": {}, "outputs": [], "source": ["# describe non-numerical features\n", "try:\n", "    with pd.option_context('display.float_format', '{:,.2f}'.format):\n", "        display(df.describe(exclude='number'))\n", "except:\n", "    print(\"No categorical data in dataset.\")"]}, {"cell_type": "code", "execution_count": null, "id": "e744a6b6", "metadata": {}, "outputs": [], "source": ["# describe numerical features\n", "try:\n", "    with pd.option_context('display.float_format', '{:,.2f}'.format):\n", "        display(df.describe(include='number'))\n", "except:\n", "    print(\"No numercial data in dataset.\")"]}, {"cell_type": "code", "execution_count": null, "id": "7a65d95d", "metadata": {}, "outputs": [], "source": ["# check missing values with missingno\n", "# https://github.com/ResidentMario/missingno\n", "import missingno as msno\n", "msno.matrix(df, labels=True, sort='descending');"]}, {"cell_type": "code", "execution_count": null, "id": "fcc604b7", "metadata": {}, "outputs": [], "source": ["# plot a histogram for each numerical feature\n", "try:\n", "    df.hist(bins=25, rwidth=.9)\n", "    plt.tight_layout()\n", "    plt.show()\n", "except:\n", "    print(\"No numercial data to plot.\") "]}, {"cell_type": "code", "execution_count": null, "id": "e13cc6eb", "metadata": {}, "outputs": [], "source": ["# continue your code here..."]}, {"cell_type": "code", "execution_count": null, "id": "18886378", "metadata": {}, "outputs": [], "source": []}, {"cell_type": "code", "execution_count": null, "id": "59fce071", "metadata": {}, "outputs": [], "source": []}, {"cell_type": "code", "execution_count": null, "id": "f42f25aa", "metadata": {}, "outputs": [], "source": []}, {"attachments": {}, "cell_type": "markdown", "id": "c6b87d83", "metadata": {}, "source": ["**Questions about the data?** Mobilit\u00e4tsreferat - Verkehrs- und Verhaltensdaten | Data Shop | "]}], "metadata": {"kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.15"}}, "nbformat": 4, "nbformat_minor": 5}