Notebook/Skolregistret Stockholms stadsarkiv.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "214957a4",
   "metadata": {},
   "source": [
    "# Skolregistret Stockholms stadsarkiv\n",
    "**Status**: started test webcrawl, version 0.7\n",
    "\n",
    "**Task:** del i [#1 Svenska skolor i Wikidata och i Bildhistoria.se](https://github.com/perrohdin/Bildhistoria-Masterdata-och-Wikidata/issues/1)\n",
    "att kunna hämta ut skolor från Wikidata... status idag är att Wikidata har bra information om Skolenheter \n",
    "* se WikiProject Sweden Schools\n",
    "* äldre skolor/skolformer är det sämre se WikiProject Svenska skolor \n",
    "   \n",
    " \n",
    "* [Denna Notebook](https://github.com/perrohdin/Bildhistoria-Masterdata-och-Wikidata/blob/main/Notebook/Skolregistret%20Stockholms%20stadsarkiv.ipynb)\n",
    " * [skapad csv fil](https://github.com/perrohdin/Bildhistoria-Masterdata-och-Wikidata/blob/main/Notebook/SkolregistretStockholmsStadsarkiv.csv) \n",
    "   * [small csv](https://github.com/perrohdin/Bildhistoria-Masterdata-och-Wikidata/blob/main/Notebook/SkolregistretStockholmsStadsarkiv_small.csv) \n",
    "   * [extra allt csv](https://github.com/perrohdin/Bildhistoria-Masterdata-och-Wikidata/blob/main/Notebook/SkolregistretStockholmsStadsarkiv_allt.csv) \n",
    "   \n",
    "Databasen \"[Skolregistret Stockholms stadsarkiv](https://sok.stadsarkivet.stockholm.se/Databas/skolregistret/Sok)\" copyright okänd, har besökt stadsarkivet och ombads skicka fråga med email 13 Maj, 09:07 , väntar svar\n",
    "* [sok.stadsarkivet.stockholm.se/Databas/skolregistret](https://sok.stadsarkivet.stockholm.se/Databas/skolregistret/Sok) 1354 poster\n",
    "* status licens inget svar efter 12 dagar\n",
    "\n",
    "**Version history**\n",
    "* 0.2 fixed bug with matched classes\n",
    "* 0.3 extrahera SSA arkiv nummer och hämtade fler fält\n",
    "* 0.4 hämtar från Riksarkivet agentid och namn\n",
    "* 0.5 se om vi kan testa Riksarkivets API skickade även en fråga till dom hur vi får ut Sveriges alla skolor över tid från Arkiven se [Riksarkivet/dataplattform/issues/1](https://github.com/Riksarkivet/dataplattform/issues/1) se även fråga hur vi gör detta i WD [Talk:Q89788826](https://www.wikidata.org/wiki/Talk:Q89788826)\n",
    "* 0.6 fick XML [exempel från Riksarkivet](https://github.com/Riksarkivet/dataplattform/blob/main/docs/examples/common/refcode_to_pid.py) som testades - **merci** \n",
    "* 0.7 test pd.read_xml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "123cb5cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last run:  2022-05-18 16:14:20.037208\n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "start_time  = datetime.now()\n",
    "print(\"Last run: \", start_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a9e6de77",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1300/1300 [08:08<00:00,  2.66it/s]\n"
     ]
    }
   ],
   "source": [
    "import urllib3, re, time\n",
    "import pandas as pd \n",
    "from bs4 import BeautifulSoup\n",
    "from csv import DictReader, DictWriter\n",
    "from tqdm import tqdm\n",
    "req = urllib3.PoolManager()\n",
    "    \n",
    "urlbase = \"https://sok.stadsarkivet.stockholm.se/Databas/skolregistret/Sok?sidindex=\"\n",
    "skolKolumner = [\"Url\",\"Namn\",\"Alt. namn\",\"Byggår och arkitekt\",\"Annexskolor\",\n",
    "           \"Adress\",\"Arkiv på Stadsarkivet\",\"Rektorsområde\",\"Arkiv hos övriga\",\"Historik\"]\n",
    "dfTot = pd.DataFrame(columns=skolKolumner)\n",
    "for i in tqdm(range(0,1300)):\n",
    "    try:\n",
    "        url = urlbase + str(i)\n",
    "        res = req.request('GET', url)\n",
    "        soup = BeautifulSoup(res.data, 'html.parser')\n",
    "        classes = [\"arende-table-row arende-table-row-alternating\", \n",
    "                   \"arende-table-row arende-table-row-nonalternating\"]\n",
    "        contents = soup.find_all(class_ = classes)\n",
    "        i = 0 \n",
    "        for tablerow in contents:\n",
    "            for rowLink in tablerow.find_all('a', href=True):\n",
    "                schoolUrl = rowLink['href'].split(\"?sidindex\")[0]\n",
    "                namnUrl = rowLink.string\n",
    "                \n",
    "             \n",
    "            td_list = tablerow.find_all(\"td\")\n",
    "            namn = td_list[0].text.strip()\n",
    "            altnamn = td_list[1].text.strip()\n",
    "            adress = td_list[2].text.strip()\n",
    "            historik = td_list[3].text.strip()\n",
    "            #print(namn,altnamn,adress,historik)\n",
    "                \n",
    "            dfTot = dfTot.append({'Url': schoolUrl, \n",
    "                                  'Namn': namn, \n",
    "                                  'Alt. namn': altnamn,\n",
    "                                  'Historik':historik,\n",
    "                                  'Adress':adress\n",
    "                                  }, ignore_index=True)\n",
    "    except Exception as e: \n",
    "        print(e)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e0a3eaf4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1354, 10)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b07fd3aa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1354 entries, 0 to 1353\n",
      "Data columns (total 10 columns):\n",
      " #   Column                 Non-Null Count  Dtype \n",
      "---  ------                 --------------  ----- \n",
      " 0   Url                    1354 non-null   object\n",
      " 1   Namn                   1354 non-null   object\n",
      " 2   Alt. namn              1354 non-null   object\n",
      " 3   Byggår och arkitekt    0 non-null      object\n",
      " 4   Annexskolor            0 non-null      object\n",
      " 5   Adress                 1354 non-null   object\n",
      " 6   Arkiv på Stadsarkivet  0 non-null      object\n",
      " 7   Rektorsområde          0 non-null      object\n",
      " 8   Arkiv hos övriga       0 non-null      object\n",
      " 9   Historik               1354 non-null   object\n",
      "dtypes: object(10)\n",
      "memory usage: 105.9+ KB\n"
     ]
    }
   ],
   "source": [
    "dfTot.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ee51c16a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Url</th>\n",
       "      <th>Namn</th>\n",
       "      <th>Alt. namn</th>\n",
       "      <th>Byggår och arkitekt</th>\n",
       "      <th>Annexskolor</th>\n",
       "      <th>Adress</th>\n",
       "      <th>Arkiv på Stadsarkivet</th>\n",
       "      <th>Rektorsområde</th>\n",
       "      <th>Arkiv hos övriga</th>\n",
       "      <th>Historik</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...</td>\n",
       "      <td>Carlgrenska Waldorfgymnasiet</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...</td>\n",
       "      <td>Larsboda ro  1960-1991</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Huvudskola:\\r\\nLarsbodaskolan 1960-1991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...</td>\n",
       "      <td>Elektrum</td>\n",
       "      <td>Electrum</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kistagången 14-18</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...</td>\n",
       "      <td>Slättgårdsskolan</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Frimurarvägen 11-17</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Visa/lovsatra-resursskola/de119201-dadb-4815-a...</td>\n",
       "      <td>Lövsätra resursskola</td>\n",
       "      <td>Skärholmens skoldaghem / Skärholmens resursskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1971-2002 Falkholmsgränd 41 (tidigare Stånghol...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1349</th>\n",
       "      <td>Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...</td>\n",
       "      <td>Kunskapsskolan Tyresö</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Solkraftsvägen 15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1350</th>\n",
       "      <td>Visa/sodermalms-privata-skola/0f0c2328-a79d-46...</td>\n",
       "      <td>Södermalms privata skola</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1938 anges Folkungagatan 55</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Privatskola</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1351</th>\n",
       "      <td>Visa/olof-fryxells-flickskola/7c1a21e8-8f27-45...</td>\n",
       "      <td>Olof Fryxells flickskola</td>\n",
       "      <td>Bataljonspredikanten Olof Fryxells flickskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Järntorget 80</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Anges finnas 1848 på given adress i Staden mel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1352</th>\n",
       "      <td>Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...</td>\n",
       "      <td>Martinskolan</td>\n",
       "      <td>Martinskolan Söders waldorfskola 7 / (Martinsk...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Munstycksvägen 18</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1353</th>\n",
       "      <td>Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...</td>\n",
       "      <td>Ekensbergsskolan</td>\n",
       "      <td>Ekensbergsskolans grundsärskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gröndalsvägen 153</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1354 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Url  \\\n",
       "0     Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...   \n",
       "1     Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...   \n",
       "2     Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...   \n",
       "3     Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...   \n",
       "4     Visa/lovsatra-resursskola/de119201-dadb-4815-a...   \n",
       "...                                                 ...   \n",
       "1349  Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...   \n",
       "1350  Visa/sodermalms-privata-skola/0f0c2328-a79d-46...   \n",
       "1351  Visa/olof-fryxells-flickskola/7c1a21e8-8f27-45...   \n",
       "1352  Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...   \n",
       "1353  Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...   \n",
       "\n",
       "                              Namn  \\\n",
       "0     Carlgrenska Waldorfgymnasiet   \n",
       "1           Larsboda ro  1960-1991   \n",
       "2                         Elektrum   \n",
       "3                 Slättgårdsskolan   \n",
       "4             Lövsätra resursskola   \n",
       "...                            ...   \n",
       "1349         Kunskapsskolan Tyresö   \n",
       "1350      Södermalms privata skola   \n",
       "1351      Olof Fryxells flickskola   \n",
       "1352                  Martinskolan   \n",
       "1353              Ekensbergsskolan   \n",
       "\n",
       "                                              Alt. namn Byggår och arkitekt  \\\n",
       "0                                                                       NaN   \n",
       "1                                                                       NaN   \n",
       "2                                              Electrum                 NaN   \n",
       "3                                                                       NaN   \n",
       "4      Skärholmens skoldaghem / Skärholmens resursskola                 NaN   \n",
       "...                                                 ...                 ...   \n",
       "1349                                                                    NaN   \n",
       "1350                                                                    NaN   \n",
       "1351      Bataljonspredikanten Olof Fryxells flickskola                 NaN   \n",
       "1352  Martinskolan Söders waldorfskola 7 / (Martinsk...                 NaN   \n",
       "1353                    Ekensbergsskolans grundsärskola                 NaN   \n",
       "\n",
       "     Annexskolor                                             Adress  \\\n",
       "0            NaN  Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...   \n",
       "1            NaN                                                      \n",
       "2            NaN                                  Kistagången 14-18   \n",
       "3            NaN                                Frimurarvägen 11-17   \n",
       "4            NaN  1971-2002 Falkholmsgränd 41 (tidigare Stånghol...   \n",
       "...          ...                                                ...   \n",
       "1349         NaN                                  Solkraftsvägen 15   \n",
       "1350         NaN                        1938 anges Folkungagatan 55   \n",
       "1351         NaN                                      Järntorget 80   \n",
       "1352         NaN                                  Munstycksvägen 18   \n",
       "1353         NaN                                  Gröndalsvägen 153   \n",
       "\n",
       "     Arkiv på Stadsarkivet Rektorsområde Arkiv hos övriga  \\\n",
       "0                      NaN           NaN              NaN   \n",
       "1                      NaN           NaN              NaN   \n",
       "2                      NaN           NaN              NaN   \n",
       "3                      NaN           NaN              NaN   \n",
       "4                      NaN           NaN              NaN   \n",
       "...                    ...           ...              ...   \n",
       "1349                   NaN           NaN              NaN   \n",
       "1350                   NaN           NaN              NaN   \n",
       "1351                   NaN           NaN              NaN   \n",
       "1352                   NaN           NaN              NaN   \n",
       "1353                   NaN           NaN              NaN   \n",
       "\n",
       "                                               Historik  \n",
       "0     Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...  \n",
       "1               Huvudskola:\\r\\nLarsbodaskolan 1960-1991  \n",
       "2     Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...  \n",
       "3     Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...  \n",
       "4     Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...  \n",
       "...                                                 ...  \n",
       "1349  Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...  \n",
       "1350                                        Privatskola  \n",
       "1351  Anges finnas 1848 på given adress i Staden mel...  \n",
       "1352  Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...  \n",
       "1353  Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...  \n",
       "\n",
       "[1354 rows x 10 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cff82e58",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfTot.to_csv(\"SkolregistretStockholmsStadsarkiv.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "91d978fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "#dfTot[{\"Namn\",\"Url\",\"Alt. namn\",\"Adress\"}].to_csv(\"SkolregistretStockholmsStadsarkiv_small.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f4c80da0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# IS A BUG preserving orders -->  https://stackoverflow.com/questions/15653688/preserving-column-order-in-python-pandas-dataframe\n",
    "dfTot.to_csv(\"SkolregistretStockholmsStadsarkiv_small.csv\",\n",
    "             header=True,columns=[\"Namn\",\"Url\",\"Alt. namn\",\"Adress\"],\n",
    "            sep=\",\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a7d34f6b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1354/1354 [08:02<00:00,  2.80it/s] \n"
     ]
    }
   ],
   "source": [
    "#Get more data \n",
    "urlBaseView =\"https://sok.stadsarkivet.stockholm.se/Databas/skolregistret/\"\n",
    "Viewids =[\"MainContent_alternativt-namn\" ,\"MainContent_namn\",\n",
    "        \"MainContent_adress\",\"MainContent_byggar-och-arkitekt\",\"MainContent_annexskolor\",\n",
    "        \"MainContent_arkiv-pa-stadsarkivet\",\"MainContent_arkiv-hos-ovriga\",\n",
    "        \"MainContent_historik\",\"MainContent_rektorsomrade\"]\n",
    "\n",
    "# Field\n",
    "contentList=[\n",
    "{\"field\":\"Namn\",\n",
    "\"content\":\"MainContent_Namn_visa_textbox\"},\n",
    "{\"field\":\"Alt. namn\",\n",
    "\"content\":\"MainContent_Alternativt namn_visa_textbox\"},\n",
    "{\"field\":\"Adress\",\n",
    "\"content\":\"MainContent_Adress_visa_textbox\"},\n",
    "{\"field\":\"Rektorsområde\",\n",
    "\"content\":\"MainContent_Rektorsområde_visa_textbox\"},\n",
    "{\"field\":\"Byggår och arkitekt\",\n",
    "\"content\":\"MainContent_Byggår och arkitekt_visa_textbox\"},\n",
    "{\"field\":\"Annexskolor\",\n",
    "\"content\":\"MainContent_Annexskolor_visa_textbox\"},\n",
    "{\"field\":\"Arkiv på Stadsarkivet\",\n",
    "\"content\":\"MainContent_Arkiv på Stadsarkivet_visa_textbox\"},\n",
    "{\"field\":\"Arkiv hos övriga\",\n",
    "\"content\":\"MainContent_Arkiv hos övriga_visa_textbox\"},\n",
    "{\"field\":\"Historik\",\n",
    "\"content\":\"MainContent_Historik_visa_textbox\"}]\n",
    "\n",
    "for index, row in tqdm(dfTot.iterrows(), total=dfTot.shape[0]):\n",
    "    itemUrl = urlBaseView + row[\"Url\"]\n",
    "    res = req.request('GET', itemUrl)\n",
    "    soupSchool = BeautifulSoup(res.data, 'html.parser')\n",
    "    for c in contentList:\n",
    "        tag = soupSchool.find(id=c[\"content\"])\n",
    "        if len(tag.text.strip()) > 0:\n",
    "            dfTot.loc[index,c[\"field\"]] = tag.text.strip()\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "39942f07",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1354, 10)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "87ed3a9d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Url</th>\n",
       "      <th>Namn</th>\n",
       "      <th>Alt. namn</th>\n",
       "      <th>Byggår och arkitekt</th>\n",
       "      <th>Annexskolor</th>\n",
       "      <th>Adress</th>\n",
       "      <th>Arkiv på Stadsarkivet</th>\n",
       "      <th>Rektorsområde</th>\n",
       "      <th>Arkiv hos övriga</th>\n",
       "      <th>Historik</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...</td>\n",
       "      <td>Carlgrenska Waldorfgymnasiet</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...</td>\n",
       "      <td>SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...</td>\n",
       "      <td>Larsboda ro  1960-1991</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hästhagsskolan 1975-1991, \\r\\nMagelungsskolan ...</td>\n",
       "      <td></td>\n",
       "      <td>SSA/1406: Larsboda ro 1960-1992.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Huvudskola:\\r\\nLarsbodaskolan 1960-1991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...</td>\n",
       "      <td>Elektrum</td>\n",
       "      <td>Electrum</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kistagången 14-18</td>\n",
       "      <td>SSA/1331: Vasa gymnasium 1987-1990.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Thorildsplans gymnasium 1990-1992 ?</td>\n",
       "      <td>Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...</td>\n",
       "      <td>Slättgårdsskolan</td>\n",
       "      <td></td>\n",
       "      <td>1968  Uddén-Wåhlström</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Frimurarvägen 11-17</td>\n",
       "      <td>SSA/2462: Mälarhöjdens ro 1966-1989.</td>\n",
       "      <td>Mälarhöjdens ro 1966-1989, \\r\\nBredängs ro 198...</td>\n",
       "      <td>Bredängsskolan (ro) 1989-1991. \\r\\nSlättgårdss...</td>\n",
       "      <td>Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Visa/lovsatra-resursskola/de119201-dadb-4815-a...</td>\n",
       "      <td>Lövsätra resursskola</td>\n",
       "      <td>Skärholmens skoldaghem / Skärholmens resursskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1971-2002 Falkholmsgränd 41 (tidigare Stånghol...</td>\n",
       "      <td>SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.</td>\n",
       "      <td>S:t Örjans ro 1971-1985, \\r\\nÖsterholms ro 198...</td>\n",
       "      <td>Österholmsskolan (ro) 1985-1990</td>\n",
       "      <td>Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1349</th>\n",
       "      <td>Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...</td>\n",
       "      <td>Kunskapsskolan Tyresö</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Solkraftsvägen 15</td>\n",
       "      <td>SSA/7093: e-arkiv: slutbetyg 2012-2021.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I övrigt på skolan.</td>\n",
       "      <td>Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1350</th>\n",
       "      <td>Visa/sodermalms-privata-skola/0f0c2328-a79d-46...</td>\n",
       "      <td>Södermalms privata skola</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1938 anges Folkungagatan 55</td>\n",
       "      <td>Folkskoledirektionen (SSA/0569) - E III a3 : d...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Privatskola</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1351</th>\n",
       "      <td>Visa/olof-fryxells-flickskola/7c1a21e8-8f27-45...</td>\n",
       "      <td>Olof Fryxells flickskola</td>\n",
       "      <td>Bataljonspredikanten Olof Fryxells flickskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Järntorget 80</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>okänt</td>\n",
       "      <td>Anges finnas 1848 på given adress i Staden mel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1352</th>\n",
       "      <td>Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...</td>\n",
       "      <td>Martinskolan</td>\n",
       "      <td>Martinskolan Söders waldorfskola 7 / (Martinsk...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Munstycksvägen 18</td>\n",
       "      <td>SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I övrigt på skolan.</td>\n",
       "      <td>Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1353</th>\n",
       "      <td>Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...</td>\n",
       "      <td>Ekensbergsskolan</td>\n",
       "      <td>Ekensbergsskolans grundsärskola</td>\n",
       "      <td>1980  Lars Bryde</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gröndalsvägen 153</td>\n",
       "      <td>SSA/2374: Ekensbergsskolan 1991-2013.</td>\n",
       "      <td>Aspuddens ro 1980-1990. \\r\\nGröndals ro 1990-1...</td>\n",
       "      <td>Aspuddens skola (ro) 1980-1990. \\r\\nGröndalssk...</td>\n",
       "      <td>Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1354 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Url  \\\n",
       "0     Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...   \n",
       "1     Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...   \n",
       "2     Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...   \n",
       "3     Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...   \n",
       "4     Visa/lovsatra-resursskola/de119201-dadb-4815-a...   \n",
       "...                                                 ...   \n",
       "1349  Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...   \n",
       "1350  Visa/sodermalms-privata-skola/0f0c2328-a79d-46...   \n",
       "1351  Visa/olof-fryxells-flickskola/7c1a21e8-8f27-45...   \n",
       "1352  Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...   \n",
       "1353  Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...   \n",
       "\n",
       "                              Namn  \\\n",
       "0     Carlgrenska Waldorfgymnasiet   \n",
       "1           Larsboda ro  1960-1991   \n",
       "2                         Elektrum   \n",
       "3                 Slättgårdsskolan   \n",
       "4             Lövsätra resursskola   \n",
       "...                            ...   \n",
       "1349         Kunskapsskolan Tyresö   \n",
       "1350      Södermalms privata skola   \n",
       "1351      Olof Fryxells flickskola   \n",
       "1352                  Martinskolan   \n",
       "1353              Ekensbergsskolan   \n",
       "\n",
       "                                              Alt. namn  \\\n",
       "0                                                         \n",
       "1                                                         \n",
       "2                                              Electrum   \n",
       "3                                                         \n",
       "4      Skärholmens skoldaghem / Skärholmens resursskola   \n",
       "...                                                 ...   \n",
       "1349                                                      \n",
       "1350                                                      \n",
       "1351      Bataljonspredikanten Olof Fryxells flickskola   \n",
       "1352  Martinskolan Söders waldorfskola 7 / (Martinsk...   \n",
       "1353                    Ekensbergsskolans grundsärskola   \n",
       "\n",
       "        Byggår och arkitekt  \\\n",
       "0                       NaN   \n",
       "1                       NaN   \n",
       "2                       NaN   \n",
       "3     1968  Uddén-Wåhlström   \n",
       "4                       NaN   \n",
       "...                     ...   \n",
       "1349                    NaN   \n",
       "1350                    NaN   \n",
       "1351                    NaN   \n",
       "1352                    NaN   \n",
       "1353       1980  Lars Bryde   \n",
       "\n",
       "                                            Annexskolor  \\\n",
       "0                                                   NaN   \n",
       "1     Hästhagsskolan 1975-1991, \\r\\nMagelungsskolan ...   \n",
       "2                                                   NaN   \n",
       "3                                                   NaN   \n",
       "4                                                   NaN   \n",
       "...                                                 ...   \n",
       "1349                                                NaN   \n",
       "1350                                                NaN   \n",
       "1351                                                NaN   \n",
       "1352                                                NaN   \n",
       "1353                                                NaN   \n",
       "\n",
       "                                                 Adress  \\\n",
       "0     Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...   \n",
       "1                                                         \n",
       "2                                     Kistagången 14-18   \n",
       "3                                   Frimurarvägen 11-17   \n",
       "4     1971-2002 Falkholmsgränd 41 (tidigare Stånghol...   \n",
       "...                                                 ...   \n",
       "1349                                  Solkraftsvägen 15   \n",
       "1350                        1938 anges Folkungagatan 55   \n",
       "1351                                      Järntorget 80   \n",
       "1352                                  Munstycksvägen 18   \n",
       "1353                                  Gröndalsvägen 153   \n",
       "\n",
       "                                  Arkiv på Stadsarkivet  \\\n",
       "0     SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...   \n",
       "1                      SSA/1406: Larsboda ro 1960-1992.   \n",
       "2                   SSA/1331: Vasa gymnasium 1987-1990.   \n",
       "3                  SSA/2462: Mälarhöjdens ro 1966-1989.   \n",
       "4     SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.   \n",
       "...                                                 ...   \n",
       "1349            SSA/7093: e-arkiv: slutbetyg 2012-2021.   \n",
       "1350  Folkskoledirektionen (SSA/0569) - E III a3 : d...   \n",
       "1351                                                NaN   \n",
       "1352  SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...   \n",
       "1353              SSA/2374: Ekensbergsskolan 1991-2013.   \n",
       "\n",
       "                                          Rektorsområde  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                                                   NaN   \n",
       "3     Mälarhöjdens ro 1966-1989, \\r\\nBredängs ro 198...   \n",
       "4     S:t Örjans ro 1971-1985, \\r\\nÖsterholms ro 198...   \n",
       "...                                                 ...   \n",
       "1349                                                NaN   \n",
       "1350                                                NaN   \n",
       "1351                                                NaN   \n",
       "1352                                                NaN   \n",
       "1353  Aspuddens ro 1980-1990. \\r\\nGröndals ro 1990-1...   \n",
       "\n",
       "                                       Arkiv hos övriga  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                   Thorildsplans gymnasium 1990-1992 ?   \n",
       "3     Bredängsskolan (ro) 1989-1991. \\r\\nSlättgårdss...   \n",
       "4                       Österholmsskolan (ro) 1985-1990   \n",
       "...                                                 ...   \n",
       "1349                                I övrigt på skolan.   \n",
       "1350                                                NaN   \n",
       "1351                                              okänt   \n",
       "1352                                I övrigt på skolan.   \n",
       "1353  Aspuddens skola (ro) 1980-1990. \\r\\nGröndalssk...   \n",
       "\n",
       "                                               Historik  \n",
       "0     Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...  \n",
       "1               Huvudskola:\\r\\nLarsbodaskolan 1960-1991  \n",
       "2     Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...  \n",
       "3     Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...  \n",
       "4     Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...  \n",
       "...                                                 ...  \n",
       "1349  Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...  \n",
       "1350                                        Privatskola  \n",
       "1351  Anges finnas 1848 på given adress i Staden mel...  \n",
       "1352  Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...  \n",
       "1353  Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...  \n",
       "\n",
       "[1354 rows x 10 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "7fd683a5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...\n",
       "1                        SSA/1406: Larsboda ro 1960-1992.\n",
       "2                     SSA/1331: Vasa gymnasium 1987-1990.\n",
       "3                    SSA/2462: Mälarhöjdens ro 1966-1989.\n",
       "4       SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.\n",
       "                              ...                        \n",
       "1349              SSA/7093: e-arkiv: slutbetyg 2012-2021.\n",
       "1350    Folkskoledirektionen (SSA/0569) - E III a3 : d...\n",
       "1351                                                  NaN\n",
       "1352    SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...\n",
       "1353                SSA/2374: Ekensbergsskolan 1991-2013.\n",
       "Name: Arkiv på Stadsarkivet, Length: 1354, dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot[\"Arkiv på Stadsarkivet\"] \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "536c6c2e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Url</th>\n",
       "      <th>Namn</th>\n",
       "      <th>Alt. namn</th>\n",
       "      <th>Byggår och arkitekt</th>\n",
       "      <th>Annexskolor</th>\n",
       "      <th>Adress</th>\n",
       "      <th>Arkiv på Stadsarkivet</th>\n",
       "      <th>Rektorsområde</th>\n",
       "      <th>Arkiv hos övriga</th>\n",
       "      <th>Historik</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...</td>\n",
       "      <td>Carlgrenska Waldorfgymnasiet</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...</td>\n",
       "      <td>SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...</td>\n",
       "      <td>Larsboda ro  1960-1991</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hästhagsskolan 1975-1991, \\r\\nMagelungsskolan ...</td>\n",
       "      <td></td>\n",
       "      <td>SSA/1406: Larsboda ro 1960-1992.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Huvudskola:\\r\\nLarsbodaskolan 1960-1991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...</td>\n",
       "      <td>Elektrum</td>\n",
       "      <td>Electrum</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kistagången 14-18</td>\n",
       "      <td>SSA/1331: Vasa gymnasium 1987-1990.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Thorildsplans gymnasium 1990-1992 ?</td>\n",
       "      <td>Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...</td>\n",
       "      <td>Slättgårdsskolan</td>\n",
       "      <td></td>\n",
       "      <td>1968  Uddén-Wåhlström</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Frimurarvägen 11-17</td>\n",
       "      <td>SSA/2462: Mälarhöjdens ro 1966-1989.</td>\n",
       "      <td>Mälarhöjdens ro 1966-1989, \\r\\nBredängs ro 198...</td>\n",
       "      <td>Bredängsskolan (ro) 1989-1991. \\r\\nSlättgårdss...</td>\n",
       "      <td>Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Visa/lovsatra-resursskola/de119201-dadb-4815-a...</td>\n",
       "      <td>Lövsätra resursskola</td>\n",
       "      <td>Skärholmens skoldaghem / Skärholmens resursskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1971-2002 Falkholmsgränd 41 (tidigare Stånghol...</td>\n",
       "      <td>SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.</td>\n",
       "      <td>S:t Örjans ro 1971-1985, \\r\\nÖsterholms ro 198...</td>\n",
       "      <td>Österholmsskolan (ro) 1985-1990</td>\n",
       "      <td>Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1348</th>\n",
       "      <td>Visa/kungsholmens-skola/67ff287c-0dae-484d-be2...</td>\n",
       "      <td>Kungsholmens skola</td>\n",
       "      <td>Kungsholmens folkskola; Kungsholms folkskola</td>\n",
       "      <td>se Historik mm !</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Mariebergsgatan 34</td>\n",
       "      <td>SSA/0232A: Kungsholms folkskolor/ro 1847-1973....</td>\n",
       "      <td>Kungsholms församling -1903, \\r\\nKungsholms öd...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Folkskola, Kommunal mellanskola, Enhetsskola, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1349</th>\n",
       "      <td>Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...</td>\n",
       "      <td>Kunskapsskolan Tyresö</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Solkraftsvägen 15</td>\n",
       "      <td>SSA/7093: e-arkiv: slutbetyg 2012-2021.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I övrigt på skolan.</td>\n",
       "      <td>Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1350</th>\n",
       "      <td>Visa/sodermalms-privata-skola/0f0c2328-a79d-46...</td>\n",
       "      <td>Södermalms privata skola</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1938 anges Folkungagatan 55</td>\n",
       "      <td>Folkskoledirektionen (SSA/0569) - E III a3 : d...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Privatskola</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1352</th>\n",
       "      <td>Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...</td>\n",
       "      <td>Martinskolan</td>\n",
       "      <td>Martinskolan Söders waldorfskola 7 / (Martinsk...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Munstycksvägen 18</td>\n",
       "      <td>SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I övrigt på skolan.</td>\n",
       "      <td>Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1353</th>\n",
       "      <td>Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...</td>\n",
       "      <td>Ekensbergsskolan</td>\n",
       "      <td>Ekensbergsskolans grundsärskola</td>\n",
       "      <td>1980  Lars Bryde</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gröndalsvägen 153</td>\n",
       "      <td>SSA/2374: Ekensbergsskolan 1991-2013.</td>\n",
       "      <td>Aspuddens ro 1980-1990. \\r\\nGröndals ro 1990-1...</td>\n",
       "      <td>Aspuddens skola (ro) 1980-1990. \\r\\nGröndalssk...</td>\n",
       "      <td>Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>898 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Url  \\\n",
       "0     Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...   \n",
       "1     Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...   \n",
       "2     Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...   \n",
       "3     Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...   \n",
       "4     Visa/lovsatra-resursskola/de119201-dadb-4815-a...   \n",
       "...                                                 ...   \n",
       "1348  Visa/kungsholmens-skola/67ff287c-0dae-484d-be2...   \n",
       "1349  Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...   \n",
       "1350  Visa/sodermalms-privata-skola/0f0c2328-a79d-46...   \n",
       "1352  Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...   \n",
       "1353  Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...   \n",
       "\n",
       "                              Namn  \\\n",
       "0     Carlgrenska Waldorfgymnasiet   \n",
       "1           Larsboda ro  1960-1991   \n",
       "2                         Elektrum   \n",
       "3                 Slättgårdsskolan   \n",
       "4             Lövsätra resursskola   \n",
       "...                            ...   \n",
       "1348            Kungsholmens skola   \n",
       "1349         Kunskapsskolan Tyresö   \n",
       "1350      Södermalms privata skola   \n",
       "1352                  Martinskolan   \n",
       "1353              Ekensbergsskolan   \n",
       "\n",
       "                                              Alt. namn  \\\n",
       "0                                                         \n",
       "1                                                         \n",
       "2                                              Electrum   \n",
       "3                                                         \n",
       "4      Skärholmens skoldaghem / Skärholmens resursskola   \n",
       "...                                                 ...   \n",
       "1348       Kungsholmens folkskola; Kungsholms folkskola   \n",
       "1349                                                      \n",
       "1350                                                      \n",
       "1352  Martinskolan Söders waldorfskola 7 / (Martinsk...   \n",
       "1353                    Ekensbergsskolans grundsärskola   \n",
       "\n",
       "        Byggår och arkitekt  \\\n",
       "0                       NaN   \n",
       "1                       NaN   \n",
       "2                       NaN   \n",
       "3     1968  Uddén-Wåhlström   \n",
       "4                       NaN   \n",
       "...                     ...   \n",
       "1348       se Historik mm !   \n",
       "1349                    NaN   \n",
       "1350                    NaN   \n",
       "1352                    NaN   \n",
       "1353       1980  Lars Bryde   \n",
       "\n",
       "                                            Annexskolor  \\\n",
       "0                                                   NaN   \n",
       "1     Hästhagsskolan 1975-1991, \\r\\nMagelungsskolan ...   \n",
       "2                                                   NaN   \n",
       "3                                                   NaN   \n",
       "4                                                   NaN   \n",
       "...                                                 ...   \n",
       "1348                                                NaN   \n",
       "1349                                                NaN   \n",
       "1350                                                NaN   \n",
       "1352                                                NaN   \n",
       "1353                                                NaN   \n",
       "\n",
       "                                                 Adress  \\\n",
       "0     Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...   \n",
       "1                                                         \n",
       "2                                     Kistagången 14-18   \n",
       "3                                   Frimurarvägen 11-17   \n",
       "4     1971-2002 Falkholmsgränd 41 (tidigare Stånghol...   \n",
       "...                                                 ...   \n",
       "1348                                 Mariebergsgatan 34   \n",
       "1349                                  Solkraftsvägen 15   \n",
       "1350                        1938 anges Folkungagatan 55   \n",
       "1352                                  Munstycksvägen 18   \n",
       "1353                                  Gröndalsvägen 153   \n",
       "\n",
       "                                  Arkiv på Stadsarkivet  \\\n",
       "0     SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...   \n",
       "1                      SSA/1406: Larsboda ro 1960-1992.   \n",
       "2                   SSA/1331: Vasa gymnasium 1987-1990.   \n",
       "3                  SSA/2462: Mälarhöjdens ro 1966-1989.   \n",
       "4     SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.   \n",
       "...                                                 ...   \n",
       "1348  SSA/0232A: Kungsholms folkskolor/ro 1847-1973....   \n",
       "1349            SSA/7093: e-arkiv: slutbetyg 2012-2021.   \n",
       "1350  Folkskoledirektionen (SSA/0569) - E III a3 : d...   \n",
       "1352  SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...   \n",
       "1353              SSA/2374: Ekensbergsskolan 1991-2013.   \n",
       "\n",
       "                                          Rektorsområde  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                                                   NaN   \n",
       "3     Mälarhöjdens ro 1966-1989, \\r\\nBredängs ro 198...   \n",
       "4     S:t Örjans ro 1971-1985, \\r\\nÖsterholms ro 198...   \n",
       "...                                                 ...   \n",
       "1348  Kungsholms församling -1903, \\r\\nKungsholms öd...   \n",
       "1349                                                NaN   \n",
       "1350                                                NaN   \n",
       "1352                                                NaN   \n",
       "1353  Aspuddens ro 1980-1990. \\r\\nGröndals ro 1990-1...   \n",
       "\n",
       "                                       Arkiv hos övriga  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                   Thorildsplans gymnasium 1990-1992 ?   \n",
       "3     Bredängsskolan (ro) 1989-1991. \\r\\nSlättgårdss...   \n",
       "4                       Österholmsskolan (ro) 1985-1990   \n",
       "...                                                 ...   \n",
       "1348                                                NaN   \n",
       "1349                                I övrigt på skolan.   \n",
       "1350                                                NaN   \n",
       "1352                                I övrigt på skolan.   \n",
       "1353  Aspuddens skola (ro) 1980-1990. \\r\\nGröndalssk...   \n",
       "\n",
       "                                               Historik  \n",
       "0     Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...  \n",
       "1               Huvudskola:\\r\\nLarsbodaskolan 1960-1991  \n",
       "2     Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...  \n",
       "3     Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...  \n",
       "4     Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...  \n",
       "...                                                 ...  \n",
       "1348  Folkskola, Kommunal mellanskola, Enhetsskola, ...  \n",
       "1349  Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...  \n",
       "1350                                        Privatskola  \n",
       "1352  Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...  \n",
       "1353  Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...  \n",
       "\n",
       "[898 rows x 10 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find columns with an Archive id SSA/\n",
    "dfTot[dfTot[\"Arkiv på Stadsarkivet\"].str.contains(\"SSA/\",na=False)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "8d2e7134",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfTot.to_csv(\"SkolregistretStockholmsStadsarkiv_allt.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "cebc0537",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfTot[\"SSA\"]=\"SSA/\" + dfTot[\"Arkiv på Stadsarkivet\"].str.extract('SSA/(\\d*)')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80fce741",
   "metadata": {},
   "source": [
    "### SSA numret används hos Riksarkivet\n",
    "SSA numret används hos Riksarkivet --> WD egenskap [Property:P5324](https://www.wikidata.org/wiki/Property:P5324) gissar att Riksarkivet inte har ett API men borde kunna gå att hämta [Property:P9713](https://www.wikidata.org/wiki/Property:P9713)\n",
    "\n",
    "Exempel \n",
    "* SSA/7131 --> [SE/SSA/7131](https://sok.riksarkivet.se/?postid=ArkisRef%20SE/SSA/7131) --> agent https://sok.riksarkivet.se/arkiv/OSjzkpBeXQPSC4OvnXVJg6\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9150bcb2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       SSA/7131\n",
       "1       SSA/1406\n",
       "2       SSA/1331\n",
       "3       SSA/2462\n",
       "4       SSA/2060\n",
       "          ...   \n",
       "1349    SSA/7093\n",
       "1350    SSA/0569\n",
       "1351         NaN\n",
       "1352    SSA/7059\n",
       "1353    SSA/2374\n",
       "Name: SSA, Length: 1354, dtype: object"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot[\"SSA\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "86f67246",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfTot.to_csv(\"SkolregistretStockholmsStadsarkiv_allt2.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1f0d359e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1354 entries, 0 to 1353\n",
      "Data columns (total 11 columns):\n",
      " #   Column                 Non-Null Count  Dtype \n",
      "---  ------                 --------------  ----- \n",
      " 0   Url                    1354 non-null   object\n",
      " 1   Namn                   1354 non-null   object\n",
      " 2   Alt. namn              1354 non-null   object\n",
      " 3   Byggår och arkitekt    325 non-null    object\n",
      " 4   Annexskolor            204 non-null    object\n",
      " 5   Adress                 1354 non-null   object\n",
      " 6   Arkiv på Stadsarkivet  947 non-null    object\n",
      " 7   Rektorsområde          262 non-null    object\n",
      " 8   Arkiv hos övriga       731 non-null    object\n",
      " 9   Historik               1354 non-null   object\n",
      " 10  SSA                    898 non-null    object\n",
      "dtypes: object(11)\n",
      "memory usage: 116.5+ KB\n"
     ]
    }
   ],
   "source": [
    "dfTot.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e61dce1",
   "metadata": {},
   "source": [
    "### Test hämta Riksarkivets agentID och arkiv namn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3310aa7c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1354/1354 [12:17<00:00,  1.83it/s]\n"
     ]
    }
   ],
   "source": [
    "baseUrlRiksarkiv = \"https://sok.riksarkivet.se/?postid=ArkisRef%20\"\n",
    "\n",
    "def isnotNaN(value):\n",
    "    return value == value\n",
    "\n",
    "for index, row in tqdm(dfTot.iterrows(), total=dfTot.shape[0]):\n",
    "    if isnotNaN(row[\"SSA\"]):\n",
    "        urlRA = baseUrlRiksarkiv + \"SE/\" + row[\"SSA\"]\n",
    "     \n",
    "        try: \n",
    "            resRA = req.request('GET', urlRA)\n",
    "            #print(\"\\tStatus:\", resRA.status)\n",
    "            #Check if more than 3000 character Riksarkivet skickar alltid status 200...\n",
    "            if len(resRA.data) > 3000:\n",
    "                soupRA = BeautifulSoup(resRA.data, 'html.parser')\n",
    "                title = soupRA.find_all(\"h1\")\n",
    "                dfTot.loc[index,\"RAarkivnamn\"] = title\n",
    "\n",
    "                copyBtn = soupRA.find(\"i\",class_=\"copy-btn\")\n",
    "                if copyBtn['data-copy-value'] is not None:\n",
    "                    agentid = copyBtn['data-copy-value'].replace(\"https://sok.riksarkivet.se/arkiv/\",\"\")\n",
    "                    dfTot.loc[index,\"agentid\"] = agentid\n",
    "                else:\n",
    "                    print(\"\\tError agent is None: \",urlRA)\n",
    " \n",
    "        except Exception as e: \n",
    "            print( e)\n",
    "            print(urlRA)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "f3643989",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Url</th>\n",
       "      <th>Namn</th>\n",
       "      <th>Alt. namn</th>\n",
       "      <th>Byggår och arkitekt</th>\n",
       "      <th>Annexskolor</th>\n",
       "      <th>Adress</th>\n",
       "      <th>Arkiv på Stadsarkivet</th>\n",
       "      <th>Rektorsområde</th>\n",
       "      <th>Arkiv hos övriga</th>\n",
       "      <th>Historik</th>\n",
       "      <th>SSA</th>\n",
       "      <th>RAarkivnamn</th>\n",
       "      <th>agentid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...</td>\n",
       "      <td>Carlgrenska Waldorfgymnasiet</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...</td>\n",
       "      <td>SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...</td>\n",
       "      <td>SSA/7131</td>\n",
       "      <td>[\\r\\nCarlgrenska Waldorfgymnasiet\\r\\n        ]</td>\n",
       "      <td>OSjzkpBeXQPSC4OvnXVJg6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...</td>\n",
       "      <td>Larsboda ro  1960-1991</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>Hästhagsskolan 1975-1991, \\r\\nMagelungsskolan ...</td>\n",
       "      <td></td>\n",
       "      <td>SSA/1406: Larsboda ro 1960-1992.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Huvudskola:\\r\\nLarsbodaskolan 1960-1991</td>\n",
       "      <td>SSA/1406</td>\n",
       "      <td>[\\r\\nLarsboda rektorsområde\\r\\n        ]</td>\n",
       "      <td>NSaspJigr4AywCL8qpLfp6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...</td>\n",
       "      <td>Elektrum</td>\n",
       "      <td>Electrum</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kistagången 14-18</td>\n",
       "      <td>SSA/1331: Vasa gymnasium 1987-1990.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Thorildsplans gymnasium 1990-1992 ?</td>\n",
       "      <td>Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...</td>\n",
       "      <td>SSA/1331</td>\n",
       "      <td>[\\r\\nVasa gymnasium\\r\\n        ]</td>\n",
       "      <td>VZ6k9IgabKQ8ySqb1EfBm3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...</td>\n",
       "      <td>Slättgårdsskolan</td>\n",
       "      <td></td>\n",
       "      <td>1968  Uddén-Wåhlström</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Frimurarvägen 11-17</td>\n",
       "      <td>SSA/2462: Mälarhöjdens ro 1966-1989.</td>\n",
       "      <td>Mälarhöjdens ro 1966-1989, \\r\\nBredängs ro 198...</td>\n",
       "      <td>Bredängsskolan (ro) 1989-1991. \\r\\nSlättgårdss...</td>\n",
       "      <td>Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...</td>\n",
       "      <td>SSA/2462</td>\n",
       "      <td>[\\r\\nMälarhöjdens rektorsområde\\r\\n        ]</td>\n",
       "      <td>a82JBOhltasB7JdRHUzTQ4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Visa/lovsatra-resursskola/de119201-dadb-4815-a...</td>\n",
       "      <td>Lövsätra resursskola</td>\n",
       "      <td>Skärholmens skoldaghem / Skärholmens resursskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1971-2002 Falkholmsgränd 41 (tidigare Stånghol...</td>\n",
       "      <td>SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.</td>\n",
       "      <td>S:t Örjans ro 1971-1985, \\r\\nÖsterholms ro 198...</td>\n",
       "      <td>Österholmsskolan (ro) 1985-1990</td>\n",
       "      <td>Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...</td>\n",
       "      <td>SSA/2060</td>\n",
       "      <td>[\\r\\nSankt Örjans skolor\\r\\n        ]</td>\n",
       "      <td>yvFak6FyxD7jGnHCzFdDwK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1349</th>\n",
       "      <td>Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...</td>\n",
       "      <td>Kunskapsskolan Tyresö</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Solkraftsvägen 15</td>\n",
       "      <td>SSA/7093: e-arkiv: slutbetyg 2012-2021.</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I övrigt på skolan.</td>\n",
       "      <td>Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...</td>\n",
       "      <td>SSA/7093</td>\n",
       "      <td>[\\r\\nKunskapsskolan Tyresö\\r\\n        ]</td>\n",
       "      <td>QT9IqGdcYKw30Ej4NiZHv2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1350</th>\n",
       "      <td>Visa/sodermalms-privata-skola/0f0c2328-a79d-46...</td>\n",
       "      <td>Södermalms privata skola</td>\n",
       "      <td></td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1938 anges Folkungagatan 55</td>\n",
       "      <td>Folkskoledirektionen (SSA/0569) - E III a3 : d...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Privatskola</td>\n",
       "      <td>SSA/0569</td>\n",
       "      <td>[\\r\\nStockholms folkskoledirektion\\r\\n        ]</td>\n",
       "      <td>nbLG5ZXzDAN2NpJx6FBIs5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1351</th>\n",
       "      <td>Visa/olof-fryxells-flickskola/7c1a21e8-8f27-45...</td>\n",
       "      <td>Olof Fryxells flickskola</td>\n",
       "      <td>Bataljonspredikanten Olof Fryxells flickskola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Järntorget 80</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>okänt</td>\n",
       "      <td>Anges finnas 1848 på given adress i Staden mel...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1352</th>\n",
       "      <td>Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...</td>\n",
       "      <td>Martinskolan</td>\n",
       "      <td>Martinskolan Söders waldorfskola 7 / (Martinsk...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Munstycksvägen 18</td>\n",
       "      <td>SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>I övrigt på skolan.</td>\n",
       "      <td>Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...</td>\n",
       "      <td>SSA/7059</td>\n",
       "      <td>[\\r\\nMartinskolan\\r\\n        ]</td>\n",
       "      <td>nek8SGk6lKoEANtUXfRJl3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1353</th>\n",
       "      <td>Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...</td>\n",
       "      <td>Ekensbergsskolan</td>\n",
       "      <td>Ekensbergsskolans grundsärskola</td>\n",
       "      <td>1980  Lars Bryde</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gröndalsvägen 153</td>\n",
       "      <td>SSA/2374: Ekensbergsskolan 1991-2013.</td>\n",
       "      <td>Aspuddens ro 1980-1990. \\r\\nGröndals ro 1990-1...</td>\n",
       "      <td>Aspuddens skola (ro) 1980-1990. \\r\\nGröndalssk...</td>\n",
       "      <td>Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...</td>\n",
       "      <td>SSA/2374</td>\n",
       "      <td>[\\r\\nEkensbergsskolan 1991-2013\\r\\n        ]</td>\n",
       "      <td>oTbTH4mSvaAw7jvDVzZDJ2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1354 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Url  \\\n",
       "0     Visa/carlgrenska-waldorfgymnasiet/bf4aeb94-88b...   \n",
       "1     Visa/larsboda-ro--1960-1991/7bb2e5c7-e0aa-4754...   \n",
       "2     Visa/elektrum/5240aebb-c129-4615-9474-006c5b4e...   \n",
       "3     Visa/slattgardsskolan/51e42e66-35c7-4a49-9355-...   \n",
       "4     Visa/lovsatra-resursskola/de119201-dadb-4815-a...   \n",
       "...                                                 ...   \n",
       "1349  Visa/kunskapsskolan-tyreso/e2dbbd7c-a06a-47ce-...   \n",
       "1350  Visa/sodermalms-privata-skola/0f0c2328-a79d-46...   \n",
       "1351  Visa/olof-fryxells-flickskola/7c1a21e8-8f27-45...   \n",
       "1352  Visa/martinskolan/fcb2590d-bda7-4c70-9616-fee8...   \n",
       "1353  Visa/ekensbergsskolan/63a521ef-92ee-452f-8cec-...   \n",
       "\n",
       "                              Namn  \\\n",
       "0     Carlgrenska Waldorfgymnasiet   \n",
       "1           Larsboda ro  1960-1991   \n",
       "2                         Elektrum   \n",
       "3                 Slättgårdsskolan   \n",
       "4             Lövsätra resursskola   \n",
       "...                            ...   \n",
       "1349         Kunskapsskolan Tyresö   \n",
       "1350      Södermalms privata skola   \n",
       "1351      Olof Fryxells flickskola   \n",
       "1352                  Martinskolan   \n",
       "1353              Ekensbergsskolan   \n",
       "\n",
       "                                              Alt. namn  \\\n",
       "0                                                         \n",
       "1                                                         \n",
       "2                                              Electrum   \n",
       "3                                                         \n",
       "4      Skärholmens skoldaghem / Skärholmens resursskola   \n",
       "...                                                 ...   \n",
       "1349                                                      \n",
       "1350                                                      \n",
       "1351      Bataljonspredikanten Olof Fryxells flickskola   \n",
       "1352  Martinskolan Söders waldorfskola 7 / (Martinsk...   \n",
       "1353                    Ekensbergsskolans grundsärskola   \n",
       "\n",
       "        Byggår och arkitekt  \\\n",
       "0                       NaN   \n",
       "1                       NaN   \n",
       "2                       NaN   \n",
       "3     1968  Uddén-Wåhlström   \n",
       "4                       NaN   \n",
       "...                     ...   \n",
       "1349                    NaN   \n",
       "1350                    NaN   \n",
       "1351                    NaN   \n",
       "1352                    NaN   \n",
       "1353       1980  Lars Bryde   \n",
       "\n",
       "                                            Annexskolor  \\\n",
       "0                                                   NaN   \n",
       "1     Hästhagsskolan 1975-1991, \\r\\nMagelungsskolan ...   \n",
       "2                                                   NaN   \n",
       "3                                                   NaN   \n",
       "4                                                   NaN   \n",
       "...                                                 ...   \n",
       "1349                                                NaN   \n",
       "1350                                                NaN   \n",
       "1351                                                NaN   \n",
       "1352                                                NaN   \n",
       "1353                                                NaN   \n",
       "\n",
       "                                                 Adress  \\\n",
       "0     Skebokvarnsvägen 370\\r\\ntidigare Tegeluddsväge...   \n",
       "1                                                         \n",
       "2                                     Kistagången 14-18   \n",
       "3                                   Frimurarvägen 11-17   \n",
       "4     1971-2002 Falkholmsgränd 41 (tidigare Stånghol...   \n",
       "...                                                 ...   \n",
       "1349                                  Solkraftsvägen 15   \n",
       "1350                        1938 anges Folkungagatan 55   \n",
       "1351                                      Järntorget 80   \n",
       "1352                                  Munstycksvägen 18   \n",
       "1353                                  Gröndalsvägen 153   \n",
       "\n",
       "                                  Arkiv på Stadsarkivet  \\\n",
       "0     SSA/7131: e-arkiv: slutbetyg 2016-2019. Vissa ...   \n",
       "1                      SSA/1406: Larsboda ro 1960-1992.   \n",
       "2                   SSA/1331: Vasa gymnasium 1987-1990.   \n",
       "3                  SSA/2462: Mälarhöjdens ro 1966-1989.   \n",
       "4     SSA/2060: S:t Örjans skolor 1971-1985, 1991-2009.   \n",
       "...                                                 ...   \n",
       "1349            SSA/7093: e-arkiv: slutbetyg 2012-2021.   \n",
       "1350  Folkskoledirektionen (SSA/0569) - E III a3 : d...   \n",
       "1351                                                NaN   \n",
       "1352  SSA/7059: e-arkiv: slutbetyg 2012-2021. Lucka ...   \n",
       "1353              SSA/2374: Ekensbergsskolan 1991-2013.   \n",
       "\n",
       "                                          Rektorsområde  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                                                   NaN   \n",
       "3     Mälarhöjdens ro 1966-1989, \\r\\nBredängs ro 198...   \n",
       "4     S:t Örjans ro 1971-1985, \\r\\nÖsterholms ro 198...   \n",
       "...                                                 ...   \n",
       "1349                                                NaN   \n",
       "1350                                                NaN   \n",
       "1351                                                NaN   \n",
       "1352                                                NaN   \n",
       "1353  Aspuddens ro 1980-1990. \\r\\nGröndals ro 1990-1...   \n",
       "\n",
       "                                       Arkiv hos övriga  \\\n",
       "0                                                   NaN   \n",
       "1                                                   NaN   \n",
       "2                   Thorildsplans gymnasium 1990-1992 ?   \n",
       "3     Bredängsskolan (ro) 1989-1991. \\r\\nSlättgårdss...   \n",
       "4                       Österholmsskolan (ro) 1985-1990   \n",
       "...                                                 ...   \n",
       "1349                                I övrigt på skolan.   \n",
       "1350                                                NaN   \n",
       "1351                                              okänt   \n",
       "1352                                I övrigt på skolan.   \n",
       "1353  Aspuddens skola (ro) 1980-1990. \\r\\nGröndalssk...   \n",
       "\n",
       "                                               Historik       SSA  \\\n",
       "0     Fristående gymnasieskola\\r\\n\\r\\nÖppnad ht 2013...  SSA/7131   \n",
       "1               Huvudskola:\\r\\nLarsbodaskolan 1960-1991  SSA/1406   \n",
       "2     Gymnasium\\r\\n\\r\\nAnnex till Vasa gymnasium 198...  SSA/1331   \n",
       "3     Grundskola åk 1-9 (före ht 2005 1-6)\\r\\n\\r\\nEg...  SSA/2462   \n",
       "4     Specialskola: Grundskola åk 1-6\\r\\n\\r\\nÖppnad ...  SSA/2060   \n",
       "...                                                 ...       ...   \n",
       "1349  Fristående grundskola åk 6-9.\\r\\n\\r\\nSkolan st...  SSA/7093   \n",
       "1350                                        Privatskola  SSA/0569   \n",
       "1351  Anges finnas 1848 på given adress i Staden mel...       NaN   \n",
       "1352  Fristående Waldorfskola åk F-9.\\r\\nHuvudman, S...  SSA/7059   \n",
       "1353  Grundskola åk F-6 (1-3 2000- )\\r\\nGrundsärskol...  SSA/2374   \n",
       "\n",
       "                                          RAarkivnamn                 agentid  \n",
       "0      [\\r\\nCarlgrenska Waldorfgymnasiet\\r\\n        ]  OSjzkpBeXQPSC4OvnXVJg6  \n",
       "1            [\\r\\nLarsboda rektorsområde\\r\\n        ]  NSaspJigr4AywCL8qpLfp6  \n",
       "2                    [\\r\\nVasa gymnasium\\r\\n        ]  VZ6k9IgabKQ8ySqb1EfBm3  \n",
       "3        [\\r\\nMälarhöjdens rektorsområde\\r\\n        ]  a82JBOhltasB7JdRHUzTQ4  \n",
       "4               [\\r\\nSankt Örjans skolor\\r\\n        ]  yvFak6FyxD7jGnHCzFdDwK  \n",
       "...                                               ...                     ...  \n",
       "1349          [\\r\\nKunskapsskolan Tyresö\\r\\n        ]  QT9IqGdcYKw30Ej4NiZHv2  \n",
       "1350  [\\r\\nStockholms folkskoledirektion\\r\\n        ]  nbLG5ZXzDAN2NpJx6FBIs5  \n",
       "1351                                              NaN                     NaN  \n",
       "1352                   [\\r\\nMartinskolan\\r\\n        ]  nek8SGk6lKoEANtUXfRJl3  \n",
       "1353     [\\r\\nEkensbergsskolan 1991-2013\\r\\n        ]  oTbTH4mSvaAw7jvDVzZDJ2  \n",
       "\n",
       "[1354 rows x 13 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfTot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "116e9491",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1354 entries, 0 to 1353\n",
      "Data columns (total 13 columns):\n",
      " #   Column                 Non-Null Count  Dtype \n",
      "---  ------                 --------------  ----- \n",
      " 0   Url                    1354 non-null   object\n",
      " 1   Namn                   1354 non-null   object\n",
      " 2   Alt. namn              1354 non-null   object\n",
      " 3   Byggår och arkitekt    325 non-null    object\n",
      " 4   Annexskolor            204 non-null    object\n",
      " 5   Adress                 1354 non-null   object\n",
      " 6   Arkiv på Stadsarkivet  947 non-null    object\n",
      " 7   Rektorsområde          262 non-null    object\n",
      " 8   Arkiv hos övriga       731 non-null    object\n",
      " 9   Historik               1354 non-null   object\n",
      " 10  SSA                    898 non-null    object\n",
      " 11  RAarkivnamn            796 non-null    object\n",
      " 12  agentid                796 non-null    object\n",
      "dtypes: object(13)\n",
      "memory usage: 137.6+ KB\n"
     ]
    }
   ],
   "source": [
    "dfTot.info()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "2170b6dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "dfTot.to_csv(\"SkolregistretStockholmsStadsarkiv_allt2.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "cc097a9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ended:  2022-05-18 16:42:52.074275\n",
      "Time elapsed (hh:mm:ss.ms) 0:28:32.047416\n"
     ]
    }
   ],
   "source": [
    "end = datetime.now()\n",
    "print(\"Ended: \", end) \n",
    "print('Time elapsed (hh:mm:ss.ms) {}'.format(datetime.now() - start_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9707bdf",
   "metadata": {},
   "source": [
    "### Riksarkivet har nytt API !!!\n",
    "* GITHUB [Riksarkivet/dataplattform/wiki/OAI-PMH](https://github.com/Riksarkivet/dataplattform/wiki/OAI-PMH)\n",
    "* exempel https://oai-pmh.riksarkivet.se/OAI?verb=GetRecord&identifier=SE/SSA/7131&metadataPrefix=oai_ape_ead\n",
    "\n",
    "verkar dock vara XML ...\n",
    "\n",
    "* skickade in en fråga om dom har någon struktur så man kan hämta ut alla arkiv om skolor se [Riksarkivet/dataplattform/issues/1](https://github.com/Riksarkivet/dataplattform/issues/1)\n",
    "  * diskussion i WD hur vi hanterar folksskolor, realskolor etc [länk](https://www.wikidata.org/wiki/Talk:Q89788826)\n",
    "  * finns nu detta [exempel](https://github.com/Riksarkivet/dataplattform/blob/main/docs/examples/common/refcode_to_pid.py) se min test nedan\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2f0a146b",
   "metadata": {},
   "outputs": [],
   "source": [
    "testurl = \"https://oai-pmh.riksarkivet.se/OAI?verb=GetRecord&identifier=SE/SSA/7131&metadataPrefix=oai_ape_ead\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "7f7ddc94",
   "metadata": {},
   "outputs": [],
   "source": [
    "import xml.etree.ElementTree as ET"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "5ca7612a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "odict_keys(['OAI-PMH'])\n"
     ]
    }
   ],
   "source": [
    "import xmltodict\n",
    "resRAxml = req.request('GET', testurl)\n",
    "xmldict = xmltodict.parse(resRAxml.data)\n",
    "print(xmldict.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "c286de84",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "odict_keys(['@xmlns', '@xmlns:xsi', '@xsi:schemaLocation', 'responseDate', 'request', 'GetRecord'])\n"
     ]
    }
   ],
   "source": [
    "print(xmldict['OAI-PMH'].keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "7d2f2800",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Element '{http://www.openarchives.org/OAI/2.0/}OAI-PMH' at 0x7f93bd85e360> \n",
      "Tree.tag:\t {http://www.openarchives.org/OAI/2.0/}OAI-PMH\n",
      "NAD Ref : \t OSjzkpBeXQPSC4OvnXVJg6 \n",
      "URL :\t\t https://sok.riksarkivet.se/arkiv/OSjzkpBeXQPSC4OvnXVJg6\n"
     ]
    }
   ],
   "source": [
    "NADurlbase = \"https://sok.riksarkivet.se/arkiv/\"\n",
    "tree = ET.fromstring(resRAxml.data) \n",
    "print(tree,\"\\nTree.tag:\\t\",tree.tag)  \n",
    "ref = tree.find('.//{*}ead/{*}archdesc/{*}otherfindaid/{*}p/{*}extref')\n",
    "if ref is not None:\n",
    "    ref_url = ref.attrib.get('{http://www.w3.org/1999/xlink}href') \n",
    "    NADref = ref_url.split('/')[-1]\n",
    "    print(\"NAD Ref : \\t\", NADref, \"\\nURL :\\t\\t\", NADurlbase + NADref)\n",
    "else:\n",
    "    print('Cannot find PID reference')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "36e48403",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['{http://www.openarchives.org/OAI/2.0/}OAI-PMH',\n",
       " '{http://www.openarchives.org/OAI/2.0/}responseDate',\n",
       " '{http://www.openarchives.org/OAI/2.0/}request',\n",
       " '{http://www.openarchives.org/OAI/2.0/}GetRecord',\n",
       " '{http://www.openarchives.org/OAI/2.0/}record',\n",
       " '{http://www.openarchives.org/OAI/2.0/}header',\n",
       " '{http://www.openarchives.org/OAI/2.0/}identifier',\n",
       " '{http://www.openarchives.org/OAI/2.0/}datestamp',\n",
       " '{http://www.openarchives.org/OAI/2.0/}setSpec',\n",
       " '{http://www.openarchives.org/OAI/2.0/}metadata',\n",
       " '{urn:isbn:1-931666-22-9}ead',\n",
       " '{urn:isbn:1-931666-22-9}eadheader',\n",
       " '{urn:isbn:1-931666-22-9}eadid',\n",
       " '{urn:isbn:1-931666-22-9}filedesc',\n",
       " '{urn:isbn:1-931666-22-9}titlestmt',\n",
       " '{urn:isbn:1-931666-22-9}titleproper',\n",
       " '{urn:isbn:1-931666-22-9}profiledesc',\n",
       " '{urn:isbn:1-931666-22-9}creation',\n",
       " '{urn:isbn:1-931666-22-9}revisiondesc',\n",
       " '{urn:isbn:1-931666-22-9}change',\n",
       " '{urn:isbn:1-931666-22-9}date',\n",
       " '{urn:isbn:1-931666-22-9}item',\n",
       " '{urn:isbn:1-931666-22-9}change',\n",
       " '{urn:isbn:1-931666-22-9}date',\n",
       " '{urn:isbn:1-931666-22-9}item',\n",
       " '{urn:isbn:1-931666-22-9}change',\n",
       " '{urn:isbn:1-931666-22-9}date',\n",
       " '{urn:isbn:1-931666-22-9}item',\n",
       " '{urn:isbn:1-931666-22-9}archdesc',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}physdesc',\n",
       " '{urn:isbn:1-931666-22-9}extent',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}accessrestrict',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}userestrict',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}dsc',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}unitdate',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref',\n",
       " '{urn:isbn:1-931666-22-9}c',\n",
       " '{urn:isbn:1-931666-22-9}did',\n",
       " '{urn:isbn:1-931666-22-9}unitid',\n",
       " '{urn:isbn:1-931666-22-9}unittitle',\n",
       " '{urn:isbn:1-931666-22-9}scopecontent',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}otherfindaid',\n",
       " '{urn:isbn:1-931666-22-9}p',\n",
       " '{urn:isbn:1-931666-22-9}extref']"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[elem.tag for elem in tree.iter()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "df121581",
   "metadata": {},
   "outputs": [],
   "source": [
    "#print(ET.tostring(tree, encoding='utf8').decode('utf8'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "c7dc72e0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Element '{urn:isbn:1-931666-22-9}titlestmt' at 0x7f942a83b590> \n",
      "tag\t {} \n",
      "text\t None\n",
      "<Element '{urn:isbn:1-931666-22-9}titleproper' at 0x7f93bd860950> \n",
      "tag\t {} \n",
      "text\t Carlgrenska Waldorfgymnasiet\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd85aae0> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/OSjzkpBeXQPSC4OvnXVJg6'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd85ac70> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/wpwS3ukSMqMYeIwAHMp6a1'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd85aa90> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/ZebtNOcJkqs8zi9LtK9Z53'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd861e50> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/NFrOA2IkT4ItxMYva3yXo0'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd861450> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/EMthkbXKTaEum922DClaF0'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd844720> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/zjEnc4BT8APUm7a9NCU020'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd844860> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/C9f91RQ6Jqob4NMTzSh5x4'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd844c20> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/0lGVwtuqV4g0EoZF0HKes3'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd844950> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/H9Tr1iR654z415alMAjwO7'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd86cb30> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/vLX47mwhI4E5wpaBRlg2X2'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd86cea0> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/rTWaHOItKawDOwkkPNvAy7'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd865220> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/rChP0rQ8P4sbgUHbvnznE3'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd865590> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/SXKI5Fpg3aIGOBeW0tSXz7'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd865900> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/ztyUmlU0ygYKVV4sKFHmkO'} \n",
      "text\t Post i NAD\n",
      "<Element '{urn:isbn:1-931666-22-9}extref' at 0x7f93bd865bd0> \n",
      "tag\t {'{http://www.w3.org/1999/xlink}href': 'https://sok.riksarkivet.se/arkiv/4b1Fk8eQd4cQWCgNwcNBC3'} \n",
      "text\t Post i NAD\n"
     ]
    }
   ],
   "source": [
    "for title in tree.iter('{urn:isbn:1-931666-22-9}titlestmt'):\n",
    "    print(title,\"\\ntag\\t\" ,title.attrib,\"\\ntext\\t\" ,title.text)\n",
    "for titleprop in tree.iter('{urn:isbn:1-931666-22-9}titleproper'):\n",
    "    print(titleprop,\"\\ntag\\t\" ,titleprop.attrib,\"\\ntext\\t\" ,titleprop.text)\n",
    "for extref in tree.iter('{urn:isbn:1-931666-22-9}extref'):\n",
    "    print(extref,\"\\ntag\\t\" ,extref.attrib,\"\\ntext\\t\" ,extref.text)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22cbfcdf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tips finns en xmlviewer https://codebeautify.org/xmlviewer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3b9d534e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>responseDate</th>\n",
       "      <th>verb</th>\n",
       "      <th>identifier</th>\n",
       "      <th>metadataPrefix</th>\n",
       "      <th>request</th>\n",
       "      <th>record</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-05-18T16:37:51.4282869Z</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>None</td>\n",
       "      <td>GetRecord</td>\n",
       "      <td>SE/SSA/7131</td>\n",
       "      <td>oai_ape_ead</td>\n",
       "      <td>https://oai-pmh.riksarkivet.se/OAI</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   responseDate       verb   identifier metadataPrefix  \\\n",
       "0  2022-05-18T16:37:51.4282869Z       None         None           None   \n",
       "1                          None  GetRecord  SE/SSA/7131    oai_ape_ead   \n",
       "2                          None       None         None           None   \n",
       "\n",
       "                              request  record  \n",
       "0                                None     NaN  \n",
       "1  https://oai-pmh.riksarkivet.se/OAI     NaN  \n",
       "2                                None     NaN  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd \n",
    "pd.read_xml(testurl)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a941285",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "51abffe2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>record</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   record\n",
       "0     NaN"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:GetRecord\",\n",
    "            namespaces={\"doc\": \"http://www.openarchives.org/OAI/2.0/\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b33066b6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>header</th>\n",
       "      <th>metadata</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   header  metadata\n",
       "0     NaN       NaN"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:record\",\n",
    "            namespaces={\"doc\": \"http://www.openarchives.org/OAI/2.0/\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "7ce8ae92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ead</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ead\n",
       "0  NaN"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:metadata\",\n",
    "            namespaces={\"doc\": \"http://www.openarchives.org/OAI/2.0/\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "84943fc6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>responseDate</th>\n",
       "      <th>verb</th>\n",
       "      <th>identifier</th>\n",
       "      <th>metadataPrefix</th>\n",
       "      <th>request</th>\n",
       "      <th>record</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-05-24T23:06:15.4264503Z</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>None</td>\n",
       "      <td>GetRecord</td>\n",
       "      <td>SE/SSA/7131</td>\n",
       "      <td>oai_ape_ead</td>\n",
       "      <td>https://oai-pmh.riksarkivet.se/OAI</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   responseDate       verb   identifier metadataPrefix  \\\n",
       "0  2022-05-24T23:06:15.4264503Z       None         None           None   \n",
       "1                          None  GetRecord  SE/SSA/7131    oai_ape_ead   \n",
       "2                          None       None         None           None   \n",
       "\n",
       "                              request  record  \n",
       "0                                None     NaN  \n",
       "1  https://oai-pmh.riksarkivet.se/OAI     NaN  \n",
       "2                                None     NaN  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\"*\",\n",
    "            namespaces={\"doc\": \"http://www.openarchives.org/OAI/2.0/\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "dd8acc8e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>identifier</th>\n",
       "      <th>datestamp</th>\n",
       "      <th>setSpec</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>SE/SSA/7131</td>\n",
       "      <td>2021-09-27T13:30:37.617Z</td>\n",
       "      <td>SE_SSA_7131</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    identifier                 datestamp      setSpec\n",
       "0  SE/SSA/7131  2021-09-27T13:30:37.617Z  SE_SSA_7131"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:header\",\n",
    "            namespaces={\"doc\": \"http://www.openarchives.org/OAI/2.0/\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "e8b1eab2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>titleproper</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Carlgrenska Waldorfgymnasiet</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    titleproper\n",
       "0  Carlgrenska Waldorfgymnasiet"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:titlestmt\",\n",
    "            namespaces={\"doc\": \"urn:isbn:1-931666-22-9\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "4715deef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>item</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2022-05-24T23:18:23.429Z</td>\n",
       "      <td>Informationen kopierad ur Arkis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2021-09-27T13:30:37.617Z</td>\n",
       "      <td>Senaste uppdatering i Arkis</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2016-11-10T15:35:15.450Z</td>\n",
       "      <td>Registrerat i Arkis</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       date                             item\n",
       "0  2022-05-24T23:18:23.429Z  Informationen kopierad ur Arkis\n",
       "1  2021-09-27T13:30:37.617Z      Senaste uppdatering i Arkis\n",
       "2  2016-11-10T15:35:15.450Z              Registrerat i Arkis"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:change\",\n",
    "            namespaces={\"doc\": \"urn:isbn:1-931666-22-9\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "ffa70e60",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unitid</th>\n",
       "      <th>unittitle</th>\n",
       "      <th>physdesc</th>\n",
       "      <th>unitdate</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>SE/SSA/7131</td>\n",
       "      <td>Carlgrenska Waldorfgymnasiet</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>SE/SSA/7131/1</td>\n",
       "      <td>Slutbetyg</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>SE/SSA/7131/2</td>\n",
       "      <td>Examensbevis och studiebevis i pappersform</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>SE/SSA/7131/2/1</td>\n",
       "      <td>2016--2020</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2016--2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>SE/SSA/7131/3</td>\n",
       "      <td>Betygskataloger</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>SE/SSA/7131/3/1</td>\n",
       "      <td>2016--2020</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2016--2020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>SE/SSA/7131/4</td>\n",
       "      <td>Elevsvar på nationella prov</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>SE/SSA/7131/4/1</td>\n",
       "      <td>2014--2015</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2014--2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>SE/SSA/7131/4/2</td>\n",
       "      <td>2015--2016</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2015--2016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>SE/SSA/7131/4/3</td>\n",
       "      <td>2016--2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2016--2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>SE/SSA/7131/4/4</td>\n",
       "      <td>2017</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>SE/SSA/7131/4/5</td>\n",
       "      <td>2018</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>SE/SSA/7131/4/6</td>\n",
       "      <td>2018</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>SE/SSA/7131/4/7</td>\n",
       "      <td>2019</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2019</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>SE/SSA/7131/5</td>\n",
       "      <td>Åtgärdsprogram</td>\n",
       "      <td>NaN</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             unitid                                   unittitle  physdesc  \\\n",
       "0       SE/SSA/7131                Carlgrenska Waldorfgymnasiet       NaN   \n",
       "1     SE/SSA/7131/1                                   Slutbetyg       NaN   \n",
       "2     SE/SSA/7131/2  Examensbevis och studiebevis i pappersform       NaN   \n",
       "3   SE/SSA/7131/2/1                                  2016--2020       NaN   \n",
       "4     SE/SSA/7131/3                             Betygskataloger       NaN   \n",
       "5   SE/SSA/7131/3/1                                  2016--2020       NaN   \n",
       "6     SE/SSA/7131/4                 Elevsvar på nationella prov       NaN   \n",
       "7   SE/SSA/7131/4/1                                  2014--2015       NaN   \n",
       "8   SE/SSA/7131/4/2                                  2015--2016       NaN   \n",
       "9   SE/SSA/7131/4/3                                  2016--2017       NaN   \n",
       "10  SE/SSA/7131/4/4                                        2017       NaN   \n",
       "11  SE/SSA/7131/4/5                                        2018       NaN   \n",
       "12  SE/SSA/7131/4/6                                        2018       NaN   \n",
       "13  SE/SSA/7131/4/7                                        2019       NaN   \n",
       "14    SE/SSA/7131/5                              Åtgärdsprogram       NaN   \n",
       "\n",
       "      unitdate  \n",
       "0         None  \n",
       "1         None  \n",
       "2         None  \n",
       "3   2016--2020  \n",
       "4         None  \n",
       "5   2016--2020  \n",
       "6         None  \n",
       "7   2014--2015  \n",
       "8   2015--2016  \n",
       "9   2016--2017  \n",
       "10        2017  \n",
       "11        2018  \n",
       "12        2018  \n",
       "13        2019  \n",
       "14        None  "
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:did\",\n",
    "            namespaces={\"doc\": \"urn:isbn:1-931666-22-9\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "8b4e2536",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>encodinganalog</th>\n",
       "      <th>p</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>summary</td>\n",
       "      <td>Carlgrenska Waldorfgymnasiet är en friskola i ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>summary</td>\n",
       "      <td>Serien i e-arkivet. Slutbetyg 2016-2019</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>summary</td>\n",
       "      <td>2016-2018 enbart examensbevis. 2019-2020 studi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>summary</td>\n",
       "      <td>Skolans elevsvar på nationella prov bevaras so...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>summary</td>\n",
       "      <td>Matematik 2014-2015. Engelska 2014. Obs! Handl...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>summary</td>\n",
       "      <td>Engelska 2015-2016. Svenska 2016.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>summary</td>\n",
       "      <td>Matematik 2016-2017.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>summary</td>\n",
       "      <td>Engelska, svenska</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>summary</td>\n",
       "      <td>Engelska.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>summary</td>\n",
       "      <td>Matematik, svenska.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>summary</td>\n",
       "      <td>Matematik, engelska, svenska.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>summary</td>\n",
       "      <td>Handlingarna kan omfattas av sekretess enligt ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   encodinganalog                                                  p\n",
       "0         summary  Carlgrenska Waldorfgymnasiet är en friskola i ...\n",
       "1         summary            Serien i e-arkivet. Slutbetyg 2016-2019\n",
       "2         summary  2016-2018 enbart examensbevis. 2019-2020 studi...\n",
       "3         summary  Skolans elevsvar på nationella prov bevaras so...\n",
       "4         summary  Matematik 2014-2015. Engelska 2014. Obs! Handl...\n",
       "5         summary                  Engelska 2015-2016. Svenska 2016.\n",
       "6         summary                               Matematik 2016-2017.\n",
       "7         summary                                  Engelska, svenska\n",
       "8         summary                                          Engelska.\n",
       "9         summary                                Matematik, svenska.\n",
       "10        summary                      Matematik, engelska, svenska.\n",
       "11        summary  Handlingarna kan omfattas av sekretess enligt ..."
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:scopecontent\",\n",
    "            namespaces={\"doc\": \"urn:isbn:1-931666-22-9\"})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfc589bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.read_xml(testurl,\n",
    "            xpath=\".//doc:scopecontent\",\n",
    "            namespaces={\"doc\": \"urn:isbn:1-931666-22-9\"})\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}