Part 1 — Webscrapping.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Парсинг и анализ блога\n",
    "\n",
    "### План\n",
    "0. Подготовка\n",
    "1. Сбор данных\n",
    "2. Анализ данных\n",
    "3. Визуализация "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup\n",
    "import requests\n",
    "import re\n",
    "\n",
    "import pandas as pd\n",
    "import datetime\n",
    "import time\n",
    "\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  1. Сбор данных\n",
    "\n",
    "Объект изучения — блог Ильи Бирмана. Удобно, что движок блога делает отдельную [страницу](https://ilyabirman.ru/meanwhile/all/) со всеми постами."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "webpage = requests.get(\"https://ilyabirman.ru/meanwhile/all/\")\n",
    "\n",
    "soup = BeautifulSoup(webpage.content, \"html.parser\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Достать дату из поста оказалось сложнее, чем другие данные. В природе она выглядит так:\n",
    "```html\n",
    "<div class=\"e2-note-meta\">\n",
    "    <span class=\"e2-read-counter\">\n",
    "        <span class=\"e2-svgi\">\n",
    "            <svg>...</svg>\n",
    "        </span>\n",
    "        &nbsp;768\n",
    "    </span>&nbsp;&nbsp;\n",
    "    <span title=\"17 ноября 2018, 00:43, GMT+05:00\">6 мес</span> ... \n",
    "</div>\n",
    "```\n",
    "\n",
    "…то есть внутри третьего `<span>` внутри `<div>`. Причем сама дата написана в `title=\"\"`, а текстом показывается давность написания. \n",
    "\n",
    "Прочитал мануал по BeautifulSoup и RE и сначала пытался найти этот `span` как-то так: `soup.find_all(\"span\", title=re.compile(\".[\\d]{4}.\")`. Разумеется, ничего не вышло.\n",
    "\n",
    "Потом ещё долго пытался достать нужный спан; в итоге добрался через родительский `div`, у которого определённ класс. Чтобы обратиться к третьему элементу внутри `div`, не сразу понял, что в этом случае надо к `soup` обращаться через `.contents`, а не `.children`.\n",
    "\n",
    "Такой подход работал, пока я отлаживал подход на первой сотне постов. Когда выкатил парсер на более старые посты, у некоторых не оказалось счетчика просмотров. Из-за этого нумерация `span` с датой изменилась и парсер выдавал ошибки. Пришлось ещё раз прочитать мануал и прийти к более изящному решению — просто обратиться к дочернему `span` без класса у нужного `div`, без всяких номеров. \n",
    "\n",
    "В итоге достал всю `span` и через регулярные выражения вытащил отдельно день, месяц, год и время публикации поста. Месяц пришлось перевести в число с помощью списка месяцев. Всё сохранил в списке в формате объекта `datetime`. \n",
    "\n",
    "Получился длинный кусок кода, поэтому вытащил в отдельную функцию. Наверное, код, который вытаскивает остальные параметры поста, тоже надо было оформить в виде отдельных функций. Получилось бы аккуратнее."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "def append_datetime_from_soup(source_soup, list_with_results):\n",
    "    \n",
    "    try:\n",
    "        \n",
    "        # first version of code: worked good only for posts with views counter; \n",
    "        # without counter, position of span with time and date is different \n",
    "        \n",
    "        # find the <div>, get the <span> with timestamp (4th child)\n",
    "        # span_datetime = str(source_soup.find(\"div\", class_=\"e2-note-meta\").contents[3])\n",
    "            \n",
    "        # updated verison doesn't care the position; it look for a span without class inside the right div\n",
    "        span_datetime = str(source_soup.find(\"div\", class_=\"e2-note-meta\")\\\n",
    "                            .find(\"span\", class_=\"\"))\n",
    "            \n",
    "        # get string with date and time from <span> title\n",
    "        date_str = re.search(\"\\\".+\\\"\", span_datetime).group(0)\n",
    "\n",
    "        # get day: one or two digits\n",
    "        day = int(re.search(\"[\\d]{1,2}\", date_str).group(0))\n",
    "\n",
    "        # get month as string\n",
    "        month_str = str(re.search(\" [\\w^(\\d)]+ \", date_str).group(0))\n",
    "        month_str = month_str.strip()\n",
    "\n",
    "        # convert string to integer\n",
    "        months = [\"января\", \"февраля\", \"марта\", \"апреля\", \"мая\", \"июня\",\n",
    "                  \"июля\", \"августа\", \"сентября\", \"октября\", \"ноября\", \"декабря\"]\n",
    "        month_int = months.index(month_str) + 1\n",
    "\n",
    "        # get year\n",
    "        year = int(re.search(\"[\\d]{4}\", date_str).group(0))\n",
    "\n",
    "        # get time\n",
    "        time_ = re.search(\"[\\d]{2}:[\\d]{2}\", date_str).group(0)\n",
    "        hour = int(time_[0:2])\n",
    "        minute = int(time_[3:5])\n",
    "\n",
    "        # make a datetime object\n",
    "        date_time = datetime.datetime(year, month_int, day, hour=hour, minute=minute)\n",
    "\n",
    "        # append to the result\n",
    "        list_with_results.append(date_time)\n",
    "    \n",
    "    except AttributeError:\n",
    "        print(\"Can't find date in string {}\"\\\n",
    "              .format(source_soup.find(\"div\", class_=\"e2-note-meta\")))\n",
    "        list_with_results.append(\"0\")\n",
    "        \n",
    "        \n",
    "        \n",
    "        \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Удобно, что в Jupyter можно разделять куски кода: парсинг обычно занимает больше времени, чем обычные операции. Чтобы не ждать каждый раз, выделил его в отедльный кусок и запускал только при изменении кода.\n",
    "\n",
    "Ещё было полезно органичить количество проходов для отладки, чтобы не парсить каждый раз все 4400+ заметок."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "titles = []\n",
    "views = []\n",
    "comments  = []\n",
    "tags = []\n",
    "datetimes = []\n",
    "images = []\n",
    "words = []\n",
    "links = []\n",
    "\n",
    "\n",
    "# lists and counter for debugging purpose\n",
    "bad_links = [] \n",
    "bad_links_meanwhile = []\n",
    "parse_count = 0\n",
    "\n",
    "\n",
    "# find all links\n",
    "for link in soup.find_all(\"a\"):\n",
    "    post_tags = []\n",
    "    parse_count += 1\n",
    "    \n",
    "    # drop not links (there are some 'None' object in scrapping results)\n",
    "    if type(link.get('href')) == type(\"string\"):\n",
    "        \n",
    "        # exclude blog engine settings links         \n",
    "        if (\"@ajax\" in link.get('href'))\\\n",
    "        | (\"/settings/\" in link.get('href'))\\\n",
    "        | (\"/tags/\" in link.get('href'))\\\n",
    "        | (len(link.get(\"href\")) <= 32) : # 32 is the length of \"https://ilyabirman.ru/meanwhile/\"\n",
    "            continue\n",
    "        \n",
    "        # drop all except links for blogposts\n",
    "        elif \"ilyabirman.ru/meanwhile/\" in link.get('href'):\n",
    "\n",
    "            # get a link itself and parse it with with BeautifulSoup\n",
    "            blog_page = requests.get(link.get('href'))\n",
    "            blog_page_soup = BeautifulSoup(blog_page.content, \"html.parser\")\n",
    "            #print(link.get(\"href\")) # debugging\n",
    "            \n",
    "            # check if a post have a ciew counter (old posts have no views counter)\n",
    "            if len(blog_page_soup.select(\".e2-read-counter\")) > 0:\n",
    "\n",
    "                # get a span block with views count\n",
    "                views_span = blog_page_soup.select(\".e2-read-counter\")[-1].get_text()\n",
    "\n",
    "                # get a number from text block and format as integer, save to list of views count\n",
    "                views.append(int(re.search(\"\\d+\", views_span).group(0)))\n",
    "\n",
    "            else:\n",
    "                views.append(1)\n",
    "            \n",
    "            # get comments count\n",
    "            if blog_page_soup.find(id=\"e2-comments-count\") != None:\n",
    "                comments_span = blog_page_soup.find(id=\"e2-comments-count\").get_text()\n",
    "                comments.append(int(re.search(\"\\d+\", comments_span).group(0)))\n",
    "            else:\n",
    "                comments.append(0)\n",
    "                \n",
    "            # get tags for each post\n",
    "            for tag in blog_page_soup.select(\".e2-tag\"):\n",
    "                post_tags.append(tag.get_text())\n",
    "            \n",
    "            # list of posts' tags\n",
    "            tags.append(post_tags)\n",
    "            \n",
    "            # get a blogpost title, save to list of titles\n",
    "            titles.append(blog_page_soup.h1.get_text())\n",
    "            \n",
    "            # append date and time for each post to the list\n",
    "            append_datetime_from_soup(blog_page_soup, datetimes)\n",
    "            \n",
    "            # get images count\n",
    "            images.append(\\\n",
    "            len(blog_page_soup.find_all(\"div\", class_=\"e2-text-picture-imgwrapper\")))\n",
    "            \n",
    "            # post's length (words count)\n",
    "            words.append(len(blog_page_soup.article.get_text().split()))\n",
    "            \n",
    "            # get link\n",
    "            links.append(link.get('href'))\n",
    "            \n",
    "        else:\n",
    "            \n",
    "            # just checking the other links\n",
    "            bad_links_meanwhile.append(link.get('href'))\n",
    "    \n",
    "    # checking, just in case\n",
    "    elif type(link.get('href')) != None:\n",
    "        bad_links.append(link.get('href'))\n",
    "    \n",
    "    else:\n",
    "        print(\"Bad link: {}\".format(link.get('href')))\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# DEBUGGING and tweaking the links filter\n",
    "# print(len(bad_links), len(bad_links_meanwhile))\n",
    "# print(bad_links_meanwhile)\n",
    "# print(len(\"https://ilyabirman.ru/meanwhile/\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "После прохода парсера на всем объёме блога проверил полность полученных данных (должно быть 4400+ записей).\n",
    "\n",
    "Проверка показала только 2000 записей. Хорошо, что проверил :-)\n",
    "\n",
    "Проблема оказалась в первом условии отбора подходящих ссылок. Отсеивал рабочие и прочие ссылки со страницы через проверку вхожения строки «meanwhile/all» в текст ссылки. Проверка простая и работала, но только на первой половине блога. На более старых постаъ частичка «/all» исчезла и записи не проходили проверку. \n",
    "\n",
    "Пришлось оставить только «meanwhile/all», но добавить перед этим ещё несколько отдельных условияй, чтобы отсеить рабочие ссылки.\n",
    "\n",
    "Итоговые цифры: списки с данными постов одинаковой длины — 4483 записи."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4595 4548 4548 4548\n"
     ]
    }
   ],
   "source": [
    "# checking lengths of the lists with scrapping results\n",
    "print(parse_count, len(views), len(images), len(words))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`dataframe` отказался принимать список списков с тегами для каждого поста, поэтому пришлось сделать список строк."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# convert list of lists with tags to list of strings (to add to a DataFrame properly)\n",
    "tags_as_string = []\n",
    "\n",
    "for item in tags:\n",
    "    string = ''\n",
    "    if type(item) == type(string): # if only 1 tag, add to the list it as is\n",
    "        string = item\n",
    "    else:\n",
    "        for tag in item: # if more than one tag, iterate through\n",
    "            string += tag + \", \"\n",
    "    tags_as_string.append(string[0:-2]) # add string of tags without last two chars: \", \""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Анализ данных\n",
    "После сбора данных, переходим к их анализу.\n",
    "\n",
    "Чтобы работать с данными, собираем списки в словарь, а словарь — в `pandas dataframe`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict = {\"title\": titles, \n",
    "       \"datetime\": datetimes,\n",
    "        \"views\": views, \n",
    "        \"comments\": comments, \n",
    "       \"length\": words,\n",
    "       \"images\": images,\n",
    "        \"tags\": tags_as_string,\n",
    "       \"link\": links} \n",
    "\n",
    "birman_frame = pd.DataFrame(data = dict)\n",
    "\n",
    "birman_frame.drop_duplicates(inplace=True)\n",
    "birman_frame.title = birman_frame.title.apply(lambda x: re.search(\"[^(\\n|\\r)]+\", x).group(0))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Проверяем колонки и длину получившегося датафрейма. Минус два дупликата. Пустых значений нет."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 4546 entries, 0 to 4547\n",
      "Data columns (total 8 columns):\n",
      "title       4546 non-null object\n",
      "datetime    4546 non-null datetime64[ns]\n",
      "views       4546 non-null int64\n",
      "comments    4546 non-null int64\n",
      "length      4546 non-null int64\n",
      "images      4546 non-null int64\n",
      "tags        4546 non-null object\n",
      "link        4546 non-null object\n",
      "dtypes: datetime64[ns](1), int64(4), object(3)\n",
      "memory usage: 319.6+ KB\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "title       0\n",
       "datetime    0\n",
       "views       0\n",
       "comments    0\n",
       "length      0\n",
       "images      0\n",
       "tags        0\n",
       "link        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "birman_frame.info()\n",
    "\n",
    "birman_frame.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# запись в файл"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "birman_frame.to_csv(\"birman_new.csv\", sep=\";\", index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4487 entries, 0 to 4486\n",
      "Data columns (total 8 columns):\n",
      "title       4487 non-null object\n",
      "datetime    4487 non-null object\n",
      "views       4487 non-null int64\n",
      "comments    4487 non-null int64\n",
      "length      4487 non-null int64\n",
      "images      4487 non-null int64\n",
      "tags        4392 non-null object\n",
      "link        4487 non-null object\n",
      "dtypes: int64(4), object(4)\n",
      "memory usage: 280.5+ KB\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4487 entries, 0 to 4486\n",
      "Data columns (total 8 columns):\n",
      "title       4487 non-null object\n",
      "datetime    4487 non-null object\n",
      "views       4487 non-null int64\n",
      "comments    4487 non-null int64\n",
      "length      4487 non-null int64\n",
      "images      4487 non-null int64\n",
      "tags        4487 non-null object\n",
      "link        4487 non-null object\n",
      "dtypes: int64(4), object(4)\n",
      "memory usage: 280.5+ KB\n"
     ]
    }
   ],
   "source": [
    "birman_from_file = pd.DataFrame()\n",
    "birman_from_file = pd.read_csv(\"birman.csv\", sep=\";\")\n",
    "\n",
    "birman_from_file.info()\n",
    "#birman_from_file.head()\n",
    "#birman_from_file.isnull().sum()\n",
    "#birman_from_file[birman_from_file.tags.isnull()].head()\n",
    "\n",
    "birman_from_file.fillna(\"без тэга\", inplace=True)\n",
    "\n",
    "birman_from_file.info()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Есть четыре числовых колонки, по которым можно сравнивать посты:\n",
    "- просмотры\n",
    "- комментарии\n",
    "- длина поста\n",
    "- количество картинок\n",
    "\n",
    "Для начала самое простое: самые просматриваемые посты, самые комментируемые, длинные и где больше всего картинок."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# tell pandas not to truncate links in column\n",
    "pd.options.display.max_colwidth = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>datetime</th>\n",
       "      <th>views</th>\n",
       "      <th>comments</th>\n",
       "      <th>length</th>\n",
       "      <th>images</th>\n",
       "      <th>tags</th>\n",
       "      <th>link</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3160</th>\n",
       "      <td>О запятой после «С уважением»</td>\n",
       "      <td>2006-06-27 12:59:00</td>\n",
       "      <td>99381</td>\n",
       "      <td>10</td>\n",
       "      <td>174</td>\n",
       "      <td>0</td>\n",
       "      <td>русский язык</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2006/06/27/2/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1719</th>\n",
       "      <td>Переплата по кредиту</td>\n",
       "      <td>2013-05-02 17:09:00</td>\n",
       "      <td>39417</td>\n",
       "      <td>0</td>\n",
       "      <td>278</td>\n",
       "      <td>0</td>\n",
       "      <td>жизнь, общество, экономика</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/pereplata/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1827</th>\n",
       "      <td>Числа π и e</td>\n",
       "      <td>2012-12-28 13:16:00</td>\n",
       "      <td>15587</td>\n",
       "      <td>0</td>\n",
       "      <td>506</td>\n",
       "      <td>2</td>\n",
       "      <td>математика</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/pi-and-e/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1083</th>\n",
       "      <td>Война</td>\n",
       "      <td>2015-10-08 18:55:00</td>\n",
       "      <td>13810</td>\n",
       "      <td>0</td>\n",
       "      <td>627</td>\n",
       "      <td>0</td>\n",
       "      <td>красная таблетка, общество</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/war/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1322</th>\n",
       "      <td>Почему люди платят налоги</td>\n",
       "      <td>2014-10-01 01:29:00</td>\n",
       "      <td>10453</td>\n",
       "      <td>194</td>\n",
       "      <td>1493</td>\n",
       "      <td>0</td>\n",
       "      <td>красная таблетка, общество, философия, экономика</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/why-people-pay-taxes/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2241</th>\n",
       "      <td>О наращении окончаний числительных</td>\n",
       "      <td>2010-11-08 14:35:00</td>\n",
       "      <td>9823</td>\n",
       "      <td>19</td>\n",
       "      <td>354</td>\n",
       "      <td>0</td>\n",
       "      <td>русский язык, студентам</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/o-naraschenii-okonchaniy-chislitelnyh/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1255</th>\n",
       "      <td>Почему нельзя перейти на доллары</td>\n",
       "      <td>2014-12-31 14:50:00</td>\n",
       "      <td>9403</td>\n",
       "      <td>0</td>\n",
       "      <td>267</td>\n",
       "      <td>0</td>\n",
       "      <td>биткоин, общество, экономика</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/pochemu-nelzya-pereyti-na-dollary/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>416</th>\n",
       "      <td>Рабочая неделя в Израиле</td>\n",
       "      <td>2018-01-27 14:05:00</td>\n",
       "      <td>7423</td>\n",
       "      <td>1</td>\n",
       "      <td>535</td>\n",
       "      <td>0</td>\n",
       "      <td>из Тель-Авива, Израиль, Тель-Авив</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/shavua/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2403</th>\n",
       "      <td>Люголь</td>\n",
       "      <td>2009-09-09 11:58:00</td>\n",
       "      <td>7010</td>\n",
       "      <td>21</td>\n",
       "      <td>111</td>\n",
       "      <td>0</td>\n",
       "      <td>медицина</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2009/09/09/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2110</th>\n",
       "      <td>Синхронное и асинхронное</td>\n",
       "      <td>2011-09-14 03:45:00</td>\n",
       "      <td>6818</td>\n",
       "      <td>38</td>\n",
       "      <td>87</td>\n",
       "      <td>0</td>\n",
       "      <td>веб-разработка, русский язык</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/sync-async/</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   title            datetime  views  comments  \\\n",
       "3160       О запятой после «С уважением» 2006-06-27 12:59:00  99381        10   \n",
       "1719                Переплата по кредиту 2013-05-02 17:09:00  39417         0   \n",
       "1827                         Числа π и e 2012-12-28 13:16:00  15587         0   \n",
       "1083                               Война 2015-10-08 18:55:00  13810         0   \n",
       "1322           Почему люди платят налоги 2014-10-01 01:29:00  10453       194   \n",
       "2241  О наращении окончаний числительных 2010-11-08 14:35:00   9823        19   \n",
       "1255    Почему нельзя перейти на доллары 2014-12-31 14:50:00   9403         0   \n",
       "416             Рабочая неделя в Израиле 2018-01-27 14:05:00   7423         1   \n",
       "2403                              Люголь 2009-09-09 11:58:00   7010        21   \n",
       "2110            Синхронное и асинхронное 2011-09-14 03:45:00   6818        38   \n",
       "\n",
       "      length  images                                              tags  \\\n",
       "3160     174       0                                      русский язык   \n",
       "1719     278       0                        жизнь, общество, экономика   \n",
       "1827     506       2                                        математика   \n",
       "1083     627       0                        красная таблетка, общество   \n",
       "1322    1493       0  красная таблетка, общество, философия, экономика   \n",
       "2241     354       0                           русский язык, студентам   \n",
       "1255     267       0                      биткоин, общество, экономика   \n",
       "416      535       0                 из Тель-Авива, Израиль, Тель-Авив   \n",
       "2403     111       0                                          медицина   \n",
       "2110      87       0                      веб-разработка, русский язык   \n",
       "\n",
       "                                                                            link  \n",
       "3160                               https://ilyabirman.ru/meanwhile/2006/06/27/2/  \n",
       "1719                              https://ilyabirman.ru/meanwhile/all/pereplata/  \n",
       "1827                               https://ilyabirman.ru/meanwhile/all/pi-and-e/  \n",
       "1083                                    https://ilyabirman.ru/meanwhile/all/war/  \n",
       "1322                   https://ilyabirman.ru/meanwhile/all/why-people-pay-taxes/  \n",
       "2241  https://ilyabirman.ru/meanwhile/all/o-naraschenii-okonchaniy-chislitelnyh/  \n",
       "1255      https://ilyabirman.ru/meanwhile/all/pochemu-nelzya-pereyti-na-dollary/  \n",
       "416                                  https://ilyabirman.ru/meanwhile/all/shavua/  \n",
       "2403                               https://ilyabirman.ru/meanwhile/2009/09/09/1/  \n",
       "2110                             https://ilyabirman.ru/meanwhile/all/sync-async/  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "birman_frame.sort_values(by='views', ascending=False).head(10)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>datetime</th>\n",
       "      <th>views</th>\n",
       "      <th>comments</th>\n",
       "      <th>length</th>\n",
       "      <th>images</th>\n",
       "      <th>tags</th>\n",
       "      <th>link</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2490</th>\n",
       "      <td>Ремонетизация</td>\n",
       "      <td>2009-04-28 15:46:00</td>\n",
       "      <td>5</td>\n",
       "      <td>200</td>\n",
       "      <td>176</td>\n",
       "      <td>0</td>\n",
       "      <td>реклама, этот сайт</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2009/04/28/2/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2384</th>\n",
       "      <td>Бананотехнология</td>\n",
       "      <td>2009-11-06 01:22:00</td>\n",
       "      <td>97</td>\n",
       "      <td>199</td>\n",
       "      <td>220</td>\n",
       "      <td>0</td>\n",
       "      <td>еда, жизнь</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2009/11/06/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1322</th>\n",
       "      <td>Почему люди платят налоги</td>\n",
       "      <td>2014-10-01 01:29:00</td>\n",
       "      <td>10453</td>\n",
       "      <td>194</td>\n",
       "      <td>1493</td>\n",
       "      <td>0</td>\n",
       "      <td>красная таблетка, общество, философия, экономика</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/why-people-pay-taxes/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2287</th>\n",
       "      <td>Опенсос</td>\n",
       "      <td>2010-06-04 00:48:00</td>\n",
       "      <td>29</td>\n",
       "      <td>174</td>\n",
       "      <td>442</td>\n",
       "      <td>0</td>\n",
       "      <td>идиоты, опенсорс, софт</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2010/06/04/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2863</th>\n",
       "      <td>Кто на чём</td>\n",
       "      <td>2007-10-08 12:44:00</td>\n",
       "      <td>5</td>\n",
       "      <td>150</td>\n",
       "      <td>88</td>\n",
       "      <td>0</td>\n",
       "      <td>браузеры</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2007/10/08/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2133</th>\n",
       "      <td>Эгея</td>\n",
       "      <td>2011-07-05 21:07:00</td>\n",
       "      <td>84</td>\n",
       "      <td>123</td>\n",
       "      <td>136</td>\n",
       "      <td>1</td>\n",
       "      <td>проекты, Эгея</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/aegea/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4126</th>\n",
       "      <td>Комментатор Саша</td>\n",
       "      <td>2004-02-06 10:42:00</td>\n",
       "      <td>38</td>\n",
       "      <td>116</td>\n",
       "      <td>279</td>\n",
       "      <td>0</td>\n",
       "      <td>Елисейкин, идиоты, снукер</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2004/02/06/2/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2561</th>\n",
       "      <td>Переводим названия фирм, коллективов и продуктов</td>\n",
       "      <td>2009-01-21 04:35:00</td>\n",
       "      <td>221</td>\n",
       "      <td>114</td>\n",
       "      <td>197</td>\n",
       "      <td>0</td>\n",
       "      <td>вопрос, русский язык</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2009/01/21/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2261</th>\n",
       "      <td>Однажды в Сбербанке</td>\n",
       "      <td>2010-09-12 02:26:00</td>\n",
       "      <td>644</td>\n",
       "      <td>104</td>\n",
       "      <td>1253</td>\n",
       "      <td>0</td>\n",
       "      <td>жизнь, переговоры, сервис</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/sberbank/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2138</th>\n",
       "      <td>Недовольство и возмущение</td>\n",
       "      <td>2011-06-30 19:52:00</td>\n",
       "      <td>55</td>\n",
       "      <td>92</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>вопрос, я</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/discontent/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2308</th>\n",
       "      <td>Как Адоби просрали Флеш</td>\n",
       "      <td>2010-04-10 13:44:00</td>\n",
       "      <td>37</td>\n",
       "      <td>89</td>\n",
       "      <td>640</td>\n",
       "      <td>0</td>\n",
       "      <td>Айпад, Айфон, Флеш сосёт, Эпл</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2010/04/10/2/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1441</th>\n",
       "      <td>Дискуссии по понедельникам: что хорошего в патриотизме?</td>\n",
       "      <td>2014-05-05 14:00:00</td>\n",
       "      <td>90</td>\n",
       "      <td>84</td>\n",
       "      <td>185</td>\n",
       "      <td>0</td>\n",
       "      <td>дискуссии по понедельникам, общество</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/monday-discussion-patriotism/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2294</th>\n",
       "      <td>Почему Опера</td>\n",
       "      <td>2010-05-21 17:35:00</td>\n",
       "      <td>24</td>\n",
       "      <td>78</td>\n",
       "      <td>457</td>\n",
       "      <td>0</td>\n",
       "      <td>браузеры, глюки, Опера</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2010/05/21/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2187</th>\n",
       "      <td>Автосовет</td>\n",
       "      <td>2011-03-14 02:41:00</td>\n",
       "      <td>19</td>\n",
       "      <td>71</td>\n",
       "      <td>573</td>\n",
       "      <td>3</td>\n",
       "      <td>автомобиль, вопрос</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2011/03/14/1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2518</th>\n",
       "      <td>Пиши „руб.“ нормально!</td>\n",
       "      <td>2009-03-20 23:13:00</td>\n",
       "      <td>84</td>\n",
       "      <td>71</td>\n",
       "      <td>157</td>\n",
       "      <td>0</td>\n",
       "      <td>русский язык, смешное, типографика</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/2009/03/20/2/</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                         title  \\\n",
       "2490                                            Ремонетизация    \n",
       "2384                                         Бананотехнология    \n",
       "1322                                 Почему люди платят налоги   \n",
       "2287                                                   Опенсос   \n",
       "2863                                               Кто на чём    \n",
       "2133                                                      Эгея   \n",
       "4126                                          Комментатор Саша   \n",
       "2561          Переводим названия фирм, коллективов и продуктов   \n",
       "2261                                       Однажды в Сбербанке   \n",
       "2138                                 Недовольство и возмущение   \n",
       "2308                                   Как Адоби просрали Флеш   \n",
       "1441  Дискуссии по понедельникам: что хорошего в патриотизме?    \n",
       "2294                                              Почему Опера   \n",
       "2187                                                 Автосовет   \n",
       "2518                                    Пиши „руб.“ нормально!   \n",
       "\n",
       "                datetime  views  comments  length  images  \\\n",
       "2490 2009-04-28 15:46:00      5       200     176       0   \n",
       "2384 2009-11-06 01:22:00     97       199     220       0   \n",
       "1322 2014-10-01 01:29:00  10453       194    1493       0   \n",
       "2287 2010-06-04 00:48:00     29       174     442       0   \n",
       "2863 2007-10-08 12:44:00      5       150      88       0   \n",
       "2133 2011-07-05 21:07:00     84       123     136       1   \n",
       "4126 2004-02-06 10:42:00     38       116     279       0   \n",
       "2561 2009-01-21 04:35:00    221       114     197       0   \n",
       "2261 2010-09-12 02:26:00    644       104    1253       0   \n",
       "2138 2011-06-30 19:52:00     55        92      37       0   \n",
       "2308 2010-04-10 13:44:00     37        89     640       0   \n",
       "1441 2014-05-05 14:00:00     90        84     185       0   \n",
       "2294 2010-05-21 17:35:00     24        78     457       0   \n",
       "2187 2011-03-14 02:41:00     19        71     573       3   \n",
       "2518 2009-03-20 23:13:00     84        71     157       0   \n",
       "\n",
       "                                                  tags  \\\n",
       "2490                                реклама, этот сайт   \n",
       "2384                                        еда, жизнь   \n",
       "1322  красная таблетка, общество, философия, экономика   \n",
       "2287                            идиоты, опенсорс, софт   \n",
       "2863                                          браузеры   \n",
       "2133                                     проекты, Эгея   \n",
       "4126                         Елисейкин, идиоты, снукер   \n",
       "2561                              вопрос, русский язык   \n",
       "2261                         жизнь, переговоры, сервис   \n",
       "2138                                         вопрос, я   \n",
       "2308                     Айпад, Айфон, Флеш сосёт, Эпл   \n",
       "1441              дискуссии по понедельникам, общество   \n",
       "2294                            браузеры, глюки, Опера   \n",
       "2187                                автомобиль, вопрос   \n",
       "2518                русский язык, смешное, типографика   \n",
       "\n",
       "                                                                   link  \n",
       "2490                      https://ilyabirman.ru/meanwhile/2009/04/28/2/  \n",
       "2384                      https://ilyabirman.ru/meanwhile/2009/11/06/1/  \n",
       "1322          https://ilyabirman.ru/meanwhile/all/why-people-pay-taxes/  \n",
       "2287                      https://ilyabirman.ru/meanwhile/2010/06/04/1/  \n",
       "2863                      https://ilyabirman.ru/meanwhile/2007/10/08/1/  \n",
       "2133                         https://ilyabirman.ru/meanwhile/all/aegea/  \n",
       "4126                      https://ilyabirman.ru/meanwhile/2004/02/06/2/  \n",
       "2561                      https://ilyabirman.ru/meanwhile/2009/01/21/1/  \n",
       "2261                      https://ilyabirman.ru/meanwhile/all/sberbank/  \n",
       "2138                    https://ilyabirman.ru/meanwhile/all/discontent/  \n",
       "2308                      https://ilyabirman.ru/meanwhile/2010/04/10/2/  \n",
       "1441  https://ilyabirman.ru/meanwhile/all/monday-discussion-patriotism/  \n",
       "2294                      https://ilyabirman.ru/meanwhile/2010/05/21/1/  \n",
       "2187                      https://ilyabirman.ru/meanwhile/2011/03/14/1/  \n",
       "2518                      https://ilyabirman.ru/meanwhile/2009/03/20/2/  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "birman_frame.sort_values(by='comments', ascending=False).head(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Самые длинные записи — конспекты книг и путеводитель по берлинскому клубу"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>datetime</th>\n",
       "      <th>views</th>\n",
       "      <th>comments</th>\n",
       "      <th>length</th>\n",
       "      <th>images</th>\n",
       "      <th>tags</th>\n",
       "      <th>link</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>Книга Николая Никулина «Воспоминания о войне»</td>\n",
       "      <td>2019-05-05 11:22:00</td>\n",
       "      <td>2353</td>\n",
       "      <td>8</td>\n",
       "      <td>6026</td>\n",
       "      <td>1</td>\n",
       "      <td>книги</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/nikolay-nikulin-vospominaniya-o-voyne/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143</th>\n",
       "      <td>Книга Россера Ривза «Реальность в рекламе»</td>\n",
       "      <td>2019-01-14 12:33:00</td>\n",
       "      <td>1634</td>\n",
       "      <td>1</td>\n",
       "      <td>3750</td>\n",
       "      <td>1</td>\n",
       "      <td>книги, реклама</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/reeves-advertizing-book/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>Книга Чарльза Тарта «Практика внимательности в повседневной жизни». Часть 2</td>\n",
       "      <td>2019-01-30 00:55:00</td>\n",
       "      <td>1236</td>\n",
       "      <td>0</td>\n",
       "      <td>3563</td>\n",
       "      <td>0</td>\n",
       "      <td>книги, медитация</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/tart-living-the-mindful-life-book-2/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>Полный гид по клубу Бергхайн</td>\n",
       "      <td>2018-12-09 12:27:00</td>\n",
       "      <td>4120</td>\n",
       "      <td>6</td>\n",
       "      <td>3493</td>\n",
       "      <td>6</td>\n",
       "      <td>Бергхайн</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/berghain-guide/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>Книга Чарльза Тарта «Практика внимательности в повседневной жизни». Часть 1</td>\n",
       "      <td>2019-01-28 14:45:00</td>\n",
       "      <td>1592</td>\n",
       "      <td>1</td>\n",
       "      <td>3198</td>\n",
       "      <td>0</td>\n",
       "      <td>книги, медитация</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/tart-living-the-mindful-life-book-1/</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                           title  \\\n",
       "94                                 Книга Николая Никулина «Воспоминания о войне»   \n",
       "143                                   Книга Россера Ривза «Реальность в рекламе»   \n",
       "133  Книга Чарльза Тарта «Практика внимательности в повседневной жизни». Часть 2   \n",
       "170                                                 Полный гид по клубу Бергхайн   \n",
       "135  Книга Чарльза Тарта «Практика внимательности в повседневной жизни». Часть 1   \n",
       "\n",
       "               datetime  views  comments  length  images              tags  \\\n",
       "94  2019-05-05 11:22:00   2353         8    6026       1             книги   \n",
       "143 2019-01-14 12:33:00   1634         1    3750       1    книги, реклама   \n",
       "133 2019-01-30 00:55:00   1236         0    3563       0  книги, медитация   \n",
       "170 2018-12-09 12:27:00   4120         6    3493       6          Бергхайн   \n",
       "135 2019-01-28 14:45:00   1592         1    3198       0  книги, медитация   \n",
       "\n",
       "                                                                           link  \n",
       "94   https://ilyabirman.ru/meanwhile/all/nikolay-nikulin-vospominaniya-o-voyne/  \n",
       "143                https://ilyabirman.ru/meanwhile/all/reeves-advertizing-book/  \n",
       "133    https://ilyabirman.ru/meanwhile/all/tart-living-the-mindful-life-book-2/  \n",
       "170                         https://ilyabirman.ru/meanwhile/all/berghain-guide/  \n",
       "135    https://ilyabirman.ru/meanwhile/all/tart-living-the-mindful-life-book-1/  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "birman_frame.sort_values(by='length', ascending=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Больше всего картинок оказалось в рассказе о проекте. Далее отчеты о телеграме и поездках"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>datetime</th>\n",
       "      <th>views</th>\n",
       "      <th>comments</th>\n",
       "      <th>length</th>\n",
       "      <th>images</th>\n",
       "      <th>tags</th>\n",
       "      <th>link</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Блог</td>\n",
       "      <td>2019-08-24 10:04:00</td>\n",
       "      <td>1031</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>55</td>\n",
       "      <td>видео, Школа стажёров, мир, Стокгольм, аудио по четвергам, техно, видеоблог, общество, подкаст, ...</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile//</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>245</th>\n",
       "      <td>Процесс создания логотипа Драйвинг-тестов. Часть 1</td>\n",
       "      <td>2018-09-06 14:06:00</td>\n",
       "      <td>1377</td>\n",
       "      <td>3</td>\n",
       "      <td>1153</td>\n",
       "      <td>36</td>\n",
       "      <td>портфолио, процесс</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/driving-tests-process-1/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>Телеграм за неделю 5—11 февраля 2018</td>\n",
       "      <td>2018-02-26 11:44:00</td>\n",
       "      <td>824</td>\n",
       "      <td>0</td>\n",
       "      <td>1006</td>\n",
       "      <td>32</td>\n",
       "      <td>телеграм-канал</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/telegram-2018-feb-12/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>380</th>\n",
       "      <td>Телеграм за неделю 12—18 февраля 2018</td>\n",
       "      <td>2018-02-28 23:44:00</td>\n",
       "      <td>831</td>\n",
       "      <td>1</td>\n",
       "      <td>1334</td>\n",
       "      <td>30</td>\n",
       "      <td>телеграм-канал</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/telegram-2018-feb-19/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231</th>\n",
       "      <td>Санкт-Петербург: Гранд-макет</td>\n",
       "      <td>2018-09-21 15:26:00</td>\n",
       "      <td>1591</td>\n",
       "      <td>2</td>\n",
       "      <td>262</td>\n",
       "      <td>29</td>\n",
       "      <td>мир, музеи и выставки, Санкт-Петербург</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/saint-petersburg-grand-maket/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>Тель-Авив: прогулка по Флорентину</td>\n",
       "      <td>2018-11-02 11:03:00</td>\n",
       "      <td>891</td>\n",
       "      <td>1</td>\n",
       "      <td>208</td>\n",
       "      <td>28</td>\n",
       "      <td>мир, Тель-Авив</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/tel-aviv-2017-2018-florentin/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>519</th>\n",
       "      <td>Телеграм за неделю 24—30 июля 2017</td>\n",
       "      <td>2017-11-01 12:47:00</td>\n",
       "      <td>17</td>\n",
       "      <td>1</td>\n",
       "      <td>1054</td>\n",
       "      <td>27</td>\n",
       "      <td>телеграм-канал</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/telegram-2017-jul-31/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>Телеграм за неделю 4—10 сентября 2017</td>\n",
       "      <td>2018-11-05 16:18:00</td>\n",
       "      <td>971</td>\n",
       "      <td>2</td>\n",
       "      <td>695</td>\n",
       "      <td>26</td>\n",
       "      <td>телеграм-канал</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/telegram-2017-sep-11/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>926</th>\n",
       "      <td>Музей БМВ в Мюнхене</td>\n",
       "      <td>2016-07-26 12:32:00</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>112</td>\n",
       "      <td>26</td>\n",
       "      <td>автомобиль, Германия, музеи и выставки, фото</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/munich-bmw-museum/</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>766</th>\n",
       "      <td>Регистрация на рейс «Аэрофлота»</td>\n",
       "      <td>2017-02-08 01:37:00</td>\n",
       "      <td>3678</td>\n",
       "      <td>0</td>\n",
       "      <td>1464</td>\n",
       "      <td>24</td>\n",
       "      <td>полёты, пользовательский интерфейс, студентам</td>\n",
       "      <td>https://ilyabirman.ru/meanwhile/all/aeroflot-registration/</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                  title            datetime  \\\n",
       "4                                                  Блог 2019-08-24 10:04:00   \n",
       "245  Процесс создания логотипа Драйвинг-тестов. Часть 1 2018-09-06 14:06:00   \n",
       "381               Телеграм за неделю 5—11 февраля 2018  2018-02-26 11:44:00   \n",
       "380              Телеграм за неделю 12—18 февраля 2018  2018-02-28 23:44:00   \n",
       "231                        Санкт-Петербург: Гранд-макет 2018-09-21 15:26:00   \n",
       "197                   Тель-Авив: прогулка по Флорентину 2018-11-02 11:03:00   \n",
       "519                 Телеграм за неделю 24—30 июля 2017  2017-11-01 12:47:00   \n",
       "196              Телеграм за неделю 4—10 сентября 2017  2018-11-05 16:18:00   \n",
       "926                                 Музей БМВ в Мюнхене 2016-07-26 12:32:00   \n",
       "766                     Регистрация на рейс «Аэрофлота» 2017-02-08 01:37:00   \n",
       "\n",
       "     views  comments  length  images  \\\n",
       "4     1031         0      22      55   \n",
       "245   1377         3    1153      36   \n",
       "381    824         0    1006      32   \n",
       "380    831         1    1334      30   \n",
       "231   1591         2     262      29   \n",
       "197    891         1     208      28   \n",
       "519     17         1    1054      27   \n",
       "196    971         2     695      26   \n",
       "926     45         0     112      26   \n",
       "766   3678         0    1464      24   \n",
       "\n",
       "                                                                                                    tags  \\\n",
       "4    видео, Школа стажёров, мир, Стокгольм, аудио по четвергам, техно, видеоблог, общество, подкаст, ...   \n",
       "245                                                                                   портфолио, процесс   \n",
       "381                                                                                       телеграм-канал   \n",
       "380                                                                                       телеграм-канал   \n",
       "231                                                               мир, музеи и выставки, Санкт-Петербург   \n",
       "197                                                                                       мир, Тель-Авив   \n",
       "519                                                                                       телеграм-канал   \n",
       "196                                                                                       телеграм-канал   \n",
       "926                                                         автомобиль, Германия, музеи и выставки, фото   \n",
       "766                                                        полёты, пользовательский интерфейс, студентам   \n",
       "\n",
       "                                                                  link  \n",
       "4                                    https://ilyabirman.ru/meanwhile//  \n",
       "245       https://ilyabirman.ru/meanwhile/all/driving-tests-process-1/  \n",
       "381          https://ilyabirman.ru/meanwhile/all/telegram-2018-feb-12/  \n",
       "380          https://ilyabirman.ru/meanwhile/all/telegram-2018-feb-19/  \n",
       "231  https://ilyabirman.ru/meanwhile/all/saint-petersburg-grand-maket/  \n",
       "197  https://ilyabirman.ru/meanwhile/all/tel-aviv-2017-2018-florentin/  \n",
       "519          https://ilyabirman.ru/meanwhile/all/telegram-2017-jul-31/  \n",
       "196          https://ilyabirman.ru/meanwhile/all/telegram-2017-sep-11/  \n",
       "926             https://ilyabirman.ru/meanwhile/all/munich-bmw-museum/  \n",
       "766         https://ilyabirman.ru/meanwhile/all/aeroflot-registration/  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "birman_frame.sort_values(by='images', ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}