seminars/seminar_09/Seminar_9.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Htk4l-7DGV0s"
      },
      "source": [
        "# Семинар 9 - Методы построения оптического потока по последовательности изображений\n",
        "\n",
        "***"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4iulI-CnGV0y"
      },
      "source": [
        "Источник - https://habr.com/ru/post/201406/\n",
        "\n",
        "$\\textbf{Task statement}$: Оптический поток (ОП) – изображение видимого движения, представляющее собой сдвиг каждой точки (пикселя) между двумя изображениями.\n",
        "\n",
        "По сути, он представляет собой поле скоростей. Суть ОП в том, что для каждой точки изображения $I_{t_0} (\\vec{r})$ находится такой вектор сдвига $\\delta \\vec{r}$, чтобы было соответсвие между исходной точкой и точкой на следущем фрейме $I_{t_1} (\\vec{r} + \\delta \\vec{r})$. В качестве метрики соответвия берут близость интенсивности пикселей, беря во внимание маленькую разницу по времени между кадрами: $\\delta{t} = t_{1} - t_{0}$. В более точных методах точку можно привязывать к объекту на основе, например, выделения ключевых точек, а также считать градиенты вокруг точки, лапласианы и проч.\n",
        "\n",
        "$\\textbf{For what}$: Определение собственной скорости, Определение локализации, Улучшение методов трекинга объектов, сегментации, Детектирование событий, Сжатие видеопотока и проч.\n",
        "\n",
        "![](data/tennis.png)\n",
        "\n",
        "Разделяют 2 вида оптического потока - плотный (dense) [Farneback method, neural nets], работающий с целым изображением, и выборочный (sparse) [Lucas-Kanade method], работающий с ключевыми точками"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "jupyter": {
          "outputs_hidden": true
        },
        "tags": [],
        "id": "ZpFfouKuGV00"
      },
      "outputs": [],
      "source": [
        "# !wget https://www.bogotobogo.com/python/OpenCV_Python/images/mean_shift_tracking/slow_traffic_small.mp4 -O data/slow_traffic_small.mp4"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "S8W9vfaKGV02"
      },
      "outputs": [],
      "source": [
        "import cv2\n",
        "from tqdm import tqdm\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import IPython\n",
        "\n",
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fmgqmn5JGV02"
      },
      "source": [
        "## Lucas-Kanade (sparse)\n",
        "\n",
        "Пусть $I_{1} = I(x, y, t_{1})$ интенсивность в некоторой точке (x, y) на первом изображении (т. е. в момент времени t). На втором изображении эта точка сдвинулась на (dx, dy), при этом прошло время dt, тогда $I_{2} = I(x + dx, y + dx, t_{1} + dt) \\approx I_{1} + I_{x}dx + I_{y}dy +  I_{t}dt$. Из постановки задачи следует, что интенсивность пикселя не изменилась, тогда $I_{1} = I_{2}$. Далее определяем $dx, dy$.\n",
        "\n",
        "Самое простое решение проблемы – алгоритм Лукаса-Канаде. У нас же на изображении объекты размером больше 1 пикселя, значит, скорее всего, в окрестности текущей точки у других точек будут примерно такие же сдвиги. Поэтому мы возьмем окно вокруг этой точки и минимизируем (по МНК) в нем суммарную погрешность с весовыми коэффициентами, распределенными по Гауссу, то есть так, чтобы наибольший вес имели пиксели, ближе всего находящиеся к исследуемому.\n",
        "\n",
        "**Полезные материалы:**\n",
        "- цикл видео-лекций от First Principles of Computer Vision, посвященный Optical Flow и алгоритму Lucas-Kanade: https://youtube.com/playlist?list=PL2zRqk16wsdoYzrWStffqBAoUY8XdvatV"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wuIEa538GV04"
      },
      "source": [
        "### Вопрос 1\n",
        "\n",
        "В `cv2.calcOpticalFlowPyrLK` есть параметр, отвечающий за ImagePiramyd. Зачем нужна пирамида изображений в случае вычисления оптического потока?\n",
        "\n",
        "**Ответ:** Для улучшения точности и устойчивости алгоритма, особенно при наличии больших перемещений объектов между кадрами. Это основано на создании нескольких уровней изображений, где каждый следующий уровень представляет собой уменьшенную версию предыдущего. Обработка начинается с самого низкого (самого мелкого) уровня пирамиды, где обнаружение больших перемещений происходит проще из-за уменьшенного размера изображения. Затем результаты используются как начальные приближения для следующего уровня, что позволяет постепенно уточнять расчёты на более детальных уровнях пирамиды. Поэтому пирамида изображений помогает эффективно справляться с различными масштабами движений и повышает общую надёжность определения оптического потока."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3U1JGf5kGV16"
      },
      "source": [
        "### Задание 1\n",
        "\n",
        "Напишите реализацию Лукаса-Канаде c помощью numpy и cv2. Сравните с реализацией `cv2.calcOpticalFlowPyrLK`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QiEBNrAmGV17"
      },
      "outputs": [],
      "source": [
        "def get_derivative_x(\n",
        "    prevImg,\n",
        "    keypoint,\n",
        "    winSize,\n",
        ") -> np.array:\n",
        "    pass\n",
        "\n",
        "\n",
        "# arguments like in cv2 lib\n",
        "def my_calcOpticalFlowPyrLK(\n",
        "    prevImg,\n",
        "    nextImg,\n",
        "    prevPts,\n",
        "    nextPts, #None is our case\n",
        "    winSize,\n",
        "    #maxLevel, if you want to be an expert in CV,\n",
        "    #uncomment it and apply in LK method :)\n",
        ") -> np.array:\n",
        "\n",
        "    '''\n",
        "    You should return output vector of 2D points\n",
        "    (with single-precision floating-point coordinates)\n",
        "    containing the calculated new positions of input features in the second image\n",
        "    '''\n",
        "    nextPts = []\n",
        "\n",
        "    for keypoint in prevPts:\n",
        "\n",
        "        derivative_x = get_derivative_x(prevImg, keypoint, winSize)\n",
        "        derivative_y = get_derivative_y(prevImg, keypoint, winSize)\n",
        "        derivative_t = get_derivative_t(prevImg, keypoint, winSize)\n",
        "\n",
        "        # find a matrix and solve linear equation system\n",
        "        pass\n",
        "\n",
        "        # find result coordinates\n",
        "        nextPts.append()\n",
        "\n",
        "    return np.expand_dims(np.stack(nextPts), axis=1)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def get_derivative_x(prevImg,\n",
        "                     keypoint,\n",
        "                     winSize):\n",
        "    sobelx = cv2.Sobel(prevImg, cv2.CV_64F, 1, 0, ksize=5)\n",
        "    x, y = int(keypoint[0][0]), int(keypoint[0][1])\n",
        "    half_win = winSize[0] // 2\n",
        "    # Извлекаем окно вокруг точки интереса\n",
        "    derivative_x = sobelx[y-half_win:y+half_win+1, x-half_win:x+half_win+1]\n",
        "    return derivative_x\n",
        "\n",
        "def get_derivative_y(prevImg,\n",
        "                     keypoint,\n",
        "                     winSize):\n",
        "    sobely = cv2.Sobel(prevImg, cv2.CV_64F, 0, 1, ksize=5)\n",
        "    x, y = int(keypoint[0][0]), int(keypoint[0][1])\n",
        "    half_win = winSize[0] // 2\n",
        "    derivative_y = sobely[y-half_win:y+half_win+1, x-half_win:x+half_win+1]\n",
        "    return derivative_y\n",
        "\n",
        "def get_derivative_t(prevImg, nextImg,\n",
        "                     keypoint, winSize):\n",
        "    x, y = int(keypoint[0][0]), int(keypoint[0][1])\n",
        "    half_win = winSize[0] // 2\n",
        "    prev_patch = prevImg[y-half_win:y+half_win+1, x-half_win:x+half_win+1]\n",
        "    next_patch = nextImg[y-half_win:y+half_win+1, x-half_win:x+half_win+1]\n",
        "    derivative_t = next_patch - prev_patch\n",
        "    return derivative_t\n",
        "\n",
        "def my_calcOpticalFlowPyrLK(prevImg, nextImg, prevPts, nextPts, winSize):\n",
        "    nextPts = []\n",
        "    for keypoint in prevPts:\n",
        "        derivative_x = get_derivative_x(prevImg, keypoint, winSize)\n",
        "        derivative_y = get_derivative_y(prevImg, keypoint, winSize)\n",
        "        derivative_t = get_derivative_t(prevImg, nextImg, keypoint, winSize)\n",
        "\n",
        "        A = np.vstack((derivative_x.flatten(), derivative_y.flatten())).T\n",
        "        b = -derivative_t.flatten()\n",
        "\n",
        "        nu = np.linalg.lstsq(A, b, rcond=None)[0]\n",
        "        dx, dy = nu\n",
        "\n",
        "        new_x = keypoint[0][0] + dx\n",
        "        new_y = keypoint[0][1] + dy\n",
        "        nextPts.append([new_x, new_y])\n",
        "\n",
        "    return np.expand_dims(np.array(nextPts), axis=1)"
      ],
      "metadata": {
        "id": "yiowDqtOIRi4"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gIxzXq00GV18"
      },
      "source": [
        "### Релизация OpenCV - cv2.calcOpticalFlowPyrLK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MXWRImCPGV18",
        "outputId": "3d36fcd5-425d-4da1-b7f5-c9a28236b467"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 913/913 [00:10<00:00, 86.97it/s] \n"
          ]
        }
      ],
      "source": [
        "video_path = '/content/data/slow_traffic_small.mp4'\n",
        "\n",
        "cap = cv2.VideoCapture(video_path)\n",
        "\n",
        "length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))\n",
        "width  = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))\n",
        "height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))\n",
        "fps    = cap.get(cv2.CAP_PROP_FPS)\n",
        "\n",
        "fourcc = cv2.VideoWriter_fourcc(*'MP4V')\n",
        "out = cv2.VideoWriter('output_LK.mp4', fourcc, fps, (width, height))\n",
        "\n",
        "# params for ShiTomasi corner detection\n",
        "feature_params = dict(\n",
        "    maxCorners = 100,\n",
        "    qualityLevel = 0.3,\n",
        "    minDistance = 7,\n",
        "    blockSize = 7,\n",
        ")\n",
        "\n",
        "# Parameters for lucas kanade optical flow\n",
        "lk_params = dict(\n",
        "    #window size\n",
        "    winSize  = (15, 15),\n",
        "    #image piramid\n",
        "    maxLevel = 2,\n",
        "    #after the specified maximum number of iterations criteria.maxCount\n",
        "    #or when the search window moves by less than criteria.epsilon.\n",
        "    criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03),\n",
        ")\n",
        "# Create some random colors\n",
        "color = np.random.randint(0, 255, (100, 3))\n",
        "# Take first frame and find corners in it\n",
        "ret, old_frame = cap.read()\n",
        "old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)\n",
        "p0 = cv2.goodFeaturesToTrack(old_gray, mask = None, **feature_params)\n",
        "# Create a mask image for drawing purposes\n",
        "mask = np.zeros_like(old_frame)\n",
        "for i in tqdm(range(1, length)):\n",
        "\n",
        "    ret, frame = cap.read()\n",
        "\n",
        "    if not ret:\n",
        "        print('No frames grabbed!')\n",
        "        break\n",
        "\n",
        "    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n",
        "\n",
        "    # calculate optical flow\n",
        "    # see params here https://docs.opencv.org/3.4/dc/d6b/group__video__track.html#ga473e4b886d0bcc6b65831eb88ed93323\n",
        "    p1, st, err = cv2.calcOpticalFlowPyrLK(\n",
        "        prevImg=old_gray,\n",
        "        nextImg=frame_gray,\n",
        "        prevPts=p0,\n",
        "        nextPts=None,\n",
        "        **lk_params,\n",
        "    )\n",
        "\n",
        "    # Select good points where status is equal 1\n",
        "    if p1 is not None:\n",
        "        good_new = p1[st==1]\n",
        "        good_old = p0[st==1]\n",
        "\n",
        "    # draw the tracks\n",
        "    for i, (new, old) in enumerate(zip(good_new, good_old)):\n",
        "        a, b = new.ravel()\n",
        "        c, d = old.ravel()\n",
        "        mask = cv2.line(mask, (int(a), int(b)), (int(c), int(d)), color[i].tolist(), 2)\n",
        "        frame = cv2.circle(frame, (int(a), int(b)), 5, color[i].tolist(), -1)\n",
        "\n",
        "    img = cv2.add(frame, mask)\n",
        "    # Now update the previous frame and previous points\n",
        "    old_gray = frame_gray.copy()\n",
        "    p0 = good_new.reshape(-1, 1, 2)\n",
        "\n",
        "    out.write(img)\n",
        "\n",
        "cap.release()\n",
        "out.release()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "resources": {
            "http://localhost:8080/output_LK.mp4": {
              "data": "CjwhRE9DVFlQRSBodG1sPgo8aHRtbCBsYW5nPWVuPgogIDxtZXRhIGNoYXJzZXQ9dXRmLTg+CiAgPG1ldGEgbmFtZT12aWV3cG9ydCBjb250ZW50PSJpbml0aWFsLXNjYWxlPTEsIG1pbmltdW0tc2NhbGU9MSwgd2lkdGg9ZGV2aWNlLXdpZHRoIj4KICA8dGl0bGU+RXJyb3IgNDA0IChOb3QgRm91bmQpISExPC90aXRsZT4KICA8c3R5bGU+CiAgICAqe21hcmdpbjowO3BhZGRpbmc6MH1odG1sLGNvZGV7Zm9udDoxNXB4LzIycHggYXJpYWwsc2Fucy1zZXJpZn1odG1se2JhY2tncm91bmQ6I2ZmZjtjb2xvcjojMjIyO3BhZGRpbmc6MTVweH1ib2R5e21hcmdpbjo3JSBhdXRvIDA7bWF4LXdpZHRoOjM5MHB4O21pbi1oZWlnaHQ6MTgwcHg7cGFkZGluZzozMHB4IDAgMTVweH0qID4gYm9keXtiYWNrZ3JvdW5kOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9lcnJvcnMvcm9ib3QucG5nKSAxMDAlIDVweCBuby1yZXBlYXQ7cGFkZGluZy1yaWdodDoyMDVweH1we21hcmdpbjoxMXB4IDAgMjJweDtvdmVyZmxvdzpoaWRkZW59aW5ze2NvbG9yOiM3Nzc7dGV4dC1kZWNvcmF0aW9uOm5vbmV9YSBpbWd7Ym9yZGVyOjB9QG1lZGlhIHNjcmVlbiBhbmQgKG1heC13aWR0aDo3NzJweCl7Ym9keXtiYWNrZ3JvdW5kOm5vbmU7bWFyZ2luLXRvcDowO21heC13aWR0aDpub25lO3BhZGRpbmctcmlnaHQ6MH19I2xvZ297YmFja2dyb3VuZDp1cmwoLy93d3cuZ29vZ2xlLmNvbS9pbWFnZXMvbG9nb3MvZXJyb3JwYWdlL2Vycm9yX2xvZ28tMTUweDU0LnBuZykgbm8tcmVwZWF0O21hcmdpbi1sZWZ0Oi01cHh9QG1lZGlhIG9ubHkgc2NyZWVuIGFuZCAobWluLXJlc29sdXRpb246MTkyZHBpKXsjbG9nb3tiYWNrZ3JvdW5kOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9sb2dvcy9lcnJvcnBhZ2UvZXJyb3JfbG9nby0xNTB4NTQtMngucG5nKSBuby1yZXBlYXQgMCUgMCUvMTAwJSAxMDAlOy1tb3otYm9yZGVyLWltYWdlOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9sb2dvcy9lcnJvcnBhZ2UvZXJyb3JfbG9nby0xNTB4NTQtMngucG5nKSAwfX1AbWVkaWEgb25seSBzY3JlZW4gYW5kICgtd2Via2l0LW1pbi1kZXZpY2UtcGl4ZWwtcmF0aW86Mil7I2xvZ297YmFja2dyb3VuZDp1cmwoLy93d3cuZ29vZ2xlLmNvbS9pbWFnZXMvbG9nb3MvZXJyb3JwYWdlL2Vycm9yX2xvZ28tMTUweDU0LTJ4LnBuZykgbm8tcmVwZWF0Oy13ZWJraXQtYmFja2dyb3VuZC1zaXplOjEwMCUgMTAwJX19I2xvZ297ZGlzcGxheTppbmxpbmUtYmxvY2s7aGVpZ2h0OjU0cHg7d2lkdGg6MTUwcHh9CiAgPC9zdHlsZT4KICA8YSBocmVmPS8vd3d3Lmdvb2dsZS5jb20vPjxzcGFuIGlkPWxvZ28gYXJpYS1sYWJlbD1Hb29nbGU+PC9zcGFuPjwvYT4KICA8cD48Yj40MDQuPC9iPiA8aW5zPlRoYXTigJlzIGFuIGVycm9yLjwvaW5zPgogIDxwPiAgPGlucz5UaGF04oCZcyBhbGwgd2Uga25vdy48L2lucz4K",
              "ok": false,
              "headers": [
                [
                  "content-length",
                  "1449"
                ],
                [
                  "content-type",
                  "text/html; charset=utf-8"
                ]
              ],
              "status": 404,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 171
        },
        "id": "yRWLVH47GV1-",
        "outputId": "71886e03-c4b6-4e11-c1e4-0a7b5f78a3ca"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.Video object>"
            ],
            "text/html": [
              "<video src=\"output_LK.mp4\" controls  >\n",
              "      Your browser does not support the <code>video</code> element.\n",
              "    </video>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "IPython.display.Video('output_LK.mp4')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DIveo_ZPGV1-"
      },
      "source": [
        "### Вопрос 2\n",
        "\n",
        "Какие проблемы в текущей реализации вы увидели при просмотре результирующего видео? Как их можно устранить?\n",
        "\n",
        "**Ответ:** Треки начинают формироваться с самого начала видео, но нет критерия остановки отслеживания. К тому же треккинг для объектов, появившихся в середине видео, не происходит.  \n",
        "Возможно решение - ограничения жизни треков по времени/динамике/достижению конца изображения и/или можно запускать параллельно алгоритм с каким-то временным сдвигом для каждого из них и затем агрегировать треки на итоговом изображении, применяя какой-то фильтр aka NMS  "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MTwmPT6NGV1-"
      },
      "source": [
        "### Задание 2\n",
        "\n",
        "Напишите код, устраняющий одну из проблем, покажите результат до/после."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def process_video_segment(start_frame, end_frame, cap, feature_params, lk_params, color, out):\n",
        "    cap.set(cv2.CAP_PROP_POS_FRAMES, start_frame)\n",
        "\n",
        "    # Initialize the first frame of the segment\n",
        "    ret, old_frame = cap.read()\n",
        "    old_gray = cv2.cvtColor(old_frame, cv2.COLOR_BGR2GRAY)\n",
        "    p0 = cv2.goodFeaturesToTrack(old_gray, mask=None, **feature_params)\n",
        "    mask = np.zeros_like(old_frame)\n",
        "\n",
        "    for i in range(start_frame, end_frame):\n",
        "        ret, frame = cap.read()\n",
        "        if not ret:\n",
        "            break\n",
        "\n",
        "        frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)\n",
        "        p1, st, err = cv2.calcOpticalFlowPyrLK(old_gray, frame_gray, p0, None, **lk_params)\n",
        "\n",
        "        if p1 is not None:\n",
        "            good_new = p1[st==1]\n",
        "            good_old = p0[st==1]\n",
        "\n",
        "            for i, (new, old) in enumerate(zip(good_new, good_old)):\n",
        "                a, b = new.ravel()\n",
        "                c, d = old.ravel()\n",
        "                mask = cv2.line(mask, (int(a), int(b)), (int(c), int(d)), color[i].tolist(), 2)\n",
        "                frame = cv2.circle(frame, (int(a), int(b)), 5, color[i].tolist(), -1)\n",
        "\n",
        "            img = cv2.add(frame, mask)\n",
        "            old_gray = frame_gray.copy()\n",
        "            p0 = good_new.reshape(-1, 1, 2)\n",
        "        else:\n",
        "            mask = np.zeros_like(old_frame)\n",
        "\n",
        "        out.write(img)"
      ],
      "metadata": {
        "id": "6gLyBXTbQX-j"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "video_path = '/content/data/slow_traffic_small.mp4'\n",
        "cap = cv2.VideoCapture(video_path)\n",
        "\n",
        "fps = cap.get(cv2.CAP_PROP_FPS)\n",
        "n_frames_per_segment = int(fps * 4)\n",
        "\n",
        "fourcc = cv2.VideoWriter_fourcc(*'MP4V')\n",
        "out = cv2.VideoWriter('output_LK_new.mp4', fourcc, fps, (width, height))\n",
        "\n",
        "total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))\n",
        "segments = [(start, min(start + n_frames_per_segment, total_frames)) for start in range(0, total_frames, n_frames_per_segment)]\n",
        "\n",
        "for start_frame, end_frame in segments:\n",
        "    process_video_segment(start_frame, end_frame, cap, feature_params, lk_params, color, out)\n",
        "\n",
        "cap.release()\n",
        "out.release()"
      ],
      "metadata": {
        "id": "QGymuMuhQX78"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Реализован параллельный запуск алгоримта каждые 4 секунды видео, что позволяет избавиться от проблем"
      ],
      "metadata": {
        "id": "iBekCTDlQ_uz"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPk-hb1pGV1-"
      },
      "source": [
        "## Farneback (dense)\n",
        "\n",
        "Метод Farneback носит несколько более глобальный характер, чем метод Лукаса-Канаде. Он опирается на предположение о том, что на всем изображении оптический поток будет достаточно гладким."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Mg9AYvEaGV1_"
      },
      "outputs": [],
      "source": [
        "cap = cv2.VideoCapture(video_path)\n",
        "\n",
        "length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))\n",
        "width  = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))\n",
        "height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))\n",
        "fps    = cap.get(cv2.CAP_PROP_FPS)\n",
        "\n",
        "fourcc = cv2.VideoWriter_fourcc(*'MP4V')\n",
        "out = cv2.VideoWriter('output_Farneback.mp4', fourcc, fps, (width, height))\n",
        "\n",
        "ret, frame1 = cap.read()\n",
        "prvs = cv2.cvtColor(frame1, cv2.COLOR_BGR2GRAY)\n",
        "hsv = np.zeros_like(frame1)\n",
        "hsv[..., 1] = 255\n",
        "\n",
        "for i in tqdm(range(length)):\n",
        "\n",
        "    ret, frame2 = cap.read()\n",
        "\n",
        "    if not ret:\n",
        "        print('No frames grabbed!')\n",
        "        break\n",
        "\n",
        "    next = cv2.cvtColor(frame2, cv2.COLOR_BGR2GRAY)\n",
        "\n",
        "    #see arguments here https://docs.opencv.org/3.4/dc/d6b/group__video__track.html#ga5d10ebbd59fe09c5f650289ec0ece5af\n",
        "    flow = cv2.calcOpticalFlowFarneback(prvs, next, None, 0.5, 3, 15, 3, 5, 1.2, 0)\n",
        "    mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])\n",
        "    hsv[..., 0] = ang*180/np.pi/2\n",
        "    hsv[..., 2] = cv2.normalize(mag, None, 0, 255, cv2.NORM_MINMAX)\n",
        "    bgr = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)\n",
        "    prvs = next\n",
        "\n",
        "    out.write(bgr)\n",
        "\n",
        "cap.release()\n",
        "out.release()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1xs89RlwGV1_"
      },
      "source": [
        "Посмотрим, что получилось"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "resources": {
            "http://localhost:8080/output_Farneback.mp4": {
              "data": "CjwhRE9DVFlQRSBodG1sPgo8aHRtbCBsYW5nPWVuPgogIDxtZXRhIGNoYXJzZXQ9dXRmLTg+CiAgPG1ldGEgbmFtZT12aWV3cG9ydCBjb250ZW50PSJpbml0aWFsLXNjYWxlPTEsIG1pbmltdW0tc2NhbGU9MSwgd2lkdGg9ZGV2aWNlLXdpZHRoIj4KICA8dGl0bGU+RXJyb3IgNDA0IChOb3QgRm91bmQpISExPC90aXRsZT4KICA8c3R5bGU+CiAgICAqe21hcmdpbjowO3BhZGRpbmc6MH1odG1sLGNvZGV7Zm9udDoxNXB4LzIycHggYXJpYWwsc2Fucy1zZXJpZn1odG1se2JhY2tncm91bmQ6I2ZmZjtjb2xvcjojMjIyO3BhZGRpbmc6MTVweH1ib2R5e21hcmdpbjo3JSBhdXRvIDA7bWF4LXdpZHRoOjM5MHB4O21pbi1oZWlnaHQ6MTgwcHg7cGFkZGluZzozMHB4IDAgMTVweH0qID4gYm9keXtiYWNrZ3JvdW5kOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9lcnJvcnMvcm9ib3QucG5nKSAxMDAlIDVweCBuby1yZXBlYXQ7cGFkZGluZy1yaWdodDoyMDVweH1we21hcmdpbjoxMXB4IDAgMjJweDtvdmVyZmxvdzpoaWRkZW59aW5ze2NvbG9yOiM3Nzc7dGV4dC1kZWNvcmF0aW9uOm5vbmV9YSBpbWd7Ym9yZGVyOjB9QG1lZGlhIHNjcmVlbiBhbmQgKG1heC13aWR0aDo3NzJweCl7Ym9keXtiYWNrZ3JvdW5kOm5vbmU7bWFyZ2luLXRvcDowO21heC13aWR0aDpub25lO3BhZGRpbmctcmlnaHQ6MH19I2xvZ297YmFja2dyb3VuZDp1cmwoLy93d3cuZ29vZ2xlLmNvbS9pbWFnZXMvbG9nb3MvZXJyb3JwYWdlL2Vycm9yX2xvZ28tMTUweDU0LnBuZykgbm8tcmVwZWF0O21hcmdpbi1sZWZ0Oi01cHh9QG1lZGlhIG9ubHkgc2NyZWVuIGFuZCAobWluLXJlc29sdXRpb246MTkyZHBpKXsjbG9nb3tiYWNrZ3JvdW5kOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9sb2dvcy9lcnJvcnBhZ2UvZXJyb3JfbG9nby0xNTB4NTQtMngucG5nKSBuby1yZXBlYXQgMCUgMCUvMTAwJSAxMDAlOy1tb3otYm9yZGVyLWltYWdlOnVybCgvL3d3dy5nb29nbGUuY29tL2ltYWdlcy9sb2dvcy9lcnJvcnBhZ2UvZXJyb3JfbG9nby0xNTB4NTQtMngucG5nKSAwfX1AbWVkaWEgb25seSBzY3JlZW4gYW5kICgtd2Via2l0LW1pbi1kZXZpY2UtcGl4ZWwtcmF0aW86Mil7I2xvZ297YmFja2dyb3VuZDp1cmwoLy93d3cuZ29vZ2xlLmNvbS9pbWFnZXMvbG9nb3MvZXJyb3JwYWdlL2Vycm9yX2xvZ28tMTUweDU0LTJ4LnBuZykgbm8tcmVwZWF0Oy13ZWJraXQtYmFja2dyb3VuZC1zaXplOjEwMCUgMTAwJX19I2xvZ297ZGlzcGxheTppbmxpbmUtYmxvY2s7aGVpZ2h0OjU0cHg7d2lkdGg6MTUwcHh9CiAgPC9zdHlsZT4KICA8YSBocmVmPS8vd3d3Lmdvb2dsZS5jb20vPjxzcGFuIGlkPWxvZ28gYXJpYS1sYWJlbD1Hb29nbGU+PC9zcGFuPjwvYT4KICA8cD48Yj40MDQuPC9iPiA8aW5zPlRoYXTigJlzIGFuIGVycm9yLjwvaW5zPgogIDxwPiAgPGlucz5UaGF04oCZcyBhbGwgd2Uga25vdy48L2lucz4K",
              "ok": false,
              "headers": [
                [
                  "content-length",
                  "1449"
                ],
                [
                  "content-type",
                  "text/html; charset=utf-8"
                ]
              ],
              "status": 404,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 171
        },
        "id": "8oaZCQCwGV1_",
        "outputId": "a99f17c4-4f85-4c87-d0ee-75fd4e6c5dee"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<IPython.core.display.Video object>"
            ],
            "text/html": [
              "<video src=\"output_Farneback.mp4\" controls  >\n",
              "      Your browser does not support the <code>video</code> element.\n",
              "    </video>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "IPython.display.Video('output_Farneback.mp4')"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "MzrIoVdSOAxp"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.13"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}