L1ga-Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Постановка задачи"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RMyD7bmweg40"
   },
   "source": [
    "Подчеркнуть из комментариев:\n",
    "* Заинтересованность в определённых продуктах\n",
    "* Сезонность (когда гарантия, участие в конкретном тендере)\n",
    "* Лизинг (машины, оборудование, сезонность)\n",
    "\n",
    "Итоговый результат - Краткая справка по клиенту.\n",
    "\n",
    "`Контакт с ГД. Сказал есть 3 расчетных счета, больше не надо. Есть овер в Альфе, рефить не хотят. Депозиты не интересуют, так как % в ВТБ выгодней. Сам является Клиентом Совкомбанка, как физик, перезвонить через месяц`\n",
    "\n",
    "Использование Named Entity Recognition для выделения сущностей таких как 'расчётные счета', 'овердрафт', 'депозиты' и т.д.\n",
    "\n",
    "```python\n",
    "bank_products = {\n",
    "    'расчётные счета': [],\n",
    "    'овердрафт': False,\n",
    "    'депозиты': False\n",
    "}\n",
    "\n",
    "for ent in doc.ents:\n",
    "    if 'расчётный счёт' in ent.text:\n",
    "        bank_products['расчётные счета'].append(ent.text)\n",
    "    elif 'овердрафт' in ent.text:\n",
    "        bank_products['овердрафт'].append(ent.text)\n",
    "```\n",
    "\n",
    "```python\n",
    "overdraft_info = {\n",
    "    'банк': None,\n",
    "    'сумма': None,\n",
    "}\n",
    "\n",
    "for ent on doc.ents:\n",
    "    if 'овердрафт' in ent.text:\n",
    "        overdraft_info['банк'] = ent.text\n",
    "    elif 'млн руб' in ent.text:\n",
    "        overdraft_info['сумма'] = ent.text\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aEPYAvDvB1B0"
   },
   "source": [
    "План проекта\n",
    "* Обучаем на выделение сущностей PRODUCT\n",
    "* Находим сущности в комментарии и по шаблону-паттернам находим, где он находится, сумму, отношение\n",
    "\n",
    "TAG | MORPH | description\n",
    "--- | --- | ---\n",
    "ROOT | СКАЗУЕМОЕ |\n",
    "NMOD | NOUN MODIFIER | зависимое от главного существительное слово ((в) альфе овер)\n",
    "NSUBJ | NOMINAL SUBJECT | подлежащее (овердрафт устраивает, депозиты интересуют)\n",
    "XCOMP | OPEN CLAUSAL COMPLEMENT| глагол / прилагательное, которое требует завершения фразы\n",
    "OBL | OBLIQUE NOMINAL | обстоятельство ((в) лизинг интересует)\n",
    "ADVMOD | ADVERBIAL MODIFIER | модификатор наречия, слово или фраза, которое уточняет значение наречия в предложении\n",
    "CONJ | CONJUNCTION | союз\n",
    "OBJ | OBJECT | существительно, на которое направлено действие\n",
    "ADJ | ADJECTIVE | прилагательное\n",
    "\n",
    "* Разметка по позитивным действиям и негативным действиям\n",
    "* **Если сущность в предложении, то выделяем часть предложения до знака препинания**\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jSKcTc0oLJnh"
   },
   "outputs": [],
   "source": [
    "!pip install spacy\n",
    "!pip install optuna\n",
    "!python -m spacy download ru_core_news_md #\n",
    "# pip install  ru_core_news_sm-3.7.0-py3-none-any.whl\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import optuna\n",
    "import spacy\n",
    "import time\n",
    "from tqdm import tqdm\n",
    "from spacy import displacy\n",
    "from spacy.training import Example\n",
    "from sklearn.utils import shuffle\n",
    "\n",
    "nlp = spacy.load('ru_core_news_md')\n",
    "\n",
    "ner = nlp.get_pipe('ner')\n",
    "ner.add_label('PROD')\n",
    "\n",
    "available_entities = nlp.get_pipe('ner').labels\n",
    "print('Доступные сущности:', available_entities)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "HPoMVnnxcO7B"
   },
   "outputs": [],
   "source": [
    "def dict_to_form(dictlist):\n",
    "    formed_list = []\n",
    "    for elem in dictlist:\n",
    "        for key, value in elem.items():\n",
    "            doc = nlp(value)\n",
    "            elem[key] = extract_main_idea(doc)\n",
    "            formed_list.append(elem)\n",
    "    return formed_list\n",
    "\n",
    "#ФУНКЦИЯ ИЗВЛЕЧЕНИЯ ГЛАВНОЙ МЫСЛИ\n",
    "def extract_main_idea(sentence):\n",
    "    main_idea = []\n",
    "    for token in sentence:\n",
    "        if token.dep_ not in ['punct']:\n",
    "            if token.dep_ == 'ROOT' or token.dep_ == 'conj':\n",
    "                main_idea.append(token.text)\n",
    "            elif token.dep_ == 'nsubj' and token.head.dep_ == 'ROOT':\n",
    "                main_idea.append(token.text)\n",
    "    return ' '.join(main_idea)\n",
    "\n",
    "#СПИСОК СЛОВАРЕЙ -> СПИСОК СЛОВАРЕЙ С ОБРАБОТАННЫМИ КЛЮЧАМИ\n",
    "data_to_train['main_idea'] = data_to_train['entlist'].apply(dict_to_form)\n",
    "\n",
    "# pd.set_option('display.max_colwidth', None)\n",
    "# pd.set_option('display.max_rows', None)\n",
    "\n",
    "data_to_train[['text', 'label', 'main_idea']]\n",
    "\n",
    "#ПЕРЕПИСЫВАЕМ ФУНКЦИЮ ИЗВЛЕЧЕНИЯ ГЛАВНОЙ МЫСЛИ\n",
    "def extract_main_idea(sentence):\n",
    "    main_idea = []\n",
    "    for token in sentence:\n",
    "        if not token.ent_type_:\n",
    "            if token.dep_ == 'ROOT':\n",
    "                main_idea.append(token.text)\n",
    "            if token.head.dep_ == 'ROOT' and token.dep_ in ['XCOMP', 'NMOD', 'ADVMOD']:\n",
    "                main_idea.append(token.text)\n",
    "    return ' '.join(main_idea)\n",
    "\n",
    "train_data = [\n",
    "    ('Мы редко ходим в кино', {'heads': [2, 2, 2, 0, 2], 'deps': ['nsubj', 'advmod', 'ROOT', 'prep', 'advmod']}),\n",
    "    ('Депозиты не интересуют', {'heads': [3, 3, 0], 'deps': ['nsubj', 'neg', 'ROOT']})\n",
    "\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "W_FxYzZ_gQZJ"
   },
   "source": [
    "`.rfind` - это метод строки в Python, который выполняет обратный поиск подстроки в строке. Он возвращает **индекс** последнего вхождения подстроки. </br>\n",
    "Если подстрока не найдена, возвращается -1 . </br>\n",
    "В приведённых примерах метод `.rfind` используется для определения позиции знаков препинания, предшествующих упоминаниям сущностей.\n",
    "\n",
    "```python\n",
    "sentence_start = max(text.rfind('.', 0, token.idx), text.find(',', 0, token_idx), 0)\n",
    "```\n",
    "Использование `.rfind` позволяет определить позицию последнего вхождения знака препинания перед упоминанием сущности в тексте.\n",
    "\n",
    "`.strip` - это метод в Python, который используется для удаления начальных (слева) и конечных (справа) пробелов или других указанных символов из строки."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "s-9iQO5BectJ",
    "outputId": "c9618969-d04b-431c-d65c-d940ce9b61c6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "овердрафт: имеется овердрафт в ВТБ 40 млн руб под 9\n",
      "реф: реф и вкл не интересует\n",
      "овердрафт: полностью устраивает овердрафт\n",
      "лизинг: кроме лизинга\n",
      "лизинг: В лизинг интересует оборудование\n",
      "рефить: рефить не хотят\n",
      "депозит: Депозиты не интересуют\n"
     ]
    }
   ],
   "source": [
    "for index, row in comment_data.iterrows():\n",
    "    text = row['comment']\n",
    "    doc = nlp(text)\n",
    "    for token in doc:\n",
    "        for entity in ['овердрафт', 'бкл', 'гарантия', 'реф', 'депозит', 'рефить', 'лизинг']:\n",
    "            if token.lemma_.lower() == entity:\n",
    "                sentence_start = max(text.rfind('.', 0, token.idx), text.rfind(',', 0, token.idx), 0)\n",
    "                min_nums = [text.find('.', token.idx, len(text)), text.find(',', token.idx, len(text)), len(text)]\n",
    "                sentence_end = min(num for num in min_nums if num != -1)\n",
    "                entity_text = text[sentence_start + 1 : sentence_end].strip()\n",
    "                print(f'{entity}: {entity_text}')\n",
    "\n",
    "#ПЕРЕХОДИМ К ИНСТРУМЕНТАМ SPACY, ТАК КАК СТАНДАРТНЫЕ ИНСТРУМЕНТЫ PYTHON ОБРЕЗАЮТ ТОКЕНЫ ВРОДЕ 9,8%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dc20u2-S9ayn"
   },
   "source": [
    "* [Пример похожей задачи на Kaggle](https://www.kaggle.com/code/rohitsingh9990/ner-training-using-spacy-ensemble/notebook)\n",
    "* [Обсуждение решения задачи](https://www.kaggle.com/code/rohitsingh9990/ner-training-using-spacy-ensemble/comments)\n",
    "* [Тренировочный датасет](https://www.kaggle.com/competitions/tweet-sentiment-extraction/data?select=train.csv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5KqRfHy0zhF7"
   },
   "source": [
    "Выделяем главную мысль из текста\n",
    "```python\n",
    "def extract_main_idea(sentence):\n",
    "    main_idea = []\n",
    "    for token in sentence:\n",
    "        if token.dep_ not in ['punct', 'neg']:\n",
    "            if token.dep_ == 'ROOT' or token.dep_ == 'conj':\n",
    "                main_idea.append(token.text)\n",
    "            elif token.dep_ == 'nsubj' and token.head.dep_ == 'ROOT':\n",
    "                main_idea.append(token.text)\n",
    "    return ' '.join(main_idea)\n",
    "\n",
    "```\n",
    "* aspect-based sentiment analysis\n",
    "* spacy dependency parser: NER достанет аспект и все слова, которые к нему относятся\n",
    "* spacy dependency parcer дообучать не нужно (это уже обученная модель и самому дообучить её не так просто), ей нужно пользоваться, чтобы извлекать морфологические зависимости в данных\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SqEvqhWGkvDa"
   },
   "source": [
    "Дообучение векторных представлений сводится к простой передаче текстовых данных для обновления векторных представлений. </br>\n",
    "Обучение векторного пространства в spaCy не затрагивает independence parser и качество определения части речи напрямую. Векторное представление обучается с использованием нейронной сети, которая учится предсказать контекстное окружение слова в тексте. Это позволяет модели понимать семантические отношения между словами и использовать их для различные задачЮ такие как сравнение синонимов или классификация текста. </br>\n",
    "Independence parser и определение части речи (Part-of-Speech tagging) в spaCy основаны на других компонентах модели, таких как морфологический анализатор и синтаксический анализатор. Эти компоненты модели обучаются отдельно и не зависят от обучения векторного пространства. </br>\n",
    "\n",
    "```python\n",
    "from spacy.util import compounding\n",
    "from spacy.util import minibatch\n",
    "```\n",
    "\n",
    "```python\n",
    "# Вычисляем косинусное расстояние между векторами\n",
    "similarity = vector_cat.dot(vector_dog) / (vector_cat.norm() * vector_dog.norm())\n",
    "```\n",
    "```python\n",
    "# Токенизируем и лемматизируем текст\n",
    "processed_corpus = []\n",
    "for doc in nlp.pipe(corpus):\n",
    "    processed_corpus.append([token.lemma_ for token in doc])\n",
    "```\n",
    "Ниже пример функции, которая очищает текст от стоп-слов, пунктуации, лишних пробелов, чисел и возвращает лемматизированный текст.\n",
    "```python\n",
    "def lemmatize(doc):\n",
    "    words = []\n",
    "    for token in doc:\n",
    "        if (token.is_stop != True) and (token.is_punct != True) and\\\n",
    "            (token.is_space != True) and (token.is_digit != True):\n",
    "            words.append(token.lemma_)\n",
    "    return ' '.join(words)\n",
    "```\n",
    "```python\n",
    "def train_model(nlp, data, n_iter=10):\n",
    "    for i in tange(n_iter):\n",
    "        random.shuffle(data)\n",
    "        losses = {}\n",
    "\n",
    "        for batch in minibatch(data, size=compounding(4.0, 32.0, 1.001)):\n",
    "            texts, - = zip(*batch)\n",
    "            nlp.update(texts, losses = losses)\n",
    "\n",
    "        print('Эпоха {}: Потери: {}'.format(i+1, losses))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UhEieS459aSX"
   },
   "source": [
    "```python\n",
    "from spacy.training import Example\n",
    "\n",
    "#ЗАМЕНИТЬ TRAIN_DATA НА РАЗМЕЧЕННЫЙ ДАТАСЕТ\n",
    "for i in range(10):\n",
    "    random.shuffle(train_data)\n",
    "    losses = {}\n",
    "    for text, annotations in train_data:\n",
    "        doc = nlp.make_doc(text)\n",
    "        example = Example.from_dict(doc, annotations)\n",
    "        nlp.update([example], drop=0.5, losses=losses)\n",
    "    print('Эпоха', i, ':', losses)\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PCkL81QVVIXE"
   },
   "outputs": [],
   "source": [
    "data_to_train = pd.read_excel('train_data.xlsx')\n",
    "\n",
    "# ФУНКЦИЯ ПРИВЕДЕНИЯ ДАННЫХ К ТИПУ ТЕКСТ - СУЩНОСТЬ\n",
    "def get_train_data(data_to_train):\n",
    "    train_data = []\n",
    "    for index, row in data_to_train.iterrows():\n",
    "        text = row.text\n",
    "        selected_text = row.selected_text\n",
    "        entity_list = []\n",
    "        for elem in selected_text.split(sep = ','):\n",
    "            elem = elem.strip()\n",
    "            start = text.find(elem)\n",
    "            end = start + len(elem)\n",
    "            entity_list.append((start, end, 'PROD'))\n",
    "        train_data.append((text, {'entities': entity_list}))\n",
    "    return train_data\n",
    "\n",
    "# ПОЛУЧЕНИЕ ДАННЫХ\n",
    "train_data = get_train_data(data_to_train)\n",
    "\n",
    "# ОБУЧЕНИЕ\n",
    "for i in range(10):\n",
    "    train_data = shuffle(train_data)\n",
    "    losses = {}\n",
    "    try:\n",
    "        for text, annotations in train_data:\n",
    "            doc = nlp.make_doc(text)\n",
    "            example = Example.from_dict(doc, annotations)\n",
    "            nlp.update([example], drop=0.1, losses=losses)\n",
    "    except ValueError as e:\n",
    "        print(f'Ошибка при обновлении модели: {e}')\n",
    "    print('Эпоха', i, ':', losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RFtziSS1VdDU",
    "outputId": "3fb54c4a-5c15-4704-9ace-53ed7f779780"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Эпоха 0 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 2.71537340282563e-12}\n",
      "Эпоха 1 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 1.118470388899019e-12}\n",
      "Эпоха 2 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 2.620586592219322e-09}\n",
      "Эпоха 3 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 1.106385799660583e-10}\n",
      "Эпоха 4 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 1.0771147732686626e-10}\n",
      "Эпоха 5 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 1.9575938102622994e-14}\n",
      "Эпоха 6 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 4.959736954821176e-12}\n",
      "Эпоха 7 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 1.0012769523486468e-14}\n",
      "Эпоха 8 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 2.2191134165643226e-12}\n",
      "Эпоха 9 : {'tok2vec': 0.0, 'morphologizer': 0.0, 'parser': 0.0, 'ner': 9.538061576222029e-14}\n"
     ]
    }
   ],
   "source": [
    "training_data = [\n",
    "    ('бк с гд, попросил набрать в пн', {\"entities\": []})\n",
    "    ]\n",
    "\n",
    "train_examples = []\n",
    "for text, annotations in training_data:\n",
    "    doc = nlp.make_doc(text)\n",
    "    example = Example.from_dict(doc, annotations)\n",
    "    train_examples.append(example)\n",
    "\n",
    "for epoch in range(10):\n",
    "    train_examples = shuffle(train_examples)\n",
    "    losses = {}\n",
    "    for example in train_examples:\n",
    "        nlp.update([example], drop = 0.1, losses = losses)\n",
    "    print('Эпоха', epoch, ':', losses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-aHx4nQG7agz",
    "outputId": "8d449a7d-e096-4d0f-acdd-6d5cf3816312"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['кредите', 'лизинг']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doc = nlp('бк с гд. потребности в кредите.! лизинг нет потребности.')\n",
    "[ent.text for ent in doc.ents]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "YlvTAmus7oTg",
    "outputId": "40504424-8ca4-449c-b34e-bc3fa92ab34b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'кредит': 'потребности в кредите '}, {'лизинг': '! лизинг нет потребности '}]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entlist_to_form('бк с гд. потребности в кредите.! лизинг нет потребности.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8gqzmOnQHB63"
   },
   "outputs": [],
   "source": [
    "# ОСТАВЛЯЕМ ТОЛЬКО СУЩНОСТИ И ИХ ОКРУЖЕНИЕ\n",
    "def entlist_to_form(comment):\n",
    "    comment = comment.lower()\n",
    "    partlist = []\n",
    "    entlist = []\n",
    "    # Пустая строки\n",
    "    current_part = ''\n",
    "    doc = nlp(comment)\n",
    "    for token in doc:\n",
    "        if (token.text == ',') or (token.text == '.'):\n",
    "            partlist.append(current_part)\n",
    "            current_part = ''\n",
    "        else:\n",
    "            current_part += token.text\n",
    "            current_part += ' '\n",
    "    # Если строка НЕ пустая, добавим часть предложения, в список\n",
    "    if current_part:\n",
    "        partlist.append(current_part)\n",
    "    # Пройдём по ранее заготовленному списку partlist\n",
    "    for part in partlist:\n",
    "        for ent in doc.ents:\n",
    "            if ent.label_ == 'PROD' and ent.text in part:\n",
    "                entlist.append({ent.lemma_: part}) #ПРИВОДИМ К ФОРМАТУ СУЩНОСТЬ-ЗНАЧЕНИЕ\n",
    "    return entlist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "id": "gxT0sj7vGsmn",
    "outputId": "19376b36-1ce6-4401-eac5-2a7b38275fc8"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-16bef38a-38cc-45a5-b92b-6b886e0c6b80\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>label</th>\n",
       "      <th>entlist</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>кт с  бухгалтером, предложение лизинга и кик, ...</td>\n",
       "      <td>[(лизинг, PROD), (кик, PROD)]</td>\n",
       "      <td>[{'лизинг': 'предложение лизинга и кик '}, {'к...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>закупка неакутальна</td>\n",
       "      <td>[(закупка, PROD)]</td>\n",
       "      <td>[{'закупка': 'закупка неакутальна '}]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>бк с фд, // есть потребность в лизинге и вкл 5...</td>\n",
       "      <td>[(лизинг, PROD), (вкл, PROD), (депозит, PROD),...</td>\n",
       "      <td>[{'лизинг': '// есть потребность в лизинге и в...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>бк с гд. потребности в кредите.! лизинг нет по...</td>\n",
       "      <td>[(кредит, PROD), (лизинг, PROD)]</td>\n",
       "      <td>[{'кредит': 'потребности в кредите '}, {'лизин...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>кт, все также кредитование, депозиты, бг неинт...</td>\n",
       "      <td>[(кредитование, PROD), (депозит, PROD), (бг, P...</td>\n",
       "      <td>[{'кредитование': 'все также кредитование '}, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>191</th>\n",
       "      <td>потребность в бг, вкс есть основная анкета, вс...</td>\n",
       "      <td>[(бг, PROD), (вкс, PROD)]</td>\n",
       "      <td>[{'бг': 'потребность в бг '}, {'вкс': 'вкс ест...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>бк с гд, очень сильно связана с втб, не уйти о...</td>\n",
       "      <td>[(лизинг, PROD), (факторинг, PROD)]</td>\n",
       "      <td>[{'лизинг': 'пробовать лизинг + факторинг пред...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>193</th>\n",
       "      <td>общение с евгением, сказал звонить после новог...</td>\n",
       "      <td>[(деньга, PROD), (кик, PROD)]</td>\n",
       "      <td>[{'деньга': 'деньги нужны на пол года '}, {'ки...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>194</th>\n",
       "      <td>ответил фд. в ближайшей перспективе не будут м...</td>\n",
       "      <td>[(гарантия, PROD), (кредит, PROD)]</td>\n",
       "      <td>[{'гарантия': 'гб принимает решение по банковс...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>бк с гд, был на встрече, попросил перенести зв...</td>\n",
       "      <td>[(финансирование, PROD)]</td>\n",
       "      <td>[{'финансирование': 'что сейчас есть потребнос...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>196 rows × 3 columns</p>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-16bef38a-38cc-45a5-b92b-6b886e0c6b80')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-16bef38a-38cc-45a5-b92b-6b886e0c6b80 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-16bef38a-38cc-45a5-b92b-6b886e0c6b80');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-ea9acb9d-8027-477f-9b0d-10dca7e7e787\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ea9acb9d-8027-477f-9b0d-10dca7e7e787')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-ea9acb9d-8027-477f-9b0d-10dca7e7e787 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                                  text  \\\n",
       "0    кт с  бухгалтером, предложение лизинга и кик, ...   \n",
       "1                                  закупка неакутальна   \n",
       "2    бк с фд, // есть потребность в лизинге и вкл 5...   \n",
       "3    бк с гд. потребности в кредите.! лизинг нет по...   \n",
       "4    кт, все также кредитование, депозиты, бг неинт...   \n",
       "..                                                 ...   \n",
       "191  потребность в бг, вкс есть основная анкета, вс...   \n",
       "192  бк с гд, очень сильно связана с втб, не уйти о...   \n",
       "193  общение с евгением, сказал звонить после новог...   \n",
       "194  ответил фд. в ближайшей перспективе не будут м...   \n",
       "195  бк с гд, был на встрече, попросил перенести зв...   \n",
       "\n",
       "                                                 label  \\\n",
       "0                        [(лизинг, PROD), (кик, PROD)]   \n",
       "1                                    [(закупка, PROD)]   \n",
       "2    [(лизинг, PROD), (вкл, PROD), (депозит, PROD),...   \n",
       "3                     [(кредит, PROD), (лизинг, PROD)]   \n",
       "4    [(кредитование, PROD), (депозит, PROD), (бг, P...   \n",
       "..                                                 ...   \n",
       "191                          [(бг, PROD), (вкс, PROD)]   \n",
       "192                [(лизинг, PROD), (факторинг, PROD)]   \n",
       "193                      [(деньга, PROD), (кик, PROD)]   \n",
       "194                 [(гарантия, PROD), (кредит, PROD)]   \n",
       "195                           [(финансирование, PROD)]   \n",
       "\n",
       "                                               entlist  \n",
       "0    [{'лизинг': 'предложение лизинга и кик '}, {'к...  \n",
       "1                [{'закупка': 'закупка неакутальна '}]  \n",
       "2    [{'лизинг': '// есть потребность в лизинге и в...  \n",
       "3    [{'кредит': 'потребности в кредите '}, {'лизин...  \n",
       "4    [{'кредитование': 'все также кредитование '}, ...  \n",
       "..                                                 ...  \n",
       "191  [{'бг': 'потребность в бг '}, {'вкс': 'вкс ест...  \n",
       "192  [{'лизинг': 'пробовать лизинг + факторинг пред...  \n",
       "193  [{'деньга': 'деньги нужны на пол года '}, {'ки...  \n",
       "194  [{'гарантия': 'гб принимает решение по банковс...  \n",
       "195  [{'финансирование': 'что сейчас есть потребнос...  \n",
       "\n",
       "[196 rows x 3 columns]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_to_train['label'] = data_to_train['text'].apply(lambda x: [(ent.lemma_, ent.label_) for ent in nlp(x).ents])\n",
    "data_to_train['entlist'] = data_to_train['text'].apply(entlist_to_form)\n",
    "\n",
    "data_to_train[['text', 'label', 'entlist']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Kavn_4eCNHun"
   },
   "source": [
    "Ниже пример функции, которая очищает текст от стоп-слов, пунктуации, лишних пробелов, чисел и возвращает лемматизированный текст.\n",
    "\n",
    "```python\n",
    "def lemmatize(doc):\n",
    "    words = []\n",
    "    for token in doc:\n",
    "        if (token.is_stop != True) and (token.is_punct != True) and\\\n",
    "            (token.is_space != True) and (token.is_digit != True):\n",
    "            words.append(token.lemma_)\n",
    "    return ' '.join(words)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q8bEy_YgS07M"
   },
   "source": [
    "Вот некоторые соображения, которые могут помочь принять решение о выборе слоёв и их количества:\n",
    "\n",
    "* Свёрточные слои `Conv1D`  эффективно работают с изображениями или текстовыми данными, где присутствует локальная структура текста (текст, где информация организуется внутри отдельных участков текста, таких как предложения и абзацы). Размер фильтров `kernel size` определяет размер области, которую рассматривает сверточный слой. Меньшие фильтры могут извлекать более мелкие особенности, в то время как большие фильтры могут улавливать более крупные особенности. Количество фильтров определяет количество различных признаков, которые извлекаются на каждом уровне свёртки.\n",
    "* Слои пулинга `MaxPooling` или `AveragePooling` уменьшают размерность данных путём агрегации информации из предыдущего слоя. Они помогают уменьшить количество параметров модели и предотвратить переобучение.\n",
    "* Полносвязанные слои `Dense` соединяются все нейроны из предыдущего слоя с каждым нейроном в текущем слое. Они обычно используются для обхединения информации из предыдущих слоёв перед выходом.\n",
    "\n",
    "Обычно рекомендуется начать с относительно простой модели и постепенно усложнять её, добавляя больше слоёв или изменяя их конфигурацию, пока не будет достигнуто желаемое качество предсказаний.\n",
    "\n",
    "Для задачи классификации текста с использованием CNN часто используется комбинация свёрточных слоёв для извлечения признаков из текста, за которым следуют слои пулинга для агрегации информации и полносвязные слои для классификации.\n",
    "\n",
    "```python\n",
    "from keras.model import Sequential\n",
    "from keras.layers import Embedding, Conv1D, GlobalMaxPooling1D, Dense\n",
    "from keras.preprocessing.sequence import pad_sequences\n",
    "from keras.utils import to_categorical\n",
    "\n",
    "# Подготовка данных\n",
    "X = comment_data['text']\n",
    "y = comment_data['sentiment']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42)\n",
    "\n",
    "# Преобразование текста в последовательности индексов слов\n",
    "X_train_sequences = w2v_model.wv[comment_data['text']]\n",
    "X_test_sequences = w2v_model.wv[X_test]\n",
    "\n",
    "# Инициализация весов word2vec\n",
    "embedding_matrix = w2v_model.wv.vectors\n",
    "\n",
    "num_words = len(w2v_model.wv.key_to_index) + 1\n",
    "output_dim = w2v_model.vector_size\n",
    "\n",
    "# Определение архитектуры модели\n",
    "model = Sequential()\n",
    "model.add(Embedding(input_dim = num_words, output_dim = embedding_dim, weights = [embedding_matrix], input_length = max_sequence_length, trainable = False))\n",
    "model.add(Conv1D(filters = 128, kernel_size = 5, activation = 'relu'))\n",
    "model.add(GlobalMaxPooling1D())\n",
    "model.add(Dense(3, activation = 'softmax'))\n",
    "\n",
    "# Компиляция модели\n",
    "model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
    "\n",
    "# Преобразование меток классов в категориальное представление\n",
    "y_train_categorical = to_categorical(y_train)\n",
    "y_test_categorical = to_categorical(y_test)\n",
    "\n",
    "# Обучение модели\n",
    "model.fit(X_train_sequences, y_train_categorical, epochs = 10, batch_size = 64, validation_split = 0.1)\n",
    "\n",
    "#Оценка модели\n",
    "loss, accuracy = model.evaluate(X_test_sequences, y_test_categorical)\n",
    "print('Loss:', loss)\n",
    "print('Accuracy:', accuracy)\n",
    "```\n",
    "\n",
    "Пример использования Optuna для оптимизации гиперпараметров модели\n",
    "\n",
    "```python\n",
    "import optuna\n",
    "from tensorflow.keras.model import Sequential\n",
    "from tensorflow.keras.layers import Conv1D, GlobalMaxPooling1, Dense\n",
    "\n",
    "def objective(trial):\n",
    "    # Define the hyperparameters to optimize\n",
    "    num_filters = trial.suggest_int('num_filters', 32, 128)\n",
    "    kernel_size = trial.suggest_int('kernel_size', 3, 7)\n",
    "    activation = trial.suggest_categorical('activation', ['relu', 'tanh', 'sigmoid'])\n",
    "\n",
    "    model = Sequential([\n",
    "        Conv1D(filters = num_filters, kernel_size = kernel_size, activation = activation, input_shape = input_shape),\n",
    "        GlobalMaxPooling1D(),\n",
    "        Dense(num_classes, activation = 'softmax')\n",
    "    ])\n",
    "\n",
    "    # Compile the model\n",
    "    model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
    "\n",
    "    # Train the model\n",
    "    model.fit(X_train, y_train, epochs = 10, batch_size = 32, validation_data = (X_val, y_val), verbose = 0)\n",
    "\n",
    "    # Evaluate the model\n",
    "    score = model.evaluate(X_val, y_val, verbose = 0)\n",
    "    return score[1]\n",
    "\n",
    "study = optuna.create_study(direction = 'maximize')\n",
    "study.optimize(objective, n_trials = 100)\n",
    "\n",
    "best_params = strudy.best_params\n",
    "best_accuracy = study.best_value\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LzPWqekMm9o1"
   },
   "source": [
    "`pad_sequences` позволяет привести данные к одной длине путём добавления нулевый значений в начало или конец последовательности\n",
    "```python\n",
    "padded_sequences = pad_sequences(sequences, maxlen = 100)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1aRRgPdIMVb1",
    "outputId": "9c993201-0da3-43b0-8c54-4823a128ca87"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "positive    203\n",
       "negative    158\n",
       "neutral      88\n",
       "opponent     75\n",
       "Name: sentiment, dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comment_data['sentiment'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XEIfOYulo_FX"
   },
   "outputs": [],
   "source": [
    "from gensim.models import Word2Vec\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.utils import shuffle\n",
    "\n",
    "from tensorflow import random\n",
    "from tensorflow.keras.callbacks import ModelCheckpoint\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Embedding, Conv1D, GlobalMaxPooling1D, Dense, Dropout\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "def lemmatize(text):\n",
    "    words = []\n",
    "    doc = nlp(text)\n",
    "    for token in doc:\n",
    "        if (token.is_punct != True) and (token.is_space != True) and (token.is_digit != True):\n",
    "            words.append(token.lemma_)\n",
    "    return ' '.join(words)\n",
    "\n",
    "comment_data = pd.read_excel('values_df.xlsx')\n",
    "comment_data['text'] = comment_data['value'].apply(lemmatize)\n",
    "\n",
    "w2v_model = Word2Vec.load('word2vec-result-50')\n",
    "\n",
    "# Подготовка данных\n",
    "encoder = LabelEncoder()\n",
    "comment_data['class'] = encoder.fit_transform(comment_data['sentiment'])\n",
    "\n",
    "comment_data = shuffle(comment_data)\n",
    "\n",
    "X = comment_data['text']\n",
    "y = comment_data['class']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42, stratify = comment_data['class'])\n",
    "\n",
    "test_data = pd.DataFrame({'X_test': X_test, 'y_test': encoder.inverse_transform(y_test)})\n",
    "\n",
    "tokenizer = Tokenizer()\n",
    "tokenizer.fit_on_texts(X_train)\n",
    "\n",
    "X_train = tokenizer.texts_to_sequences(X_train)\n",
    "X_test = tokenizer.texts_to_sequences(X_test)\n",
    "\n",
    "vocab_size = len(tokenizer.word_index) + 1\n",
    "\n",
    "embedding_matrix = np.zeros((vocab_size, 100))\n",
    "for word, i in tokenizer.word_index.items():\n",
    "    if word in w2v_model.wv:\n",
    "        embedding_matrix[i] = w2v_model.wv[word]\n",
    "\n",
    "max_length = 100\n",
    "X_train = pad_sequences(X_train, maxlen=max_length, padding='post')\n",
    "X_test = pad_sequences(X_test, maxlen=max_length, padding='post')\n",
    "\n",
    "# Инициализация весов word2vec\n",
    "embedding_matrix = w2v_model.wv.vectors\n",
    "\n",
    "num_words = len(w2v_model.wv.key_to_index) #embedding_matrix.shape[0]\n",
    "output_dim = w2v_model.vector_size #embedding_matrix.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2mDglOuAQQE2"
   },
   "outputs": [],
   "source": [
    "# Random Seed\n",
    "random.set_seed(49)\n",
    "\n",
    "# Определение архитектуры модели\n",
    "model = Sequential()\n",
    "model.add(Embedding(input_dim = num_words, output_dim = 100, weights = [embedding_matrix], input_length = 100, trainable = False))\n",
    "model.add(Conv1D(filters = 98, kernel_size = 10, activation = 'sigmoid')),\n",
    "model.add(Dropout(rate = 0.35)),\n",
    "model.add(GlobalMaxPooling1D())\n",
    "model.add(Dense(4, activation = 'softmax'))\n",
    "\n",
    "# Компиляция модели\n",
    "model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
    "\n",
    "checkpoint_path = 'model_checkpoint_opponent.h5'\n",
    "checkpoint = ModelCheckpoint(checkpoint_path, monitor = 'val_accuracy', save_best_only = True, mode = 'max', verbose = 1)\n",
    "\n",
    "# Преобразование меток классов в категориальное представление\n",
    "y_train_categorical = to_categorical(y_train)\n",
    "y_test_categorical = to_categorical(y_test)\n",
    "\n",
    "# Обучение модели\n",
    "model.fit(X_train, y_train_categorical, epochs = 30, batch_size = 16, validation_data = (X_test, y_test_categorical), callbacks = [checkpoint])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "LYaWrYSg0gq5",
    "outputId": "f2381706-555e-4d85-b3db-bff310e72a17"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "repr_error": "cannot convert float NaN to integer",
       "type": "dataframe"
      },
      "text/html": [
       "\n",
       "  <div id=\"df-cd8770c5-5b41-412e-9c06-8c898a621ac7\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dropout_rate</th>\n",
       "      <th>random_seed</th>\n",
       "      <th>num_filters</th>\n",
       "      <th>kernel_size</th>\n",
       "      <th>activation</th>\n",
       "      <th>num_epochs</th>\n",
       "      <th>bath_size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.395476</td>\n",
       "      <td>74</td>\n",
       "      <td>53</td>\n",
       "      <td>5</td>\n",
       "      <td>tanh</td>\n",
       "      <td>30</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cd8770c5-5b41-412e-9c06-8c898a621ac7')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-cd8770c5-5b41-412e-9c06-8c898a621ac7 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-cd8770c5-5b41-412e-9c06-8c898a621ac7');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "   dropout_rate  random_seed  num_filters  kernel_size activation  num_epochs  \\\n",
       "0      0.395476           74           53            5       tanh          30   \n",
       "\n",
       "   bath_size  \n",
       "0         16  "
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(best_params, index = [0]) #два свёрточных слоя 0.7809523940086365"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "uWk5BYkDOwL5",
    "outputId": "dedb755d-7c16-4332-9d96-3c98aed97c99"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "repr_error": "cannot convert float NaN to integer",
       "type": "dataframe"
      },
      "text/html": [
       "\n",
       "  <div id=\"df-f6c0225d-8ca3-4958-9434-01048a7cc0d9\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dropout_rate</th>\n",
       "      <th>random_seed</th>\n",
       "      <th>num_filters</th>\n",
       "      <th>kernel_size</th>\n",
       "      <th>activation</th>\n",
       "      <th>num_epochs</th>\n",
       "      <th>bath_size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.353593</td>\n",
       "      <td>49</td>\n",
       "      <td>98</td>\n",
       "      <td>10</td>\n",
       "      <td>sigmoid</td>\n",
       "      <td>20</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f6c0225d-8ca3-4958-9434-01048a7cc0d9')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-f6c0225d-8ca3-4958-9434-01048a7cc0d9 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-f6c0225d-8ca3-4958-9434-01048a7cc0d9');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "   dropout_rate  random_seed  num_filters  kernel_size activation  num_epochs  \\\n",
       "0      0.353593           49           98           10    sigmoid          20   \n",
       "\n",
       "   bath_size  \n",
       "0         16  "
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(best_params, index = [0]) #один свёрточный слой 0.7904762029647827"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X-iLoZBv94V3",
    "outputId": "bbf5bbd3-ef7d-4a10-8837-3e462b1dfdb2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4/4 [==============================] - 0s 23ms/step - loss: 0.7319 - accuracy: 0.7810\n",
      "Loss: 0.7319156527519226\n",
      "Accuracy: 0.7809523940086365\n"
     ]
    }
   ],
   "source": [
    "model.load_weights(checkpoint_path)\n",
    "\n",
    "loss, accuracy = model.evaluate(X_test, y_test_categorical)\n",
    "print('Loss:', loss)\n",
    "print('Accuracy:', accuracy) #  0.7809523940086365"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9uVaCU8tP8Sp"
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "def objective(trial):\n",
    "    # Define the hyperparameters to optimize\n",
    "    # dense_units = trial.suggest_int('dense_units', 32, 512)\n",
    "    dropout_rate = trial.suggest_float('dropout_rate', 0.1, 0.5)\n",
    "    random_seed = trial.suggest_int('random_seed', 1, 100)\n",
    "    num_filters = trial.suggest_int('num_filters', 32, 128)\n",
    "    kernel_size = trial.suggest_int('kernel_size', 3, 10)\n",
    "    activation = trial.suggest_categorical('activation', ['relu', 'tanh', 'sigmoid'])\n",
    "    num_epochs = trial.suggest_categorical('num_epochs', [10, 20, 30])\n",
    "    batch_size = trial.suggest_categorical('bath_size', [16, 32, 64])\n",
    "\n",
    "    model = Sequential([\n",
    "        Embedding(input_dim = num_words, output_dim = 100, weights = [embedding_matrix], input_length = 100, trainable = False),\n",
    "        Conv1D(filters = num_filters, kernel_size = kernel_size, activation = activation),\n",
    "        # Conv1D(filters = num_filters, kernel_size = kernel_size, activation = activation),\n",
    "        # Conv1D(filters = num_filters, kernel_size = kernel_size, activation = activation),\n",
    "        Dropout(rate = dropout_rate),\n",
    "        GlobalMaxPooling1D(),\n",
    "        # Dense(units = dense_units, activation = 'softmax'),\n",
    "        Dense(4, activation = 'softmax')\n",
    "    ])\n",
    "\n",
    "    # Compile the model\n",
    "    random.set_seed(random_seed)\n",
    "    model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
    "\n",
    "    # Train the model\n",
    "    model.fit(X_train, y_train_categorical, epochs = num_epochs, batch_size = batch_size, validation_data = (X_test, y_test_categorical), verbose = 0)\n",
    "\n",
    "    # Evaluate the model\n",
    "    loss, accuracy = model.evaluate(X_test, y_test_categorical, verbose = 1)\n",
    "    return accuracy\n",
    "\n",
    "study = optuna.create_study(direction = 'maximize')\n",
    "study.optimize(objective, n_trials = 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mX3Bp-9sMlDQ",
    "outputId": "d58e7c7c-fa27-413e-83bc-436a7115bc74"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7904762029647827\n",
      "{'dropout_rate': 0.35359335912346673, 'random_seed': 49, 'num_filters': 98, 'kernel_size': 10, 'activation': 'sigmoid', 'num_epochs': 20, 'bath_size': 16}\n"
     ]
    }
   ],
   "source": [
    "best_params = study.best_params\n",
    "best_accuracy = study.best_value\n",
    "\n",
    "print(best_accuracy)\n",
    "print(best_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5ujfHSNemgxz"
   },
   "source": [
    "Если вы используете функцию `sparse_categorical_crossentropy` в TensorFlow для компиляции модели, то нет необходимости использовать `to_categorical` для преобразования меток классов в one-hot кодировку. Функция `sparse_categorical_crossentropy` автоматически принимает целочисленные метки классов и выполняет необходимую конвертацию внутри себя. </br>\n",
    "Параметр `verbose` в функции `fit` библиотеки Keras определяет, какая информация будет выводиться в процессе обучения модели:\n",
    "* `verbose=0`: В процессе обучения модели никакая информация не будет выводить на экран\n",
    "* `verbose=1`: Выводится полоса прогресса progress bar в процессе обучения модели\n",
    "* `verbose=2`: Выводится одна строка для каждой эпохи обучения, содержащая информациб о текущей эпохи и значениях выбранных метрик\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 224
    },
    "id": "H923udmkUDxL",
    "outputId": "2a588d44-41d9-484f-c6e5-916a17a8cbd6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3/3 [==============================] - 0s 10ms/step\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-5f4ad18b-242e-4d38-8add-3938333a9058\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X_test</th>\n",
       "      <th>y_test</th>\n",
       "      <th>predicted_class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>депозиты размещают</td>\n",
       "      <td>positive</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>280</th>\n",
       "      <td>по депозитам без изменений</td>\n",
       "      <td>neutral</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>340</th>\n",
       "      <td>счет открыт + вкл выдана</td>\n",
       "      <td>positive</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>266</th>\n",
       "      <td>продукты фт )</td>\n",
       "      <td>neutral</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>138</th>\n",
       "      <td>кредит 100 - 300 млн руб. + рамку на бг</td>\n",
       "      <td>neutral</td>\n",
       "      <td>neutral</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5f4ad18b-242e-4d38-8add-3938333a9058')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-5f4ad18b-242e-4d38-8add-3938333a9058 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-5f4ad18b-242e-4d38-8add-3938333a9058');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-e5197163-7f75-4e4c-9068-efa501f4475a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e5197163-7f75-4e4c-9068-efa501f4475a')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-e5197163-7f75-4e4c-9068-efa501f4475a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                       X_test    y_test predicted_class\n",
       "79                        депозиты размещают   positive         neutral\n",
       "280               по депозитам без изменений    neutral        positive\n",
       "340                 счет открыт + вкл выдана   positive        positive\n",
       "266                            продукты фт )    neutral         neutral\n",
       "138  кредит 100 - 300 млн руб. + рамку на бг    neutral         neutral"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions = model.predict(X_test)\n",
    "\n",
    "# Преобразуем предсказания в индексы классов, выбрав класс с наибольшей вероятностью\n",
    "predicted_classes = np.argmax(predictions, axis = 1)\n",
    "\n",
    "test_data['predicted_class'] = encoder.inverse_transform(predicted_classes)\n",
    "test_data.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "mtXz6iLvcMtW"
   },
   "outputs": [],
   "source": [
    "def form_sentiment_data(comment_data):\n",
    "    sentiment_dict = {}\n",
    "    # На вход подаём список из словарей\n",
    "    for elem in comment_data:\n",
    "        # Проходим по каждому из словарей списка и формируем sentiment_dict\n",
    "        for key, value in elem.items():\n",
    "            if key not in sentiment_dict:\n",
    "                sentiment_dict[key] = 'neutral'\n",
    "            # Переводим комментарий, значение словаря в уподобимый формат для модели\n",
    "            tokenized_sentence = tokenizer.texts_to_sequences([lemmatize(value)])\n",
    "            # Дополнений или обрезка последовательности\n",
    "            padded_sentence = pad_sequences(tokenized_sentence, maxlen=100, padding='post')\n",
    "            # Получаем предсказанный класс\n",
    "            predicted_class = model.predict(padded_sentence)\n",
    "            # Выбираем класс с наибольшей вероятностью\n",
    "            predicted_class = np.argmax(predicted_class, axis = 1)\n",
    "            # Переводим в текстовый формат\n",
    "            predicted_class = encoder.inverse_transform(predicted_class)\n",
    "            # Добавляем это значение по ключу Продукта\n",
    "            sentiment_dict[key] = predicted_class\n",
    "    return sentiment_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "b0EAwS4eggOg",
    "outputId": "049875de-4cf8-4095-e6ac-9ac87bedfb6c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'потребность': 'потребностей пока нет ! '}]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "create_partlist('кт. показатели снизились, за 2023 тоже, потребностей пока нет!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_a8SLXsk5Rdz"
   },
   "outputs": [],
   "source": [
    "def entity_extract(partlist):\n",
    "    entlist = []\n",
    "    for part in partlist:\n",
    "        doc = nlp(part)\n",
    "        for ent in doc.ents:\n",
    "            if ent.label_ == 'PROD':\n",
    "                entlist.append({ent.lemma_: part}) #ПРИВОДИМ К ФОРМАТУ СУЩНОСТЬ-ЗНАЧЕНИЕ\n",
    "    return entlist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "FmHTWSZfx_yf",
    "outputId": "d2ff106f-26e4-43f9-a5a3-c521ad298a4f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-01d9352a-ec43-4215-8456-fe7e66e4e02f\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>кредит</th>\n",
       "      <th>лизинг</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>positive</td>\n",
       "      <td>negative</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-01d9352a-ec43-4215-8456-fe7e66e4e02f')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-01d9352a-ec43-4215-8456-fe7e66e4e02f button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-01d9352a-ec43-4215-8456-fe7e66e4e02f');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "     кредит    лизинг\n",
       "0  positive  negative"
      ]
     },
     "execution_count": 271,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data_to_train['sentiment'].iloc[3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "djQrVYcKj0W9",
    "outputId": "259da13b-6c9c-4185-edf4-263a16d5c6b7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-dc939042-294e-4e5d-bd7a-fb6059aeeb10\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>кредитония</th>\n",
       "      <th>депозит</th>\n",
       "      <th>бг</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>positive</td>\n",
       "      <td>positive</td>\n",
       "      <td>negative</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-dc939042-294e-4e5d-bd7a-fb6059aeeb10')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-dc939042-294e-4e5d-bd7a-fb6059aeeb10 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-dc939042-294e-4e5d-bd7a-fb6059aeeb10');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "  кредитония   депозит        бг\n",
       "0   positive  positive  negative"
      ]
     },
     "execution_count": 274,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data_to_train['sentiment'].iloc[9])"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}