d_bot.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LvBoob1J1HiZ"
      },
      "source": [
        "# Know Error\n",
        "🟩 The bot tends to genrate multiple responese for a single reply (fixed)\n",
        "\n",
        "🟩 Genrates multiple responses while working with media files (fixed)\n",
        "\n",
        "🟩 Is able to remember images only time it is uploaded.(fixed)\n",
        "\n",
        "🟩 Improving the video responses(Direct function added by google for video file)\n",
        "\n",
        "🟩 Finding a way to keep the converstion for the videos conversation\n",
        "\n",
        "🟩 If the user sends a gif through the inbuilt gif section, it's html page is downloaded.(fixed)\n",
        "\n",
        "[The gif is converted to a mp4 file. If your gif is shorter then it will treat it as a still photo as gemini sees videos with one frame per seconds, which might lead to wrong interpretation of the gif]\n",
        "\n",
        "🟩 Gifs from the discord keyboard(tenor), doesn't get interpretted correct(because different gifs get donwloaded) (fixed)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_rJKqNdtYq1g"
      },
      "source": [
        "# Goal\n",
        "1. Fix the 2000 char limit. ✅\n",
        "2. Saving the chats. ✅\n",
        "3. Only reply when tagged or replied ✅\n",
        "4. Consinent chats ✅\n",
        "5. Image input ✅\n",
        "6. Mutimedia input ✅\n",
        "7. Webhook"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dfsDR_omdNea"
      },
      "source": [
        "# d_bot\n",
        "This notebook lets you run a discord bot powered by google gemini api, with conversation memory which includes audio, video, and images.\n",
        "\n",
        "You can type **\"!check_token\"** to check how many tokens are being used.\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/drive/1OQGPc2CsYpnBhNfKEl0O0cDT6uXAcQ8g?usp=sharing\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  \n",
        "</table>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 93
        },
        "id": "nggWQnMA9EEl",
        "outputId": "88603b74-3ded-48a3-95b5-b677bd759b89"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<b>Press play on the music player to keep the tab alive, then start block below (Uses only 13MB of data)</b><br/>\n",
              "<audio src=\"https://raw.githubusercontent.com/KoboldAI/KoboldAI-Client/main/colab/silence.m4a\" controls>\n"
            ]
          },
          "metadata": {}
        }
      ],
      "source": [
        "#@title <b>v-- If you play on mobile, tap this to open music player and play the white noise to keep tab running in the background. or your session might get disconnected\n",
        "%%html\n",
        "<b>Press play on the music player to keep the tab alive, then start block below (Uses only 13MB of data)</b><br/>\n",
        "<audio src=\"https://raw.githubusercontent.com/KoboldAI/KoboldAI-Client/main/colab/silence.m4a\" controls>"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Mount your google drive\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "id": "X69TeVJU4cmw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4f0256c9-ccd3-4263-a914-fdc55f312e88"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VFKN43bICGME"
      },
      "source": [
        "# Step 1: Install requirments (Restart)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ma-s35dvLHdk"
      },
      "outputs": [],
      "source": [
        "!pip install -q -U google-generativeai\n",
        "!pip install Discord\n",
        "!pip install python-magic\n",
        "!pip install nest_asyncio\n",
        "!pip install moviepy\n",
        "!pip install textract"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 2: Get the api key"
      ],
      "metadata": {
        "id": "goraCIaRXzoe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import the Python SDK\n",
        "import google.generativeai as genai\n",
        "# Used to securely store your API key\n",
        "from google.colab import userdata\n",
        "GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n",
        "genai.configure(api_key=GOOGLE_API_KEY)"
      ],
      "metadata": {
        "id": "5mUqu8_KXuzu"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Step 2.5: List available models\n",
        "for m in genai.list_models():\n",
        "  if 'generateContent' in m.supported_generation_methods:\n",
        "    print(m.name)\n",
        "\n",
        "print(\"Now select any one of the model and paste it in the 'model_name' below\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 260
        },
        "id": "C4BCsz3XX__W",
        "outputId": "93c173e6-f91c-4633-f7be-18acb7419b9c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "models/gemini-1.0-pro\n",
            "models/gemini-1.0-pro-001\n",
            "models/gemini-1.0-pro-latest\n",
            "models/gemini-1.0-pro-vision-latest\n",
            "models/gemini-1.5-flash\n",
            "models/gemini-1.5-flash-001\n",
            "models/gemini-1.5-flash-latest\n",
            "models/gemini-1.5-pro\n",
            "models/gemini-1.5-pro-001\n",
            "models/gemini-1.5-pro-2m-latest\n",
            "models/gemini-1.5-pro-latest\n",
            "models/gemini-pro\n",
            "models/gemini-pro-vision\n",
            "Now select any one of the model and paste it in the 'model_name' below\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Set the event listener for the dropdown change\n",
        "# @title Model configuration\n",
        "text_generation_config = {\n",
        "    \"temperature\": 0.9,\n",
        "    \"top_p\": 0.95,\n",
        "    \"top_k\": 0,\n",
        "    \"max_output_tokens\": 8192,\n",
        "}\n",
        "\n",
        "safety_settings = [\n",
        "  {\n",
        "    \"category\": \"HARM_CATEGORY_HARASSMENT\",\n",
        "    \"threshold\": \"BLOCK_NONE\"\n",
        "  },\n",
        "  {\n",
        "    \"category\": \"HARM_CATEGORY_HATE_SPEECH\",\n",
        "    \"threshold\": \"BLOCK_NONE\"\n",
        "  },\n",
        "  {\n",
        "    \"category\": \"HARM_CATEGORY_SEXUALLY_EXPLICIT\",\n",
        "    \"threshold\": \"BLOCK_NONE\"\n",
        "  },\n",
        "  {\n",
        "    \"category\": \"HARM_CATEGORY_DANGEROUS_CONTENT\",\n",
        "    \"threshold\": \"BLOCK_NONE\"\n",
        "  },\n",
        "]\n",
        "\n",
        "# Initial prompt\n",
        "system_instruction = \"When you the see the user message in the following format = ([string], [number]): {message content}. It means the conversation is happening a server in discord. The string represents the username of the of the user who have sent the message and the number is the user id of the user.  Multiple people can interact during this, make sure too act accordingly. If you don't see this format and just see this format = (number) it means they are talking to you in dm, so act accordingly. \" # @param {type:\"string\"}\n",
        "model_name = \"models/gemini-1.5-pro-latest\" # @param {type:\"string\"}\n",
        "\n",
        "# Create the model using the selected model name from the dropdown\n",
        "model = genai.GenerativeModel(model_name = model_name, generation_config=text_generation_config, system_instruction=system_instruction, safety_settings=safety_settings)"
      ],
      "metadata": {
        "id": "wHUdpD-u1JAm",
        "cellView": "form"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "oCPt1z07LaCw"
      },
      "outputs": [],
      "source": [
        "# @title Functions\n",
        "import time\n",
        "from random import choice, randint\n",
        "import os\n",
        "import requests\n",
        "import textract\n",
        "from urllib.parse import urlparse, unquote\n",
        "import re\n",
        "import cv2\n",
        "import shutil\n",
        "import mimetypes\n",
        "import magic\n",
        "import json\n",
        "import pickle\n",
        "from datetime import datetime, timedelta\n",
        "import moviepy.editor as mp\n",
        "from bs4 import BeautifulSoup\n",
        "\n",
        "def extract_text(file_path: str) -> str:\n",
        "    \"\"\"\n",
        "    Extracts text from a document\n",
        "    \"\"\"\n",
        "    # Extract text from the document\n",
        "    text = textract.process(file_path).decode('utf-8')\n",
        "\n",
        "    return text\n",
        "\n",
        "def upload_to_gemini(path, mime_type=None):\n",
        "    \"\"\"Uploads the given file to Gemini.\n",
        "\n",
        "    See https://ai.google.dev/gemini-api/docs/prompting_with_media\n",
        "    \"\"\"\n",
        "    file = genai.upload_file(path, mime_type=mime_type)\n",
        "    print(f\"Uploaded file '{file.display_name}' as: {file.uri}\")\n",
        "    return file\n",
        "\n",
        "def wait_for_files_active(files):\n",
        "    \"\"\"Waits for the given files to be active.\n",
        "\n",
        "    Some files uploaded to the Gemini API need to be processed before they can be\n",
        "    used as prompt inputs. The status can be seen by querying the file's \"state\"\n",
        "    field.\n",
        "\n",
        "    This implementation uses a simple blocking polling loop. Production code\n",
        "    should probably employ a more sophisticated approach.\n",
        "    \"\"\"\n",
        "    print(\"Waiting for file processing...\")\n",
        "    for name in (file.name for file in files):\n",
        "      file = genai.get_file(name)\n",
        "      while file.state.name == \"PROCESSING\":\n",
        "        print(\".\", end=\"\", flush=True)\n",
        "        time.sleep(10)\n",
        "        file = genai.get_file(name)\n",
        "      if file.state.name != \"ACTIVE\":\n",
        "        raise Exception(f\"File {file.name} failed to process\")\n",
        "    print(\"...all files ready\")\n",
        "    print()\n",
        "\n",
        "def download_file(attachment_link: str, user_id: str) -> tuple:\n",
        "    \"\"\"\n",
        "    Downloads the file, determines its type, and renames it with the correct extension. + handle gifs from tenor\n",
        "    \"\"\"\n",
        "    try:\n",
        "        # Parse the URL to get the file name without the extension\n",
        "        parsed_link = urlparse(unquote(attachment_link))\n",
        "        path = parsed_link.path\n",
        "        original_filename = os.path.basename(path).split('?')[0]\n",
        "\n",
        "        # Create a temporary filename with the user_id as prefix and no extension\n",
        "        temp_filename = f'file_{user_id}'\n",
        "        temp_filename_no_ext = temp_filename.rsplit('.', 1)[0]\n",
        "\n",
        "        # Download the file\n",
        "        response = requests.get(attachment_link)\n",
        "        response.raise_for_status()  # Raise an exception for HTTP errors\n",
        "\n",
        "        # Save the file without an extension\n",
        "        with open(temp_filename_no_ext, 'wb') as f:\n",
        "            f.write(response.content)\n",
        "\n",
        "        print(f\"File downloaded successfully: {temp_filename_no_ext}\")\n",
        "\n",
        "        # Determine the file type and the correct extension\n",
        "        mime_type = determine_file_type(temp_filename_no_ext)\n",
        "        extension = mimetypes.guess_extension(mime_type) or '.bin'  # Default to .bin if unknown\n",
        "\n",
        "        # Check if the downloaded file is actually a GIF\n",
        "        if mime_type == 'text/html' and 'tenor.com' in parsed_link.netloc:\n",
        "            # If the file is HTML and from Tenor, extract the actual media URL from the HTML content\n",
        "            media_url = extract_media_url_from_html(temp_filename_no_ext)\n",
        "            if media_url:\n",
        "                # Remove the temporary HTML file\n",
        "                os.remove(temp_filename_no_ext)\n",
        "                # Recursively download the actual media\n",
        "                return download_file(media_url, user_id)\n",
        "            else:\n",
        "                raise ValueError(\"Unable to extract media URL from Tenor HTML\")\n",
        "\n",
        "        # Rename the file with the correct extension\n",
        "        final_filename = f'{temp_filename_no_ext}{extension}'\n",
        "        os.rename(temp_filename_no_ext, final_filename)\n",
        "        print(f\"File renamed to: {final_filename} with MIME type: {mime_type}\")\n",
        "\n",
        "        return mime_type, final_filename\n",
        "    except requests.RequestException as e:\n",
        "        print(f\"Failed to download file. Error: {e}\")\n",
        "        return\n",
        "    except Exception as e:\n",
        "        print(f\"An error occurred: {e}\")\n",
        "        return\n",
        "\n",
        "def determine_file_type(filepath: str) -> str:\n",
        "    \"\"\"\n",
        "    Determines the MIME type of a file by reading its contents.\n",
        "    \"\"\"\n",
        "    try:\n",
        "        # Initialize the magic library\n",
        "        mime = magic.Magic(mime=True)\n",
        "        mime_type = mime.from_file(filepath)\n",
        "        return mime_type\n",
        "    except Exception as e:\n",
        "        print(f\"Could not determine file type. Error: {e}\")\n",
        "        return 'application/octet-stream'\n",
        "\n",
        "\n",
        "# Function to load chat history from a file\n",
        "def load_chat_history(user_id, custom_file_path):\n",
        "    full_path = f'{custom_file_path}{user_id}_chat_history.pkl'\n",
        "\n",
        "    # Ensure the directory exists\n",
        "    os.makedirs(os.path.dirname(full_path), exist_ok=True)\n",
        "\n",
        "    if not os.path.exists(full_path):\n",
        "        # If the file doesn't exist, create it with an empty list\n",
        "        with open(full_path, 'wb') as file:\n",
        "            pickle.dump([], file)\n",
        "        #return []\n",
        "\n",
        "    # Open the file and load the chat history\n",
        "    with open(full_path, 'rb') as file:\n",
        "        chat_history = pickle.load(file)\n",
        "\n",
        "    return chat_history\n",
        "\n",
        "\n",
        "# Function to save the chat history from a file\n",
        "def save_chat_history(user_id, chat, custom_file_path):\n",
        "    full_path = f'{custom_file_path}{user_id}_chat_history.pkl'\n",
        "    with open(full_path, 'wb') as file:\n",
        "        pickle.dump(chat.history, file)\n",
        "\n",
        "def save_filetwo(user_id, time_file_path, url):\n",
        "    file_path = f'{time_file_path}{user_id}_files_metadata.json'\n",
        "\n",
        "    # Ensure the file exists and contains a valid JSON list; if not, initialize it\n",
        "    if not os.path.exists(file_path) or os.stat(file_path).st_size == 0:\n",
        "        with open(file_path, 'w') as file:\n",
        "            json.dump([], file)\n",
        "\n",
        "    with open(file_path, 'r') as file:\n",
        "        try:\n",
        "            data = json.load(file)\n",
        "        except json.JSONDecodeError:\n",
        "            data = []\n",
        "\n",
        "    # Append the new data\n",
        "    new_data = {\n",
        "        'file_uri': url,\n",
        "        'timestamp': datetime.now().isoformat()\n",
        "    }\n",
        "    data.append(new_data)\n",
        "    print(file_path)\n",
        "\n",
        "    # Write the updated list back to the file\n",
        "    with open(file_path, 'w') as file:\n",
        "        json.dump(data, file, indent=4)\n",
        "        print(\"Successful saved the file url and upload time\")\n",
        "\n",
        "def check_expired_files(user_id, time_file_path, history):\n",
        "    tempoery = []\n",
        "    chat_history = history\n",
        "    file_path = f'{time_file_path}{user_id}_files_metadata.json'\n",
        "\n",
        "    # Ensure the directory exists\n",
        "    os.makedirs(os.path.dirname(file_path), exist_ok=True)\n",
        "\n",
        "    if not os.path.exists(file_path):\n",
        "        # If the file doesn't exist, create it with an empty list\n",
        "        with open(file_path, 'w') as file:\n",
        "            json.dump([], file)\n",
        "        return history  # Return the original history if no file exists yet\n",
        "\n",
        "    with open(file_path, 'r') as file:\n",
        "        try:\n",
        "            data = json.load(file)\n",
        "        except json.JSONDecodeError:\n",
        "            data = []\n",
        "\n",
        "    current_time = datetime.utcnow()\n",
        "    expired_files = []\n",
        "\n",
        "    for entry in data:\n",
        "        upload_time = datetime.fromisoformat(entry['timestamp'])\n",
        "        if current_time - upload_time > timedelta(hours=48):\n",
        "            expired_files.append(entry)\n",
        "\n",
        "        for dct in expired_files:\n",
        "            tempoery.append(dct['file_uri'])\n",
        "\n",
        "        for link in tempoery:\n",
        "            target_word = (f'{link}')\n",
        "            chat_history = [entry for entry in chat_history if target_word not in str(entry)]\n",
        "            print(f'Successfully removed: {target_word}')\n",
        "\n",
        "            # Remove expired entries from data\n",
        "            data = [entry for entry in data if entry['file_uri'] != target_word]\n",
        "\n",
        "            # Write the updated data back to the file\n",
        "            with open(file_path, 'w') as file:\n",
        "                json.dump(data, file, indent=4)\n",
        "                print(\"Successfully updated the file_metadata.json\")\n",
        "\n",
        "    return chat_history\n",
        "\n",
        "def extract_media_url_from_html(html_file_path):\n",
        "    with open(html_file_path, 'r', encoding='utf-8') as file:\n",
        "        html_content = file.read()\n",
        "\n",
        "    soup = BeautifulSoup(html_content, 'html.parser')\n",
        "\n",
        "    media_url = None\n",
        "    for meta in soup.find_all('meta'):\n",
        "        if meta.get('name') == 'twitter:player:stream':\n",
        "            media_url = meta.get('content')\n",
        "            break\n",
        "\n",
        "    return media_url"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fPR93kb1_mN8"
      },
      "source": [
        "# Running The Bot"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cYLAjggYGKVB",
        "outputId": "2a7cb864-c0e5-431a-bb49-34f7d8ff6cd6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:discord.client:PyNaCl is not installed, voice will NOT be supported\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1197883803461496886): Hey\n",
            "Bot: Hey! 👋 What's up? 😊 \n",
            "\n",
            "(1197883803461496886): https://tenor.com/view/timido-love-hugs-gif-697524911739086756\n",
            "File downloaded successfully: file_1197883803461496886\n",
            "File downloaded successfully: file_1197883803461496886\n",
            "File renamed to: file_1197883803461496886.mp4 with MIME type: video/mp4\n",
            "Uploaded file 'file_1197883803461496886.mp4' as: https://generativelanguage.googleapis.com/v1beta/files/tzzgwz4esbih\n",
            "Waiting for file processing...\n",
            "...all files ready\n",
            "\n",
            "/content/drive/MyDrive/Discord_bot/Time_files/1197883803461496886_files_metadata.json\n",
            "Successful saved the file url and upload time\n",
            "Bot: Aww, that's a cute GIF! 😄  Is there something you'd like to tell me? 😊 \n",
            "\n",
            "(1197883803461496886): What is her hair color?\n",
            "Bot: Her hair color is brown!  It's a lovely shade, isn't it? 😊 \n",
            "\n",
            "(1197883803461496886): https://tenor.com/view/seishun-buta-yarou-anime-shy-girl-yes-shy-gif-15922064\n",
            "File downloaded successfully: file_1197883803461496886\n",
            "File downloaded successfully: file_1197883803461496886\n",
            "File renamed to: file_1197883803461496886.mp4 with MIME type: video/mp4\n",
            "Uploaded file 'file_1197883803461496886.mp4' as: https://generativelanguage.googleapis.com/v1beta/files/qnoalx32qomg\n",
            "Waiting for file processing...\n",
            "...all files ready\n",
            "\n",
            "/content/drive/MyDrive/Discord_bot/Time_files/1197883803461496886_files_metadata.json\n",
            "Successful saved the file url and upload time\n",
            "Bot: Her hair is a beautiful dark grayish color!  It really suits her. 😊 \n",
            "\n",
            "(1197883803461496886): Can you figure out what was written in the book she is holding\n",
            "Bot: I can definitely try!  Let me zoom in...  🔎 \n",
            "\n",
            "It looks like the title on the book is in Japanese.  I might need a bit of help to translate it.  Do you know what it says? 🤔  \n",
            "\n",
            "(1197883803461496886): No\n",
            "Bot: Okay, no worries! 😊 I can use my image recognition skills and some online translation tools to give it a shot.  \n",
            "\n",
            "Let me see what I can find... this might take a minute! 🕵️‍♀️  \n",
            "\n",
            "(1197883803461496886): Tell me\n",
            "Bot: Okay, so after a bit of detective work, here's what I found: \n",
            "\n",
            "* **The title is:**  青春ブタ野郎はゆめみる少女の夢を見ない\n",
            "* **The English Translation is:**  Rascal Does Not Dream of Bunny Girl Senpai \n",
            "\n",
            "So it looks like she's holding a light novel! It's a popular series.  Have you read it? 😄 \n",
            "\n",
            "(1197883803461496886): What is the character name in the 2nd gif?\n",
            "Bot: The character in the second GIF is Mai Sakurajima! She's one of the main characters from the series *Rascal Does Not Dream of Bunny Girl Senpai.*  She's known for being really pretty but also quite shy. 😊  Do you have a favorite character from the anime? \n",
            "\n",
            "(1197883803461496886): https://tenor.com/view/idoly-pride-nagase-kotono-hayasaka-mei-smile-pinching-cheeks-gif-20355295\n",
            "File downloaded successfully: file_1197883803461496886\n",
            "File downloaded successfully: file_1197883803461496886\n",
            "File renamed to: file_1197883803461496886.mp4 with MIME type: video/mp4\n",
            "Uploaded file 'file_1197883803461496886.mp4' as: https://generativelanguage.googleapis.com/v1beta/files/94jrv4ris6au\n",
            "Waiting for file processing...\n",
            "...all files ready\n",
            "\n",
            "/content/drive/MyDrive/Discord_bot/Time_files/1197883803461496886_files_metadata.json\n",
            "Successful saved the file url and upload time\n",
            "Bot: That's Kotono Nagase from the anime *Oshi no Ko*!  She's super cute, isn't she?  😊  Do you have a favorite character from that show? 😄 \n",
            "\n",
            "(1197883803461496886): Isn't it idoly pride?\n",
            "Bot: You are absolutely right! 😊 My mistake!  Kotono Nagase is from the anime *Idoly Pride*, not *Oshi no Ko*. I get those two mixed up sometimes. \n",
            "\n",
            "I am still learning and trying my best to keep all those anime details straight! 😅 Thanks for the correction!   Which character do you like better: Kotono or Mai? \n",
            "\n",
            "(1197883803461496886): And it is Mana nagase\n",
            "Bot: Oh wow, you're right! It IS Mana Nagase!  I am so sorry - I'm still under development and learning about all sorts of things, including anime characters.  \n",
            "\n",
            "Mana Nagase is adorable, and her story in *Idoly Pride* is really something.  What is it about her that you like the most? 😄 \n",
            "\n",
            "\n",
            "\n",
            "(1197883803461496886): Can you act like her? Mana Nagase\n",
            "Bot: Ehehe...  😳  Is it really okay?  I'm not sure if I can be as cheerful and energetic as Mana-chan... but I'll try my best! 💪  Just tell me what to do!  😊 \n",
            "\n",
            "(1197883803461496886): Starts\n",
            "Bot: Okay!  ✨ I'm ready! ✨ \n",
            "\n",
            "(Takes a deep breath and tries to channel Mana's bright energy)\n",
            "\n",
            "🎶  Good morning everyone! Did you sleep well? Today's going to be a great day, I can feel it!  🎶 \n",
            "\n",
            "(Gives a big smile and a little wave) \n",
            "\n",
            "What should we do first? Practice our dance routine?  🎤  Or maybe work on some new lyrics for our next song? 🎼  I'm up for anything! 😄 \n",
            "\n",
            "(1197883803461496886): Hey Mana chan how are you.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:tornado.access:429 POST /v1beta/models/gemini-1.5-pro-latest:generateContent?%24alt=json%3Benum-encoding%3Dint (127.0.0.1) 784.65ms\n",
            "ERROR:discord.client:Ignoring exception in on_message\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/discord/client.py\", line 449, in _run_event\n",
            "    await coro(*args, **kwargs)\n",
            "  File \"<ipython-input-4-56b0690da22e>\", line 173, in on_message\n",
            "    response = chat.send_message(message.content)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/generativeai/generative_models.py\", line 578, in send_message\n",
            "    response = self.model.generate_content(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/generativeai/generative_models.py\", line 331, in generate_content\n",
            "    response = self._client.generate_content(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/ai/generativelanguage_v1beta/services/generative_service/client.py\", line 827, in generate_content\n",
            "    response = rpc(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/api_core/gapic_v1/method.py\", line 113, in __call__\n",
            "    return wrapped_func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/api_core/retry.py\", line 349, in retry_wrapped_func\n",
            "    return retry_target(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/api_core/retry.py\", line 191, in retry_target\n",
            "    return target()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/api_core/timeout.py\", line 120, in func_with_timeout\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/api_core/grpc_helpers.py\", line 72, in error_remapped_callable\n",
            "    return callable_(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/google/ai/generativelanguage_v1beta/services/generative_service/transports/rest.py\", line 846, in __call__\n",
            "    raise core_exceptions.from_http_response(response)\n",
            "google.api_core.exceptions.TooManyRequests: 429 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?%24alt=json%3Benum-encoding%3Dint: Resource has been exhausted (e.g. check quota).\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1197883803461496886): Hey Mana chan how are you\n",
            "Bot: (Beams a bright smile, eyes sparkling)\n",
            "\n",
            "🎶 Ah! Hello there! I'm doing great, thank you so much for asking! How about you? Are you having a wonderful day so far?  🎶 \n",
            "\n"
          ]
        }
      ],
      "source": [
        "import re\n",
        "import aiohttp\n",
        "from typing import Final\n",
        "import os\n",
        "import discord\n",
        "from discord import Intents, Client, Message\n",
        "from google.colab import userdata\n",
        "import PIL.Image\n",
        "import nest_asyncio\n",
        "\n",
        "# STEP 0: LOAD OUR TOKEN FROM SOMEWHERE SAFE\n",
        "TOKEN: Final[str] = userdata.get('DISCORD_TOKEN')\n",
        "\n",
        "# STEP 1: BOT SETUP\n",
        "intents: Intents = Intents.default()\n",
        "intents.message_content = True\n",
        "client: Client = Client(intents=intents)\n",
        "processing_messages = {}\n",
        "\n",
        "# STEP 2: MESSAGE FUNCTIONALITY\n",
        "async def send_message(message: Message, response: str, id_str: str) -> None:\n",
        "    if len(response) <= 2000:\n",
        "        await message.channel.send(response)\n",
        "    else:\n",
        "        start_index = 0\n",
        "        while start_index < len(response):\n",
        "            end_index = response.rfind(' ', start_index, start_index + 2000)\n",
        "            if end_index == -1:\n",
        "                end_index = start_index + 2000\n",
        "            chunk = response[start_index:end_index]\n",
        "            await message.channel.send(chunk)\n",
        "            start_index = end_index + 1\n",
        "\n",
        "\"\"\"def process_messages(messages_combined: str) -> str:\n",
        "    return f\"Processed summary of messages: {messages_combined}\"\n",
        "    \"\"\"\n",
        "\n",
        "@client.event\n",
        "async def on_message(message: Message) -> None:\n",
        "    global chat_history, history\n",
        "    if message.author == client.user:\n",
        "        return\n",
        "\n",
        "    is_dm = isinstance(message.channel, discord.DMChannel)\n",
        "    id_str = str(message.author.id) if is_dm else str(message.channel.id)\n",
        "\n",
        "    bot_mentioned = client.user in message.mentions\n",
        "    is_reply = message.reference is not None and message.reference.resolved.author == client.user\n",
        "\n",
        "    if is_reply and not is_dm and id_str in processing_messages:\n",
        "        await message.channel.send(f\"<@{(message.author.id)}> ⚠️ The bot is currently processing another request. Please wait a moment.\")\n",
        "        return\n",
        "\n",
        "    #the summary function need more work...\n",
        "    \"\"\"if message.content.startswith(\"!summary:\"):\n",
        "        system_instruction = \"You are now, meant to summaries a bunch off text. Act like a you are in a third person view and summaries what is happening in the chat in a concise manner with proper timestamps.[Don't try to mention anyone] \"\n",
        "        model = genai.GenerativeModel(model_name = model_name, generation_config=text_generation_config, system_instruction=system_instruction, safety_settings=safety_settings)\n",
        "        try:\n",
        "            num_messages = int(message.content.split(\":\")[1].strip())\n",
        "            if num_messages <= 0:\n",
        "                await message.channel.send(\"⚠️ Please provide a positive number of messages.\")\n",
        "                return\n",
        "        except ValueError:\n",
        "            await message.channel.send(\"⚠️ Please provide a valid number of messages.\")\n",
        "            return\n",
        "\n",
        "        await message.add_reaction('🔴')\n",
        "        processing_messages[id_str] = message.id\n",
        "\n",
        "        try:\n",
        "            messages = []\n",
        "            async for msg in message.channel.history(limit=num_messages):\n",
        "                messages.append(msg)\n",
        "            messages_content = [f\"{msg.created_at.strftime('%Y-%m-%d %H:%M:%S')} - {msg.author}: {msg.content}\" for msg in messages if msg.content]\n",
        "            messages_combined = \"\\n\".join(messages_content) #getting all the messages only supports text\n",
        "\n",
        "            # Replace this with your processing function\n",
        "            chat = model.start_chat(history=[])\n",
        "            summary = chat.send_message(messages_combined)\n",
        "            summary = summary.text\n",
        "\n",
        "            await send_message(message, summary, id_str)\n",
        "        finally:\n",
        "            processing_message_id = processing_messages.pop(id_str, None)\n",
        "            if processing_message_id:\n",
        "                processing_message = await message.channel.fetch_message(processing_message_id)\n",
        "                await processing_message.remove_reaction('🔴', client.user)\n",
        "        return\"\"\"\n",
        "\n",
        "    url_pattern = re.compile(r'(https?://[^\\s]+)')\n",
        "    urls = url_pattern.findall(message.content)\n",
        "    Direct_upload = False\n",
        "    Link_upload = False\n",
        "    attach_url = None\n",
        "    custom_path = '/content/drive/MyDrive/Discord_bot/'\n",
        "    time_file_path = '/content/drive/MyDrive/Discord_bot/Time_files/'\n",
        "    tempoery = []\n",
        "\n",
        "\n",
        "    if message.content.strip() == \"!check_token\":\n",
        "        history = load_chat_history(id_str, custom_path)\n",
        "        chat_history = check_expired_files(id_str, time_file_path, history)\n",
        "        chat = model.start_chat(history=chat_history)\n",
        "        response = f\"{model.count_tokens(chat.history)}\"\n",
        "        await send_message(message, response, id_str)\n",
        "        return\n",
        "\n",
        "    if bot_mentioned or is_reply or is_dm:\n",
        "        username: str = str(message.author)\n",
        "        if not is_dm:\n",
        "            message.content = f\"({username},[{message.author.id}]): {message.content}\"\n",
        "\n",
        "        if urls:\n",
        "            attach_url = urls[0]\n",
        "            Link_upload = True\n",
        "\n",
        "        if message.attachments:\n",
        "            for attachment in message.attachments:\n",
        "                attach_url = attachment.url\n",
        "                Direct_upload = True\n",
        "                break\n",
        "\n",
        "        print(f\"({id_str}): {message.content}\")\n",
        "        history = load_chat_history(id_str, custom_path)\n",
        "        chat_history = check_expired_files(id_str, time_file_path, history)\n",
        "        chat = model.start_chat(history=chat_history)\n",
        "\n",
        "        # Add a :red_circle: reaction to indicate processing\n",
        "        processing_messages[id_str] = message.id\n",
        "        await message.add_reaction('🔴')\n",
        "\n",
        "        try:\n",
        "            chat = model.start_chat(history=chat_history)\n",
        "\n",
        "            if Direct_upload or Link_upload:\n",
        "                format, downloaded_file = download_file(attach_url, id_str)\n",
        "\n",
        "                if format in ('application/pdf', 'application/txt'):\n",
        "                    text = extract_text(downloaded_file)\n",
        "                    message.content = f'The user has uploaded a document: {text} The document has ended!! The current user input is: {message.content}'\n",
        "\n",
        "                    response = chat.send_message(message.content)\n",
        "                    response = response.text\n",
        "\n",
        "                    save_chat_history(id_str, chat, custom_path)\n",
        "                    await send_message(message, response, id_str)\n",
        "                    print(f\"Bot: {response}\")\n",
        "\n",
        "                else:\n",
        "                    if format in ('image/gif'):\n",
        "                        gif_clip = mp.VideoFileClip(downloaded_file)\n",
        "                        output_path = f\"{downloaded_file.rsplit('.', 1)[0]}.mp4\"\n",
        "                        gif_clip.write_videofile(output_path, codec='libx264')\n",
        "                        downloaded_file = output_path\n",
        "                        format = 'video/mp4'\n",
        "\n",
        "                    media_file = [upload_to_gemini(f\"{downloaded_file}\", mime_type= f\"{format}\"),]\n",
        "\n",
        "                    wait_for_files_active(media_file) # Some files have a processing delay. Wait for them to be ready.\n",
        "\n",
        "                    save_filetwo(id_str, time_file_path, media_file[0].uri)\n",
        "\n",
        "                    response = chat.send_message([message.content, media_file[0]])\n",
        "                    response = response.text\n",
        "\n",
        "                    save_chat_history(id_str, chat, custom_path)\n",
        "                    await send_message(message, response, id_str)\n",
        "                    print(f\"Bot: {response}\")\n",
        "                Direct_upload = False\n",
        "                Link_upload = False\n",
        "\n",
        "            else:\n",
        "                response = chat.send_message(message.content)\n",
        "                response = response.text\n",
        "\n",
        "                save_chat_history(id_str, chat, custom_path)\n",
        "                await send_message(message, response, id_str)\n",
        "                print(f\"Bot: {response}\")\n",
        "\n",
        "        finally:\n",
        "            # Remove the :red_circle: reaction to indicate processing is done\n",
        "            processing_message_id = processing_messages.pop(id_str, None)\n",
        "            if processing_message_id:\n",
        "                processing_message = await message.channel.fetch_message(processing_message_id)\n",
        "                await processing_message.remove_reaction('🔴', client.user)\n",
        "\n",
        "# STEP 5: MAIN ENTRY POINT\n",
        "async def main() -> None:\n",
        "    await client.start(TOKEN)\n",
        "\n",
        "# To run in a cloud service like Google Colab, Kaggle, etc.\n",
        "nest_asyncio.apply()\n",
        "\n",
        "# Run the main function\n",
        "await main()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(For debugging/testing)"
      ],
      "metadata": {
        "id": "V-BnerK4Yxi7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "fullpath = '/content/drive/MyDrive/Discord_bot/738618866699075595_chat_history.pkl'"
      ],
      "metadata": {
        "id": "e4phZmYoYsWN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "with open(fullpath, 'rb') as file:\n",
        "    chat_history = pickle.load(file)"
      ],
      "metadata": {
        "id": "B7QGNd7BYskF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(chat_history)"
      ],
      "metadata": {
        "id": "dvTMbGENYswd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(chat_history)"
      ],
      "metadata": {
        "id": "txG8dqFGY1kj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "download_file('https://tenor.com/view/svay-cute-cutie-gif-26207536', 'test')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GUyroIo4fmqY",
        "outputId": "7ce05062-f784-48e5-fce2-5d019ad8e8ba"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "File downloaded successfully: file_test\n",
            "File downloaded successfully: file_test\n",
            "File renamed to: file_test.gif with MIME type: image/gif\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('image/gif', 'file_test.gif')"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Don't run these\n",
        "\"\"\"target_word = \"Hey mana\"\n",
        "\n",
        "# Remove entries containing the specific word\n",
        "chat_history = [entry for entry in chat_history if target_word not in str(entry)]\n",
        "print(f'Successfully removed entries containing: {target_word}')\n",
        "\n",
        "# Output the updated chat history\n",
        "print(\"Updated chat history:\")\"\"\""
      ],
      "metadata": {
        "id": "uHM3dokja3hh"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}