ALMA_7B_Ja_V2_GPTQ_Ja_En_batch_translation_sample.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/webbigdata-jp/python_sample/blob/main/ALMA_7B_Ja_V2_GPTQ_Ja_En_batch_translation_sample.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9tCLTsfT_qgA"
      },
      "source": [
        "# [ALMA-7B-Ja-V2-GPTQ-Ja-En](https://huggingface.co/webbigdata/ALMA-7B-Ja-V2-GPTQ-Ja-En) batch translaion sample.\n",
        "\n",
        "アップロードされたファイルを英語から日本語、または日本語から英語に一括翻訳し、ファイルとして出力します。  \n",
        "Translate uploaded file from English to Japanese or from Japanese to English in bulk and output it as file.  \n",
        "\n",
        "\n",
        "以下は既知の問題です。  \n",
        "Below are the known issues  \n",
        "\n",
        "- 長い文章を入力するとエラーになります。\n",
        "- 意味のない文章や日本語でない文章を入力すると、出力がおかしくなることがあります。\n",
        "\n",
        "- If you give a long sentence, an error will occur.(This is a limitation of free Colab)\n",
        "- If you provide meaningless sentences or sentences that are not Japanese, the output may become strange."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## (1)Install required libraries"
      ],
      "metadata": {
        "id": "zedYDn634dMn"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ygJ-cNOc_qTi"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "%%shell\n",
        "#@title Install auto-GPTQ\n",
        "pip install auto-gptq==0.4.2"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## (2)Setting Up"
      ],
      "metadata": {
        "id": "pXAuW7B54q1o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Upload Text File(.txt only)\n",
        "import os\n",
        "from google.colab import files\n",
        "import shutil\n",
        "\n",
        "uploaded = files.upload()"
      ],
      "metadata": {
        "id": "7crRQAAE5CGP",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Translation Setting\n",
        "Translation_direction = 'Japanese to English' #@param [\"Japanese to English\", \"English to Japanese\"]"
      ],
      "metadata": {
        "id": "0yUosS3ZK9Pt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%%capture\n",
        "#@title Download Model (may take a few minutes)\n",
        "import torch\n",
        "from transformers import AutoTokenizer\n",
        "from auto_gptq import AutoGPTQForCausalLM\n",
        "import re\n",
        "\n",
        "\n",
        "def contains_japanese(text):\n",
        "    # 日本語の文字範囲を確認するための正規表現パターン\n",
        "    # 平仮名: 3040-309F, 片仮名: 30A0-30FF, 漢字: 4E00-9FAF (旧字体、新字体)\n",
        "    pattern = re.compile('[\\u3040-\\u309F\\u30A0-\\u30FF\\u4E00-\\u9FAF]')\n",
        "    return re.search(pattern, text) is not None\n",
        "\n",
        "\n",
        "def translate_text(prompt, model, tokenizer, Translation_direction):\n",
        "    #print(prompt)\n",
        "    input_ids = tokenizer(prompt, return_tensors=\"pt\", padding=True, max_length=512, truncation=True).input_ids.cuda()\n",
        "    with torch.no_grad():\n",
        "        generated_ids = model.generate(input_ids=input_ids, num_beams=3, max_new_tokens=512, do_sample=True, temperature=0.4, top_p=0.9)\n",
        "\n",
        "    full_output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]\n",
        "\n",
        "    # Extract the part after \"English:\"\n",
        "    if Translation_direction == 'Japanese to English':\n",
        "      translated_output = full_output.split(\"English:\")[-1].strip()\n",
        "    else:\n",
        "      translated_output = full_output.split(\"Japanese:\")[-1].strip()\n",
        "    return translated_output\n",
        "\n",
        "quantized_model_dir = \"webbigdata/ALMA-7B-Ja-V2-GPTQ-Ja-En\"\n",
        "model_basename = \"model\"\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir)\n",
        "model = AutoGPTQForCausalLM.from_quantized(\n",
        "        quantized_model_dir,\n",
        "        model_basename=model_basename,\n",
        "        use_safetensors=True,\n",
        "        device=\"cuda:0\")\n"
      ],
      "metadata": {
        "id": "aypy-hpcKSaG",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## (3)Do translation"
      ],
      "metadata": {
        "id": "2A6hrL9rptoi"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UewwoJ2FAOP4",
        "cellView": "form"
      },
      "outputs": [],
      "source": [
        "\n",
        "\n",
        "#@title Translate line by line and write to file\n",
        "import chardet  # Required for character encoding detection\n",
        "\n",
        "for filename in uploaded.keys():\n",
        "    translated_sentences = []\n",
        "\n",
        "    with open(filename, \"rb\") as file:  # Read file in binary mode\n",
        "        binary_content = file.read()\n",
        "        detected_encoding = chardet.detect(binary_content)[\"encoding\"] or \"sjis\"\n",
        "        content = binary_content.decode(detected_encoding).encode(\"utf-8\").decode(\"utf-8\")  # Convert to utf-8\n",
        "\n",
        "        if Translation_direction == 'Japanese to English':\n",
        "            if contains_japanese(content):\n",
        "              sentences = [s for s in content.split('。') if s]\n",
        "              sentences = [item for sublist in [s.split('\\n') for s in sentences] for item in sublist]\n",
        "            else:\n",
        "              print(content)\n",
        "              sentences = [content]\n",
        "        else:\n",
        "            sentences = [s for s in content.split('.') if s]\n",
        "            sentences = [item for sublist in [s.split('\\n') for s in sentences] for item in sublist]\n",
        "\n",
        "    for sentence in sentences:\n",
        "        sentence = sentence.strip()\n",
        "        if len(sentence) > 0:\n",
        "            if Translation_direction == 'Japanese to English':\n",
        "                if contains_japanese(content):\n",
        "                    ja_prompt = f\"Translate this from Japanese to English:\\nJapanese: {sentence}。\\nEnglish:\"\n",
        "                    translated_sentences.append(translate_text(ja_prompt, model, tokenizer, Translation_direction))\n",
        "                else:\n",
        "                    translated_sentences.append(sentence)\n",
        "            else:\n",
        "                en_prompt = f\"Translate this from English to Japanese:\\nEnglish: {sentence}.\\nJapanese:\"\n",
        "                translated_sentences.append(translate_text(en_prompt, model, tokenizer, Translation_direction))\n",
        "        else:\n",
        "          translated_sentences.append(\"\")\n",
        "\n",
        "    output_filename = filename.replace('.txt', '_Ja_to_En.txt') if Translation_direction == 'Japanese to English' else filename.replace('.txt', '_En_to_Ja.txt')\n",
        "    with open(output_filename, 'w', encoding='utf-8') as f:\n",
        "        f.write('\\n'.join(translated_sentences))\n",
        "\n",
        "    print(f\"Translation compleated. please download files.: {output_filename}\")\n",
        "    files.download(output_filename)\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "mount_file_id": "1RgbYSzxcfWyYSgfiOoPy30SBrRS5k1Rl",
      "authorship_tag": "ABX9TyMF46O0BMulvTb0f2b+SZw7",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}