UPDATE-6-5-2023-TRAIN_NEW_GPT2_MEDIUM_BIG_DATASET_20_6_2021.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "TRAIN-NEW-GPT2-MEDIUM-BIG-DATASET-20-6-2021.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "k2gFeZlGUFSJ",
        "JcBh6PhU6KEX",
        "H_4JO-rMpnrA",
        "WFM1DLd2j75D",
        "kH-lpzHenLSi",
        "SjoHMuaF370m",
        "GEwJhlBr5Cxj",
        "E_z58GEvGCYw"
      ],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Twenkid/GPT2-Bulgarian-Training-Tips-and-Tools/blob/main/UPDATE-6-5-2023-TRAIN_NEW_GPT2_MEDIUM_BIG_DATASET_20_6_2021.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Training GPT2-MEDIUM from scratch on Colab\n",
        "#UPDATE: 6-5-2023: float32 fix (no mixed precision - error) and\n#Chained-generation with unlimited length with directed prompt-injection (see the other video from Todor's youtube channel)\n",
        "\n",
        "Author: Todor Arnaudov - Twenkid, ~ 6/2021 - 9/2021\n",
        "License: MIT\n",
        "\n",
        "See a video tutorial: https://youtu.be/F-Xt-cK4L-g\n",
        "\n",
        "GPT2-Medium Training from Scratch on Colab for Any Language - Tips & Tricks by Twenkid\n",
        "...\n",
        "\n",
        "Съвети за машинно обучение на GPT2-Medium модел на български или друг език от нулата през Google Colaboratory, от Тодор Арнаудов - Тош/Twenkid. Следва продължение.\n",
        "\n",
        "The code was based on a tutorial for training locally the SMALL model by Arshabhi Kayal: Thanks!\n",
        "https://towardsdatascience.com/train-gpt-2-in-your-own-language-fc6ad4d60171 \n",
        "\n",
        "However it was extended with the proper parametrs, for loading data/saving back to Google drive, for randomly selecting part of the dataset in order to allow finishing iterations within the allowed Colab time etc. His example was for local training on an Nvidia Geforce RTX 2060 (6 GB) for GPT2-Small (3 times smaller) and a fixed dataset.\n",
        "\n",
        "The parameters here are for GPT2-MEDIUM and run only with Tesla T4 16GB (or another future card with 16 GB or more).\n",
        "\n",
        "The code in the experiments in this video was extended and debugged for application in the Colaboratory which has its hurdles and for gradual extension of the dataset after each training epoch, without retraining the tokenizer after the corpus is updated (see Dataset Skip None in the video).\n",
        "\n",
        "I trained on a custom-built Bulgarian dataset. \n",
        "\n",
        "The code needs cleaning and more explanations - consult with the video for now.\n",
        "\n",
        "\n",
        "\n",
        "I'm open for collaboration on interesting projects in various R&D domains, AGI and AI etc.\n",
        "\n",
        "http://artificial-mind.blogspot.com\n",
        "http://github.com/twenkid\n",
        "http://research.twenkid.com"
      ],
      "metadata": {
        "id": "k2gFeZlGUFSJ"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PZht80Vn9Q2K",
        "outputId": "ab747294-1119-45f3-a585-8d98f3022615"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive') "
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-U7C4VhLglqD"
      },
      "source": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qJ8cLWyZKI84"
      },
      "source": [],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 661
        },
        "id": "xK5_Zo5Y9eYA",
        "outputId": "a0eeefbe-b67e-4063-934b-d6d0df43e37b"
      },
      "source": [
        "from datetime import datetime\n",
        "!pip install gensim\n",
        "!pip install argparse\n",
        "!pip install tokenizers\n",
        "!pip install pathlib\n",
        "!pip install transformers"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: gensim in /usr/local/lib/python3.10/dist-packages (4.3.1)\n",
            "Requirement already satisfied: scipy>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from gensim) (1.10.1)\n",
            "Requirement already satisfied: numpy>=1.18.5 in /usr/local/lib/python3.10/dist-packages (from gensim) (1.22.4)\n",
            "Requirement already satisfied: smart-open>=1.8.1 in /usr/local/lib/python3.10/dist-packages (from gensim) (6.3.0)\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting argparse\n",
            "  Using cached argparse-1.4.0-py2.py3-none-any.whl (23 kB)\n",
            "Installing collected packages: argparse\n",
            "Successfully installed argparse-1.4.0\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "argparse"
                ]
              }
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: tokenizers in /usr/local/lib/python3.10/dist-packages (0.13.3)\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: pathlib in /usr/local/lib/python3.10/dist-packages (1.0.1)\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.28.1)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2022.10.31)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.12.0)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.13.3)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers) (4.65.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (1.22.4)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.14.1)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.27.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (23.1)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.5.0)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (2023.4.0)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2.0.12)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (1.26.15)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2022.12.7)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "GwwewYrDU_y7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B9ZuM6qS9oCM",
        "outputId": "1dc79f63-f570-4580-c015-6f4b9490a75b"
      },
      "source": [
        "!nvidia-smi"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fri May  5 20:16:14 2023       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 525.85.12    Driver Version: 525.85.12    CUDA Version: 12.0     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   37C    P8     9W /  70W |      0MiB / 15360MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9tRQnjHT2O4w"
      },
      "source": [
        "# TRY mixed-precision 16bit float\n",
        "\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras import mixed_precision\n"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Gx-LcUCW2c8K",
        "outputId": "b01c841d-d137-483e-fe5d-1699fab43b1e"
      },
      "source": [
        "policy = mixed_precision.Policy('mixed_float16')\n",
        "#mixed_precision.set_global_policy('mixed_float16')\n",
        "mixed_precision.set_global_policy('float32')\n",
        "\n",
        "print('Compute dtype: %s' % policy.compute_dtype)\n",
        "print('Variable dtype: %s' % policy.variable_dtype)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Compute dtype: float16\n",
            "Variable dtype: float32\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zVEXRX8X6Pgr",
        "outputId": "a5efb65e-49e0-4730-8766-40e46cd94cbd"
      },
      "source": [
        "!du -sh /content/ab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "du: cannot access '/content/ab': No such file or directory\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "coL72ls_NEHP"
      },
      "source": [
        "!cp -r '/content/drive/MyDrive/corpus/' /content/ab/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JcBh6PhU6KEX"
      },
      "source": [
        "#**Initial run for configuring the model and creating a tokenizer for the initial dataset.**\n",
        "\n",
        "### **WARNING**: Don't run if you have already saved a tokenizer and a model. Run only the first time in order to create a tokenizer and configuration. Then after the first training save the model and afterwards just load it. Tokenize the corpus (you may have extended it in the meantime), select parts of it based on different ctireria if it is too big; create the dataset for tf and filter None items; compile the model and train for another epoch. Then save it etc."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "brIZ2WLh9pf8",
        "outputId": "3bc7d9fb-3a0b-4fa6-f9ec-aa093e9f507f"
      },
      "source": [
        "#DONT RUN IF NOT ALREADY SAVED TOKENIZER ETC.!\n",
        "import os\n",
        "from tokenizers.models import BPE\n",
        "from tokenizers import Tokenizer\n",
        "from tokenizers.decoders import ByteLevel as ByteLevelDecoder\n",
        "from tokenizers.normalizers import NFKC, Sequence\n",
        "from tokenizers.pre_tokenizers import ByteLevel\n",
        "from tokenizers.trainers import BpeTrainer\n",
        "\n",
        "from pathlib import Path\n",
        "\n",
        "#paths = [str(x) for x in Path(\"/content/drive/MyDrive/corpus/\").glob(\"*.txt\")]  \n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")]  #Or edit accordingly to a Drive folder etc.\n",
        "print(paths)\n",
        "class BPE_token(object):\n",
        "    def __init__(self):\n",
        "        self.tokenizer = Tokenizer(BPE())\n",
        "        self.tokenizer.normalizer = Sequence([\n",
        "            NFKC()\n",
        "        ])\n",
        "        self.tokenizer.pre_tokenizer = ByteLevel()\n",
        "        self.tokenizer.decoder = ByteLevelDecoder()\n",
        "\n",
        "    def bpe_train(self, paths):\n",
        "        print(paths)\n",
        "        trainer = BpeTrainer(vocab_size=50000, show_progress=True, inital_alphabet=ByteLevel.alphabet(), special_tokens=[\n",
        "            \"<s>\",\n",
        "            \"<pad>\",\n",
        "            \"</s>\",\n",
        "            \"<unk>\",\n",
        "            \"<mask>\"\n",
        "        ])\n",
        "        #self.tokenizer.train(trainer, paths)\n",
        "        self.tokenizer.train(paths,trainer)\n",
        "\n",
        "    def bpe_train_2(self, paths):\n",
        "        print(paths)\n",
        "        #trainer = BpeTrainer(vocab_size=50000, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        trainer = BpeTrainer(vocab_size=2000, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        #self.tokenizer.train(trainer, paths)\n",
        "        self.tokenizer.train(paths,trainer)\n",
        "\n",
        "    def bpe_train_3(self, paths):\n",
        "        print(paths)\n",
        "        #trainer = BpeTrainer(vocab_size=50000, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        #trainer = BpeTrainer(vocab_size=2000, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        #self.tokenizer.train(trainer, paths)\n",
        "        trainer = BpeTrainer(show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        self.tokenizer.train(paths,trainer)\n",
        "    def bpe_train_4(self, paths):\n",
        "        print(paths)\n",
        "        #trainer = BpeTrainer(vocab_size=50000, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        #trainer = BpeTrainer(vocab_size=2000, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        #self.tokenizer.train(trainer, paths)\n",
        "        trainer = BpeTrainer(vocab_size=50250, show_progress=True, inital_alphabet=ByteLevel.alphabet())\n",
        "        self.tokenizer.train(paths,trainer)\n",
        "\n",
        "    def save_tokenizer(self, location, prefix=None):\n",
        "        if not os.path.exists(location):\n",
        "            os.makedirs(location)\n",
        "        self.tokenizer.model.save(location, prefix)\n",
        "\n",
        "#from tokenise import BPE_token\n",
        "from pathlib import Path\n",
        "import os\n",
        "# the folder 'text' contains all the files\n",
        "#paths = [str(x) for x in Path(\"./text/\").glob(\"**/*.txt\")]\n",
        "#paths = [str(x) for x in Path(\"./en/\").glob(\"**/*.txt\")]\n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")]  #In one file for all, otherwise errors with the shapes?\n",
        "#paths = [str(x) for x in Path(\"/content/ab/UTF8/\").glob(\"*.txt\")] \n",
        "\n",
        "\n",
        "#paths = [str(x) for x in Path(\"/content/ab2/corpus2/UTF8/\").glob(\"*.txt\")]\n",
        "\n",
        "print(paths)\n",
        "'''\n",
        "for i in paths:\n",
        "  print(i)\n",
        "  f = open(i, \"wt\", encoding='utf-8')\n",
        "  r = f.read()\n",
        "  f.write(i,r)\n",
        "'''  \n",
        "tokenizerBIG = BPE_token()\n",
        "# train the tokenizer model\n",
        "tokenizerBIG.bpe_train_4(paths) #no ###########\n",
        "# saving the tokenized data in our specified folder \n",
        "save_path = 'tokenized_data'\n",
        "tokenizerBIG.save_tokenizer(save_path)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['/content/ab/radost.txt', '/content/ab/kakvomu_contents.txt', '/content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt', '/content/ab/telefonni-suveti.txt', '/content/ab/pred4.txt', '/content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', '/content/ab/RISC-CISC.txt', '/content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/democratius_prilozhenie_drugi.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt', '/content/ab/comment-nerds-2-nerds-128-clean1.txt', '/content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', '/content/ab/zaklyuchena-poetry.txt', '/content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', '/content/ab/upaduk_dzbe.txt', '/content/ab/darenie.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', '/content/ab/Isaac-Asimov_-_Fondatsijata_-_1021-b.txt', '/content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt', '/content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', '/content/ab/igri-za.txt', '/content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt', '/content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', '/content/ab/gnyav.txt', '/content/ab/john-1.txt', '/content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', '/content/ab/vyarata_i_svobodata.txt', '/content/ab/doktoratut.txt', '/content/ab/terminator-i-herkules.txt', '/content/ab/ritum.txt', '/content/ab/kakvomu_notes.txt', '/content/ab/seksut.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', '/content/ab/Gore-Vidal_-_Julian_-_7517-b.txt', '/content/ab/alyosha2.txt', '/content/ab/Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt', '/content/ab/igrach-na.txt', '/content/ab/pred3.txt', '/content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', '/content/ab/baladi.txt', '/content/ab/igrach-mekichar.txt', '/content/ab/necessary_illusions.txt', '/content/ab/zmee.txt', '/content/ab/stari.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', '/content/ab/DZBE_ponyatiya_7-2020_eim-33.txt', '/content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', '/content/ab/yunaka-i-taz.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/podaruk.txt', '/content/ab/Isaac-Asimov_-_Samite_bogove_-_1197-b.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/kreativen.txt', '/content/ab/AMD-Intel.txt', '/content/ab/goden.txt', '/content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', '/content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', '/content/ab/igrachut-i-lyubovta.txt', '/content/ab/alyosha1.txt', '/content/ab/tupanasov.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', '/content/ab/bivalica.txt', '/content/ab/prehvur-vidyo.txt', '/content/ab/democratius_stamboliyski.txt', '/content/ab/chlen.txt', '/content/ab/seks-glavna.txt', '/content/ab/Mitut.txt', '/content/ab/igrachut.txt', '/content/ab/rechnik.txt', '/content/ab/stranichni.txt', '/content/ab/razumir-br-4.txt', '/content/ab/dve-narechiya.txt', '/content/ab/trus.txt', '/content/ab/135-godini.txt', '/content/ab/ezikovi-raboti.txt', '/content/ab/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt', '/content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', '/content/ab/mainstream_media.txt', '/content/ab/voyna.txt', '/content/ab/toj-vyara.txt', '/content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', '/content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', '/content/ab/hristo-bozhinov.txt', '/content/ab/podvizi-akro-mezo-stih.txt', '/content/ab/kopiraneto-EIM-1-bez-imena.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', '/content/ab/sreshtu-chuzhdico.txt', '/content/ab/belezhki-istinata.txt', '/content/ab/igrachi-yunaci.txt', '/content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt', '/content/ab/kakvomu_notes_e.txt', '/content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', '/content/ab/kakvomu.txt', '/content/ab/john-2.txt', '/content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt']\n",
            "['/content/ab/radost.txt', '/content/ab/kakvomu_contents.txt', '/content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt', '/content/ab/telefonni-suveti.txt', '/content/ab/pred4.txt', '/content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', '/content/ab/RISC-CISC.txt', '/content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/democratius_prilozhenie_drugi.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt', '/content/ab/comment-nerds-2-nerds-128-clean1.txt', '/content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', '/content/ab/zaklyuchena-poetry.txt', '/content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', '/content/ab/upaduk_dzbe.txt', '/content/ab/darenie.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', '/content/ab/Isaac-Asimov_-_Fondatsijata_-_1021-b.txt', '/content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt', '/content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', '/content/ab/igri-za.txt', '/content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt', '/content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', '/content/ab/gnyav.txt', '/content/ab/john-1.txt', '/content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', '/content/ab/vyarata_i_svobodata.txt', '/content/ab/doktoratut.txt', '/content/ab/terminator-i-herkules.txt', '/content/ab/ritum.txt', '/content/ab/kakvomu_notes.txt', '/content/ab/seksut.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', '/content/ab/Gore-Vidal_-_Julian_-_7517-b.txt', '/content/ab/alyosha2.txt', '/content/ab/Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt', '/content/ab/igrach-na.txt', '/content/ab/pred3.txt', '/content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', '/content/ab/baladi.txt', '/content/ab/igrach-mekichar.txt', '/content/ab/necessary_illusions.txt', '/content/ab/zmee.txt', '/content/ab/stari.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', '/content/ab/DZBE_ponyatiya_7-2020_eim-33.txt', '/content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', '/content/ab/yunaka-i-taz.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/podaruk.txt', '/content/ab/Isaac-Asimov_-_Samite_bogove_-_1197-b.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/kreativen.txt', '/content/ab/AMD-Intel.txt', '/content/ab/goden.txt', '/content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', '/content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', '/content/ab/igrachut-i-lyubovta.txt', '/content/ab/alyosha1.txt', '/content/ab/tupanasov.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', '/content/ab/bivalica.txt', '/content/ab/prehvur-vidyo.txt', '/content/ab/democratius_stamboliyski.txt', '/content/ab/chlen.txt', '/content/ab/seks-glavna.txt', '/content/ab/Mitut.txt', '/content/ab/igrachut.txt', '/content/ab/rechnik.txt', '/content/ab/stranichni.txt', '/content/ab/razumir-br-4.txt', '/content/ab/dve-narechiya.txt', '/content/ab/trus.txt', '/content/ab/135-godini.txt', '/content/ab/ezikovi-raboti.txt', '/content/ab/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt', '/content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', '/content/ab/mainstream_media.txt', '/content/ab/voyna.txt', '/content/ab/toj-vyara.txt', '/content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', '/content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', '/content/ab/hristo-bozhinov.txt', '/content/ab/podvizi-akro-mezo-stih.txt', '/content/ab/kopiraneto-EIM-1-bez-imena.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', '/content/ab/sreshtu-chuzhdico.txt', '/content/ab/belezhki-istinata.txt', '/content/ab/igrachi-yunaci.txt', '/content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt', '/content/ab/kakvomu_notes_e.txt', '/content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', '/content/ab/kakvomu.txt', '/content/ab/john-2.txt', '/content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt']\n",
            "['/content/ab/radost.txt', '/content/ab/kakvomu_contents.txt', '/content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt', '/content/ab/telefonni-suveti.txt', '/content/ab/pred4.txt', '/content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', '/content/ab/RISC-CISC.txt', '/content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/democratius_prilozhenie_drugi.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt', '/content/ab/comment-nerds-2-nerds-128-clean1.txt', '/content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', '/content/ab/zaklyuchena-poetry.txt', '/content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', '/content/ab/upaduk_dzbe.txt', '/content/ab/darenie.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', '/content/ab/Isaac-Asimov_-_Fondatsijata_-_1021-b.txt', '/content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt', '/content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', '/content/ab/igri-za.txt', '/content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt', '/content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', '/content/ab/gnyav.txt', '/content/ab/john-1.txt', '/content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', '/content/ab/vyarata_i_svobodata.txt', '/content/ab/doktoratut.txt', '/content/ab/terminator-i-herkules.txt', '/content/ab/ritum.txt', '/content/ab/kakvomu_notes.txt', '/content/ab/seksut.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', '/content/ab/Gore-Vidal_-_Julian_-_7517-b.txt', '/content/ab/alyosha2.txt', '/content/ab/Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt', '/content/ab/igrach-na.txt', '/content/ab/pred3.txt', '/content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', '/content/ab/baladi.txt', '/content/ab/igrach-mekichar.txt', '/content/ab/necessary_illusions.txt', '/content/ab/zmee.txt', '/content/ab/stari.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', '/content/ab/DZBE_ponyatiya_7-2020_eim-33.txt', '/content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', '/content/ab/yunaka-i-taz.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/podaruk.txt', '/content/ab/Isaac-Asimov_-_Samite_bogove_-_1197-b.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/kreativen.txt', '/content/ab/AMD-Intel.txt', '/content/ab/goden.txt', '/content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', '/content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', '/content/ab/igrachut-i-lyubovta.txt', '/content/ab/alyosha1.txt', '/content/ab/tupanasov.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', '/content/ab/bivalica.txt', '/content/ab/prehvur-vidyo.txt', '/content/ab/democratius_stamboliyski.txt', '/content/ab/chlen.txt', '/content/ab/seks-glavna.txt', '/content/ab/Mitut.txt', '/content/ab/igrachut.txt', '/content/ab/rechnik.txt', '/content/ab/stranichni.txt', '/content/ab/razumir-br-4.txt', '/content/ab/dve-narechiya.txt', '/content/ab/trus.txt', '/content/ab/135-godini.txt', '/content/ab/ezikovi-raboti.txt', '/content/ab/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt', '/content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', '/content/ab/mainstream_media.txt', '/content/ab/voyna.txt', '/content/ab/toj-vyara.txt', '/content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', '/content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', '/content/ab/hristo-bozhinov.txt', '/content/ab/podvizi-akro-mezo-stih.txt', '/content/ab/kopiraneto-EIM-1-bez-imena.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', '/content/ab/sreshtu-chuzhdico.txt', '/content/ab/belezhki-istinata.txt', '/content/ab/igrachi-yunaci.txt', '/content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt', '/content/ab/kakvomu_notes_e.txt', '/content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', '/content/ab/kakvomu.txt', '/content/ab/john-2.txt', '/content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "resources": {
            "http://localhost:8080/nbextensions/google.colab/files.js": {
              "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7CgpmdW5jdGlvbiBfdXBsb2FkRmlsZXMoaW5wdXRJZCwgb3V0cHV0SWQpIHsKICBjb25zdCBzdGVwcyA9IHVwbG9hZEZpbGVzU3RlcChpbnB1dElkLCBvdXRwdXRJZCk7CiAgY29uc3Qgb3V0cHV0RWxlbWVudCA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKG91dHB1dElkKTsKICAvLyBDYWNoZSBzdGVwcyBvbiB0aGUgb3V0cHV0RWxlbWVudCB0byBtYWtlIGl0IGF2YWlsYWJsZSBmb3IgdGhlIG5leHQgY2FsbAogIC8vIHRvIHVwbG9hZEZpbGVzQ29udGludWUgZnJvbSBQeXRob24uCiAgb3V0cHV0RWxlbWVudC5zdGVwcyA9IHN0ZXBzOwoKICByZXR1cm4gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpOwp9CgovLyBUaGlzIGlzIHJvdWdobHkgYW4gYXN5bmMgZ2VuZXJhdG9yIChub3Qgc3VwcG9ydGVkIGluIHRoZSBicm93c2VyIHlldCksCi8vIHdoZXJlIHRoZXJlIGFyZSBtdWx0aXBsZSBhc3luY2hyb25vdXMgc3RlcHMgYW5kIHRoZSBQeXRob24gc2lkZSBpcyBnb2luZwovLyB0byBwb2xsIGZvciBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcC4KLy8gVGhpcyB1c2VzIGEgUHJvbWlzZSB0byBibG9jayB0aGUgcHl0aG9uIHNpZGUgb24gY29tcGxldGlvbiBvZiBlYWNoIHN0ZXAsCi8vIHRoZW4gcGFzc2VzIHRoZSByZXN1bHQgb2YgdGhlIHByZXZpb3VzIHN0ZXAgYXMgdGhlIGlucHV0IHRvIHRoZSBuZXh0IHN0ZXAuCmZ1bmN0aW9uIF91cGxvYWRGaWxlc0NvbnRpbnVlKG91dHB1dElkKSB7CiAgY29uc3Qgb3V0cHV0RWxlbWVudCA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKG91dHB1dElkKTsKICBjb25zdCBzdGVwcyA9IG91dHB1dEVsZW1lbnQuc3RlcHM7CgogIGNvbnN0IG5leHQgPSBzdGVwcy5uZXh0KG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSk7CiAgcmV0dXJuIFByb21pc2UucmVzb2x2ZShuZXh0LnZhbHVlLnByb21pc2UpLnRoZW4oKHZhbHVlKSA9PiB7CiAgICAvLyBDYWNoZSB0aGUgbGFzdCBwcm9taXNlIHZhbHVlIHRvIG1ha2UgaXQgYXZhaWxhYmxlIHRvIHRoZSBuZXh0CiAgICAvLyBzdGVwIG9mIHRoZSBnZW5lcmF0b3IuCiAgICBvdXRwdXRFbGVtZW50Lmxhc3RQcm9taXNlVmFsdWUgPSB2YWx1ZTsKICAgIHJldHVybiBuZXh0LnZhbHVlLnJlc3BvbnNlOwogIH0pOwp9CgovKioKICogR2VuZXJhdG9yIGZ1bmN0aW9uIHdoaWNoIGlzIGNhbGxlZCBiZXR3ZWVuIGVhY2ggYXN5bmMgc3RlcCBvZiB0aGUgdXBsb2FkCiAqIHByb2Nlc3MuCiAqIEBwYXJhbSB7c3RyaW5nfSBpbnB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIGlucHV0IGZpbGUgcGlja2VyIGVsZW1lbnQuCiAqIEBwYXJhbSB7c3RyaW5nfSBvdXRwdXRJZCBFbGVtZW50IElEIG9mIHRoZSBvdXRwdXQgZGlzcGxheS4KICogQHJldHVybiB7IUl0ZXJhYmxlPCFPYmplY3Q+fSBJdGVyYWJsZSBvZiBuZXh0IHN0ZXBzLgogKi8KZnVuY3Rpb24qIHVwbG9hZEZpbGVzU3RlcChpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IGlucHV0RWxlbWVudCA9IGRvY3VtZW50LmdldEVsZW1lbnRCeUlkKGlucHV0SWQpOwogIGlucHV0RWxlbWVudC5kaXNhYmxlZCA9IGZhbHNlOwoKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIG91dHB1dEVsZW1lbnQuaW5uZXJIVE1MID0gJyc7CgogIGNvbnN0IHBpY2tlZFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgaW5wdXRFbGVtZW50LmFkZEV2ZW50TGlzdGVuZXIoJ2NoYW5nZScsIChlKSA9PiB7CiAgICAgIHJlc29sdmUoZS50YXJnZXQuZmlsZXMpOwogICAgfSk7CiAgfSk7CgogIGNvbnN0IGNhbmNlbCA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2J1dHRvbicpOwogIGlucHV0RWxlbWVudC5wYXJlbnRFbGVtZW50LmFwcGVuZENoaWxkKGNhbmNlbCk7CiAgY2FuY2VsLnRleHRDb250ZW50ID0gJ0NhbmNlbCB1cGxvYWQnOwogIGNvbnN0IGNhbmNlbFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgY2FuY2VsLm9uY2xpY2sgPSAoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9OwogIH0pOwoKICAvLyBXYWl0IGZvciB0aGUgdXNlciB0byBwaWNrIHRoZSBmaWxlcy4KICBjb25zdCBmaWxlcyA9IHlpZWxkIHsKICAgIHByb21pc2U6IFByb21pc2UucmFjZShbcGlja2VkUHJvbWlzZSwgY2FuY2VsUHJvbWlzZV0pLAogICAgcmVzcG9uc2U6IHsKICAgICAgYWN0aW9uOiAnc3RhcnRpbmcnLAogICAgfQogIH07CgogIGNhbmNlbC5yZW1vdmUoKTsKCiAgLy8gRGlzYWJsZSB0aGUgaW5wdXQgZWxlbWVudCBzaW5jZSBmdXJ0aGVyIHBpY2tzIGFyZSBub3QgYWxsb3dlZC4KICBpbnB1dEVsZW1lbnQuZGlzYWJsZWQgPSB0cnVlOwoKICBpZiAoIWZpbGVzKSB7CiAgICByZXR1cm4gewogICAgICByZXNwb25zZTogewogICAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgICAgfQogICAgfTsKICB9CgogIGZvciAoY29uc3QgZmlsZSBvZiBmaWxlcykgewogICAgY29uc3QgbGkgPSBkb2N1bWVudC5jcmVhdGVFbGVtZW50KCdsaScpOwogICAgbGkuYXBwZW5kKHNwYW4oZmlsZS5uYW1lLCB7Zm9udFdlaWdodDogJ2JvbGQnfSkpOwogICAgbGkuYXBwZW5kKHNwYW4oCiAgICAgICAgYCgke2ZpbGUudHlwZSB8fCAnbi9hJ30pIC0gJHtmaWxlLnNpemV9IGJ5dGVzLCBgICsKICAgICAgICBgbGFzdCBtb2RpZmllZDogJHsKICAgICAgICAgICAgZmlsZS5sYXN0TW9kaWZpZWREYXRlID8gZmlsZS5sYXN0TW9kaWZpZWREYXRlLnRvTG9jYWxlRGF0ZVN0cmluZygpIDoKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgJ24vYSd9IC0gYCkpOwogICAgY29uc3QgcGVyY2VudCA9IHNwYW4oJzAlIGRvbmUnKTsKICAgIGxpLmFwcGVuZENoaWxkKHBlcmNlbnQpOwoKICAgIG91dHB1dEVsZW1lbnQuYXBwZW5kQ2hpbGQobGkpOwoKICAgIGNvbnN0IGZpbGVEYXRhUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICAgIGNvbnN0IHJlYWRlciA9IG5ldyBGaWxlUmVhZGVyKCk7CiAgICAgIHJlYWRlci5vbmxvYWQgPSAoZSkgPT4gewogICAgICAgIHJlc29sdmUoZS50YXJnZXQucmVzdWx0KTsKICAgICAgfTsKICAgICAgcmVhZGVyLnJlYWRBc0FycmF5QnVmZmVyKGZpbGUpOwogICAgfSk7CiAgICAvLyBXYWl0IGZvciB0aGUgZGF0YSB0byBiZSByZWFkeS4KICAgIGxldCBmaWxlRGF0YSA9IHlpZWxkIHsKICAgICAgcHJvbWlzZTogZmlsZURhdGFQcm9taXNlLAogICAgICByZXNwb25zZTogewogICAgICAgIGFjdGlvbjogJ2NvbnRpbnVlJywKICAgICAgfQogICAgfTsKCiAgICAvLyBVc2UgYSBjaHVua2VkIHNlbmRpbmcgdG8gYXZvaWQgbWVzc2FnZSBzaXplIGxpbWl0cy4gU2VlIGIvNjIxMTU2NjAuCiAgICBsZXQgcG9zaXRpb24gPSAwOwogICAgZG8gewogICAgICBjb25zdCBsZW5ndGggPSBNYXRoLm1pbihmaWxlRGF0YS5ieXRlTGVuZ3RoIC0gcG9zaXRpb24sIE1BWF9QQVlMT0FEX1NJWkUpOwogICAgICBjb25zdCBjaHVuayA9IG5ldyBVaW50OEFycmF5KGZpbGVEYXRhLCBwb3NpdGlvbiwgbGVuZ3RoKTsKICAgICAgcG9zaXRpb24gKz0gbGVuZ3RoOwoKICAgICAgY29uc3QgYmFzZTY0ID0gYnRvYShTdHJpbmcuZnJvbUNoYXJDb2RlLmFwcGx5KG51bGwsIGNodW5rKSk7CiAgICAgIHlpZWxkIHsKICAgICAgICByZXNwb25zZTogewogICAgICAgICAgYWN0aW9uOiAnYXBwZW5kJywKICAgICAgICAgIGZpbGU6IGZpbGUubmFtZSwKICAgICAgICAgIGRhdGE6IGJhc2U2NCwKICAgICAgICB9LAogICAgICB9OwoKICAgICAgbGV0IHBlcmNlbnREb25lID0gZmlsZURhdGEuYnl0ZUxlbmd0aCA9PT0gMCA/CiAgICAgICAgICAxMDAgOgogICAgICAgICAgTWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCk7CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPSBgJHtwZXJjZW50RG9uZX0lIGRvbmVgOwoKICAgIH0gd2hpbGUgKHBvc2l0aW9uIDwgZmlsZURhdGEuYnl0ZUxlbmd0aCk7CiAgfQoKICAvLyBBbGwgZG9uZS4KICB5aWVsZCB7CiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICB9CiAgfTsKfQoKc2NvcGUuZ29vZ2xlID0gc2NvcGUuZ29vZ2xlIHx8IHt9OwpzY29wZS5nb29nbGUuY29sYWIgPSBzY29wZS5nb29nbGUuY29sYWIgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYi5fZmlsZXMgPSB7CiAgX3VwbG9hZEZpbGVzLAogIF91cGxvYWRGaWxlc0NvbnRpbnVlLAp9Owp9KShzZWxmKTsK",
              "headers": [
                [
                  "content-type",
                  "application/javascript"
                ]
              ],
              "ok": true,
              "status": 200,
              "status_text": ""
            }
          },
          "base_uri": "https://localhost:8080/",
          "height": 108
        },
        "id": "m54xtBTWGgyh",
        "outputId": "22956be8-2a5b-4d36-c7fe-3138dd240528"
      },
      "source": [
        "from google.colab import files\n",
        "\n",
        "uploaded = files.upload()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "\n",
              "     <input type=\"file\" id=\"files-b89f3882-9998-4a77-a79d-dd269fba0da6\" name=\"files[]\" multiple disabled\n",
              "        style=\"border:none\" />\n",
              "     <output id=\"result-b89f3882-9998-4a77-a79d-dd269fba0da6\">\n",
              "      Upload widget is only available when the cell has been executed in the\n",
              "      current browser session. Please rerun this cell to enable.\n",
              "      </output>\n",
              "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "Saving computers-Linus-gt-1.txt to computers-Linus-gt-1.txt\n",
            "Saving computers-Linus-gtplus-2.txt to computers-Linus-gtplus-2.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vBriZeuq537Y",
        "outputId": "2f3b6ad1-18fd-4948-f098-3092118e7788"
      },
      "source": [
        "!cat /proc/cpuinfo"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "processor\t: 0\n",
            "vendor_id\t: GenuineIntel\n",
            "cpu family\t: 6\n",
            "model\t\t: 63\n",
            "model name\t: Intel(R) Xeon(R) CPU @ 2.30GHz\n",
            "stepping\t: 0\n",
            "microcode\t: 0xffffffff\n",
            "cpu MHz\t\t: 2299.998\n",
            "cache size\t: 46080 KB\n",
            "physical id\t: 0\n",
            "siblings\t: 2\n",
            "core id\t\t: 0\n",
            "cpu cores\t: 1\n",
            "apicid\t\t: 0\n",
            "initial apicid\t: 0\n",
            "fpu\t\t: yes\n",
            "fpu_exception\t: yes\n",
            "cpuid level\t: 13\n",
            "wp\t\t: yes\n",
            "flags\t\t: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat md_clear arch_capabilities\n",
            "bugs\t\t: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs mmio_stale_data retbleed\n",
            "bogomips\t: 4599.99\n",
            "clflush size\t: 64\n",
            "cache_alignment\t: 64\n",
            "address sizes\t: 46 bits physical, 48 bits virtual\n",
            "power management:\n",
            "\n",
            "processor\t: 1\n",
            "vendor_id\t: GenuineIntel\n",
            "cpu family\t: 6\n",
            "model\t\t: 63\n",
            "model name\t: Intel(R) Xeon(R) CPU @ 2.30GHz\n",
            "stepping\t: 0\n",
            "microcode\t: 0xffffffff\n",
            "cpu MHz\t\t: 2299.998\n",
            "cache size\t: 46080 KB\n",
            "physical id\t: 0\n",
            "siblings\t: 2\n",
            "core id\t\t: 0\n",
            "cpu cores\t: 1\n",
            "apicid\t\t: 1\n",
            "initial apicid\t: 1\n",
            "fpu\t\t: yes\n",
            "fpu_exception\t: yes\n",
            "cpuid level\t: 13\n",
            "wp\t\t: yes\n",
            "flags\t\t: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat md_clear arch_capabilities\n",
            "bugs\t\t: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs mmio_stale_data retbleed\n",
            "bogomips\t: 4599.99\n",
            "clflush size\t: 64\n",
            "cache_alignment\t: 64\n",
            "address sizes\t: 46 bits physical, 48 bits virtual\n",
            "power management:\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mYAnk8dCgb8p",
        "outputId": "b3943d54-5a34-4572-dd6a-08d68955fb34"
      },
      "source": [
        "!du -sh ab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "3.0M\tab\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3FmJxJYeNLn_"
      },
      "source": [
        "!cp -r '/content/drive/MyDrive/tokenized_data'  /content/\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9S0khKSye8EQ"
      },
      "source": [
        "#Or load directly\n",
        "!cp -r '/content/drive/MyDrive/gpt2-medium'  /1024/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "93DqYFypIiH6"
      },
      "source": [
        "!cp -r /content/ab/UTF8/* '/content/drive/MyDrive/corpus/'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k894AyUbSu1Q"
      },
      "source": [
        "!cp -r /content/ab/* '/content/drive/MyDrive/corpus/'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OFfrGkFPIx-B"
      },
      "source": [
        "#COPY TOKENIZED DATA TO DRIVE\n",
        "\n",
        "!cp -r /content/tokenized_data/* '/content/drive/MyDrive/tokenized_data/'\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3SicF4kIJKrR"
      },
      "source": [
        "#AFTER the model is saved\n",
        "\n",
        "!cp -r /content/1024/* '/content/drive/MyDrive/1024/'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "myjHw6Tv95GE"
      },
      "source": [
        "#if needed \n",
        "\n",
        "from google.colab import files\n",
        "files.upload()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hQfBhBJ1l7ix"
      },
      "source": [
        "`#**Create a single-string and tokenize (encode) the raw text**`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 231
        },
        "id": "eRpW2f2KPsrG",
        "outputId": "0897f19c-fa5c-4c0f-f7d4-740a294e49ff"
      },
      "source": [
        "tokenized = True\n",
        "#IF NOT TOKENIZED!\n",
        "'''\n",
        "single_string = ''\n",
        "tokenizerBIG.add_special_tokens({ #should add in text when feeding?!\n",
        "    \"eos_token\": \"</s>\",\n",
        "    \"bos_token\": \"<s>\",\n",
        "    \"unk_token\": \"<unk>\",\n",
        "    \"pad_token\": \"<pad>\",\n",
        "    \"mask_token\": \"<mask>\"\n",
        "  })\n",
        "'''\n",
        "items = []\n",
        "for filename in paths:\n",
        "  with open(filename, \"r\", encoding='utf-8') as f:    \n",
        "    x = f.read()\n",
        "    items.append(x)    \n",
        "    #print(x)\n",
        "  #single_string += x + tokenizerBIG.eos_token\n",
        "  items.append(tokenizerBIG.eos_token)\n",
        "single_string = ''.join(items)\n",
        "print(len(single_string))\n",
        "\n",
        "if (len(single_string)<999): print(\"\\n===============\\n\"+single_string)\n",
        "string_tokenized = tokenizerBIG.encode(single_string) #, truncation=True) #, max_length=100)\n",
        "if (len(string_tokenized)<200000): print(string_tokenized)\n",
        "print(len(string_tokenized))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-8-727329e53654>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     18\u001b[0m     \u001b[0;31m#print(x)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     19\u001b[0m   \u001b[0;31m#single_string += x + tokenizerBIG.eos_token\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m   \u001b[0mitems\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtokenizerBIG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0meos_token\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     21\u001b[0m \u001b[0msingle_string\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mitems\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     22\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msingle_string\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'tokenizerBIG' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H_4JO-rMpnrA"
      },
      "source": [
        "# **Create the model when running for the first time**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "7T9SGRRG9_cm",
        "outputId": "24b580ad-22ee-4cfc-b36e-28f426d9cf55"
      },
      "source": [
        "# CONFIG - BIG MODEL  BIG ...\n",
        "# BIG BIG BIG\n",
        "import tensorflow as tf\n",
        "from transformers import GPT2Config, TFGPT2LMHeadModel, GPT2Tokenizer\n",
        "#from str import join\n",
        "\n",
        "save_path = \"/tokenized_data/\"\n",
        "# loading tokenizer from the saved model path\n",
        "tokenizerBIG = GPT2Tokenizer(vocab_file='/content/tokenized_data/vocab.json', merges_file='/content/tokenized_data/merges.txt')\n",
        "#Will it start now?  !!! merges.txt --> I used vocab.json!!! error bec... ?\n",
        "#tokenizer = GPT2Tokenizer.from_pretrained('gpt2')\n",
        "#SLOW?\n",
        "\n",
        "from pathlib import Path\n",
        "import os\n",
        "# the folder 'text' contains all the files\n",
        "#paths = [str(x) for x in Path(\"./text/\").glob(\"**/*.txt\")]\n",
        "#paths = [str(x) for x in Path(\"./en/\").glob(\"**/*.txt\")]\n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")]  #In one file for all, otherwise errors with the shapes?\n",
        "paths = [str(x) for x in Path(\"/content/ab2/\").glob(\"*.txt\")] \n",
        "#paths = [str(x) for x in Path(\"/content/ab2/corpus2/UTF8/\").glob(\"*.txt\")]\n",
        "\n",
        "print(paths)\n",
        "\n",
        "\n",
        "'''\n",
        "tokenizer.add_special_tokens({\n",
        "  \"eos_token\": \"</s>\",\n",
        "  \"bos_token\": \"<s>\",\n",
        "  \"unk_token\": \"<unk>\",\n",
        "  \"pad_token\": \"<pad>\",\n",
        "  \"mask_token\": \"<mask>\"\n",
        "})\n",
        "'''\n",
        "\n",
        "loaded = False #True\n",
        "if (not loaded):\n",
        "  tokenizerBIG.add_special_tokens({ #should add in text when feeding?!\n",
        "    \"eos_token\": \"</s>\",\n",
        "    \"bos_token\": \"<s>\",\n",
        "    \"unk_token\": \"<unk>\",\n",
        "    \"pad_token\": \"<pad>\",\n",
        "    \"mask_token\": \"<mask>\"\n",
        "  })\n",
        "\n",
        "# creating the configurations from which the model can be made\n",
        "configBIG = GPT2Config(\n",
        "  vocab_size=tokenizerBIG.vocab_size,\n",
        "  bos_token_id=tokenizerBIG.bos_token_id,\n",
        "  eos_token_id=tokenizerBIG.eos_token_id,\n",
        "  n_embd=1280,\n",
        "  n_head = 20,\n",
        "  n_layer = 20 #24 ... experiments\n",
        ")\n",
        "\n",
        "#~ MEDIUM\n",
        "num_heads = 16\n",
        "configBIG = GPT2Config(\n",
        "  vocab_size=tokenizerBIG.vocab_size,\n",
        "  bos_token_id=tokenizerBIG.bos_token_id,\n",
        "  eos_token_id=tokenizerBIG.eos_token_id,\n",
        "  n_embd= 1024,\n",
        "  n_head = num_heads,\n",
        "  n_layer = 24 #num_layers #24 #24\n",
        ")\n",
        "\n",
        "'''\n",
        "num_heads = 18\n",
        "configBIG = GPT2Config(\n",
        "  vocab_size=tokenizerBIG.vocab_size,\n",
        "  bos_token_id=tokenizerBIG.bos_token_id,\n",
        "  eos_token_id=tokenizerBIG.eos_token_id,\n",
        "  n_embd= 1152,\n",
        "  n_head = num_heads,\n",
        "  n_layer = 36 #num_layers #24 #24\n",
        ")\n",
        "'''\n",
        "'''\n",
        "n_embd=1280,\n",
        "n_head = 20,\n",
        "n_layer = 20 \n",
        "block_size = 100BATCH_SIZE = 20\n",
        "BUFFER_SIZE = 5000\n",
        "BATCH_SIZE == n_layer  ? (otherwise can't fit, see shapes)\n",
        "'''\n",
        "\n",
        "# X OK -- fits 14218 MB - Large model -- no.\n",
        "''' OK - fits also!\n",
        "configBIG = GPT2Config(\n",
        "  vocab_size=tokenizerBIG.vocab_size,\n",
        "  bos_token_id=tokenizerBIG.bos_token_id,\n",
        "  eos_token_id=tokenizerBIG.eos_token_id,\n",
        "  n_embd=1600,  \n",
        ")\n",
        "'''\n",
        "''' NO:\n",
        "configBIG = GPT2Config(\n",
        "  vocab_size=tokenizerBIG.vocab_size,\n",
        "  bos_token_id=tokenizerBIG.bos_token_id,\n",
        "  eos_token_id=tokenizerBIG.eos_token_id,\n",
        "  n_embd=1600,\n",
        "  n_layer=48,\n",
        "  n_ctx = 2048,\n",
        "  n_head = 12,\n",
        ")\n",
        "'''\n",
        "\n",
        "configBIG.vocab_size+=5\n",
        "\n",
        "print(\"Vocab size = \"+ str(configBIG.vocab_size))\n",
        "\n",
        "# creating the model\n",
        "modelBIG = TFGPT2LMHeadModel(configBIG)\n",
        "\n",
        "print(\"single_string-->\")\n",
        "single_string = ''\n",
        "items = []\n",
        "n = 1\n",
        "for filename in paths:\n",
        "  with open(filename, \"r\", encoding='utf-8') as f:    \n",
        "    x = f.read()\n",
        "    items.append(x) \n",
        "    print(n, filename)\n",
        "    n+=1\n",
        "    #print(x)\n",
        "  #single_string += x + tokenizerBIG.eos_token\n",
        "  items.append(tokenizerBIG.eos_token)\n",
        "single_string = ''.join(items)\n",
        "print(len(single_string))\n",
        "\n",
        "if (len(single_string)<999): print(\"\\n===============\\n\"+single_string)\n",
        "string_tokenized = tokenizerBIG.encode(single_string) #, truncation=True) #, max_length=100)\n",
        "if (len(string_tokenized)<200000): print(string_tokenized)\n",
        "print(len(string_tokenized))\n",
        "\n",
        "examples = []\n",
        "#block_size = 100\n",
        "#BATCH_SIZE = 12\n",
        "#BUFFER_SIZE = 1000\n",
        "#block_size = 32\n",
        "#BATCH_SIZE = 4\n",
        "#BUFFER_SIZE = 64\n",
        "#... msg BatchDataset: dataset BatchDataset with 1101 items ?\n",
        "# shape [10,99],  [2, 10, 12, 99]  10,12 should be equal? ... 1101919 tokens\n",
        "#if batch_size = 100--> [100,99], [2,100, 12, 99] - last is block_size-1?\n",
        "#!! OK: with: block_size=100; batch_size=12; BUFFER_SIZE=2000\n",
        "\n",
        "block_size = 100 #100 #256 - resources exhausted, block 256, bs=20 #100\n",
        "BATCH_SIZE = num_heads  # # 20 # #24 #for 20 layers - model 1280,36,20; 12 for small model 768 ... #12\n",
        "BUFFER_SIZE = 3000 #  4000 - interrupts connection after one minute; 5000 #3072 #9216 #1000\n",
        "\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "smaller = string_tokenized[0:len(string_tokenized)//3]\n",
        "for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "#  examples.append(string_tokenized[i:i + block_size])\n",
        "  smaller = string_tokenized[0:len(string_tokenized)/3]\n",
        "inputs, labels = [], []\n",
        "k = 0\n",
        "for ex in examples:\n",
        "  inputs.append(ex[:-1])\n",
        "  labels.append(ex[1:])\n",
        "  if k==0:\n",
        "    print(inputs[:100])\n",
        "    print(labels[:100])\n",
        "  k+=1\n",
        "  #print(inputs)\n",
        "  #print(labels)\n",
        "print(k)\n",
        "print(len(inputs))\n",
        "print(len(labels))\n",
        "i1 = len(inputs); i2 = len(labels)\n",
        "i3 = i1*BATCH_SIZE\n",
        "i4 = i1*block_size\n",
        "print(i3)\n",
        "print(i4)\n",
        "datasetBIG = tf.data.Dataset.from_tensor_slices((inputs, labels))\n",
        "datasetBIG = datasetBIG.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\n",
        "print(len(string_tokenized))\n",
        "\n",
        "# defining our optimizer\n",
        "optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)\n",
        "# definining our loss function\n",
        "loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "# defining our metric which we want to observe\n",
        "metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')\n",
        "# compiling the model\n",
        "modelBIG.compile(optimizer=optimizer, loss=[loss, *[None] * modelBIG.config.n_layer], metrics=[metric])\n",
        "print(\"MODEL COMPILED\")\n",
        "\n",
        "#PICKLE!\n",
        "\n",
        "import pickle\n",
        "f = open(\"string_tokenized.pickle\", \"wb\")\n",
        "pickle.Pickler(f,protocol=4).dump(string_tokenized)  # pickle.HIGHEST_PROTOCOL\n",
        "f.close()\n",
        "f2 = open(\"dataset.pickle\", \"wb\")\n",
        "pickle.Pickler(f,protocol=4).dump(dataset)  # pickle.HIGHEST_PROTOCOL\n",
        "f.close()\n",
        "print(\"dataset = \", dataset)\n",
        "\n",
        "!cp -r /content/dataset.pickle '/content/drive/MyDrive/dataset.pickle\n",
        "print(\"Copied Dataset to drive\")\n",
        "\n",
        "num_epoch = 1\n",
        "historyBIG = modelBIG.fit(datasetBIG, epochs=num_epoch)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['/content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt', '/content/ab/necessary_illusions.txt', '/content/ab/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt', '/content/ab/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt', '/content/ab/Anglijski_narodni_prikazki_-_3637-b.txt', '/content/ab/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt', '/content/ab/Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', '/content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', '/content/ab/Platon_-_Dyrzhavata_-_9825-b.txt', '/content/ab/Hristo Botev - V - . Shte vidite - 3687.txt', '/content/ab/democratius_prilozhenie_drugi.txt', '/content/ab/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/telefonni-suveti.txt', '/content/ab/sreshtu-chuzhdico.txt', '/content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt', '/content/ab/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt', '/content/ab/Omir_-_Odiseja_-_6123-b.txt', '/content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt', '/content/ab/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt', '/content/ab/Lev_Tolstoj_-_Vojna_i_mir_-_Pyrvi_i_vtori_tom_-_4403-b.txt', '/content/ab/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt', '/content/ab/kakvomu_contents.txt', '/content/ab/Virginia-Henley -  - . Arabskijat prints - 13610.txt', '/content/ab/Hristo Botev - V - . Videno - 3676.txt', '/content/ab/RISC-CISC.txt', '/content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt', '/content/ab/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', '/content/ab/pred3.txt', '/content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', '/content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', '/content/ab/tupanasov.txt', '/content/ab/Hristo Botev -  - . Poslanie - 3240.txt', '/content/ab/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt', '/content/ab/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt', '/content/ab/William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt', '/content/ab/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt', '/content/ab/Hristo Botev - V - . Retsept - 3678.txt', '/content/ab/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt', '/content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', '/content/ab/Erich-Maria-Remarque_-_Na_Zapadnija_front_nishto_novo._Obratnijat_pyt_-_4001-b.txt', '/content/ab/Alexandre-Jardin_-_Zebrata_-_6966-b.txt', '/content/ab/ezikovi-raboti.txt', '/content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', '/content/ab/Mitut.txt', '/content/ab/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt', '/content/ab/Mircea-Eliade_-_Sakralnoto_i_profannoto_-_1736-b.txt', '/content/ab/Aleksander_-_Synovnik_-_1566-b.txt', '/content/ab/Hristo Botev -  - . Borba - 3236.txt', '/content/ab/ritum.txt', '/content/ab/Arthur-Hailey_-_Hotel_-_2134-b.txt', '/content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt', '/content/ab/Hristo Botev -  - . Kym brata si - 3228.txt', '/content/ab/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt', '/content/ab/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt', '/content/ab/Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt', '/content/ab/Jack-London_-_Belija_zyb._Divoto_zove_-_2135-b.txt', '/content/ab/Hristo Botev - V - . Tova vi chaka! - 3668.txt', '/content/ab/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt', '/content/ab/Anton_Pavlovich_Chehov_-_Damata_s_kuchentseto_-_685-b.txt', '/content/ab/prehvur-vidyo.txt', '/content/ab/Skandinavska_mitologija_-_8141-b.txt', '/content/ab/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt', '/content/ab/zaklyuchena-poetry.txt', '/content/ab/Hiljada_i_edna_nosht_-_Tom_II_-_6538-b.txt', '/content/ab/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt', '/content/ab/john-1.txt', '/content/ab/Jack-London_-_Tom_4._Razkazi_i_noveli_-_331-b.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt', '/content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt', '/content/ab/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt', '/content/ab/Jaroslav-HaYek_-_Prikljuchenijata_na_dobrija_vojnik_Shvejk_prez_Svetovnata_vojna_-_300-b.txt', '/content/ab/AMD-Intel.txt', '/content/ab/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt', '/content/ab/Hristo Botev -  - . Patriot - 3239.txt', '/content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt', '/content/ab/kakvomu_notes.txt', '/content/ab/John-R-R-Tolkien_-_Hobit_-_Bilbo_Begins_ili_dotam_i_obratno_-_3435-b.txt', '/content/ab/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt', '/content/ab/Eshil_-_Tragedii_-_334-b.txt', '/content/ab/Jamamoto_Tsunetomo_-_Hagakure_-_Zapiskite_na_samuraja_-_8839-b.txt', '/content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt', '/content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt', '/content/ab/Hristo Botev -  - . Delba - 3230.txt', '/content/ab/J-D-Salinger_-_Spasiteljat_v_ryzhta_-_1389-b.txt', '/content/ab/chlen.txt', '/content/ab/kakvomu_notes_e.txt', '/content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt', '/content/ab/belezhki-istinata.txt', '/content/ab/vyarata_i_svobodata.txt', '/content/ab/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt', '/content/ab/alyosha1.txt', '/content/ab/goden.txt', '/content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt', '/content/ab/john-2.txt', '/content/ab/terminator-i-herkules.txt', '/content/ab/ -  - 2. Biblija - Nov zavet - 2256.txt', '/content/ab/podaruk.txt', '/content/ab/Hristo Botev - V - . Dolapyt - 3670.txt', '/content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt', '/content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt', '/content/ab/Hristo Botev - V - . Vazhno neshto - 3681.txt', '/content/ab/Hristo Botev -  - . Nej - 3234.txt', '/content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', '/content/ab/Hristo Botev - V - . Fabrika za popove - 3673.txt', '/content/ab/kakvomu.txt', '/content/ab/toj-vyara.txt', '/content/ab/Fxodor_Dostoevski_-_Bratja_Karamazovi_-_306-b.txt', '/content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt', '/content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', '/content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt', '/content/ab/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', '/content/ab/Hristo Botev -  - . Mojata molitva - 3244.txt', '/content/ab/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt', '/content/ab/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt', '/content/ab/Hristo Botev -  - . Strannik - 3237.txt', '/content/ab/Hristo Botev -  - . Pristanala - 3233.txt', '/content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt', '/content/ab/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt', '/content/ab/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt', '/content/ab/Hristo Botev - V - . Podslushano - 3675.txt', '/content/ab/Hristo Botev - V - . Gatanki - 3685.txt', '/content/ab/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt', '/content/ab/Lev_Tolstoj_-_Ana_Karenina_-_3066-b.txt', '/content/ab/zmee.txt', '/content/ab/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt', '/content/ab/Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt', '/content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt', '/content/ab/John-Steinbeck_-_Na_iztok_ot_Raja_-_1953-b.txt', '/content/ab/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt', '/content/ab/DZBE_ponyatiya_7-2020_eim-33.txt', '/content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt', '/content/ab/democratius_stamboliyski.txt', '/content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt', '/content/ab/Hristo Botev -  - . Gergxovden - 3238.txt', '/content/ab/Ernest-Hemingway_-_Za_kogo_bie_kambanata_-_5942-b.txt', '/content/ab/Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt', '/content/ab/kreativen.txt', '/content/ab/William-Shakespeare_-_Soneti_-_9888-b.txt', '/content/ab/comment-nerds-2-nerds-128-clean1.txt', '/content/ab/Jack-London_-_Zheljaznata_peta_-_2042-b.txt', '/content/ab/Hristo Botev -  - . Zadade se oblak temen - 3245.txt', '/content/ab/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt', '/content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt', '/content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt', '/content/ab/Hristo Botev -  - . Majtse si - 3227.txt', '/content/ab/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt', '/content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', '/content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt', '/content/ab/Biblija_-_1435-b.txt', '/content/ab/Terry-Pratchett - Snd - 27. Poslednijat geroj - 1801.txt', '/content/ab/upaduk_dzbe.txt', '/content/ab/seks-glavna.txt', '/content/ab/Hristo Botev - V - . Epitafii - 3679.txt', '/content/ab/pred4.txt', '/content/ab/Hristo Botev - V - . Synovnik - 3680.txt', '/content/ab/Hristo Botev - V - . Politicheska zima - 3666.txt', '/content/ab/135-godini.txt', '/content/ab/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt', '/content/ab/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt', '/content/ab/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt', '/content/ab/kopiraneto-EIM-1-bez-imena.txt', '/content/ab/Hristo Botev - V - . Kazano - 3677.txt', '/content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', '/content/ab/alyosha2.txt', '/content/ab/Hristo Botev -  - . Elegija - 3229.txt', '/content/ab/Hristo Botev - Vnbe - . Namesto programa - 3635.txt', '/content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt', '/content/ab/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', '/content/ab/Pesen_za_nibelungite_-_6576-b.txt', '/content/ab/doktoratut.txt', '/content/ab/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt', '/content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt', '/content/ab/Hristo Botev -  - . Na proshtavane - 3235.txt', '/content/ab/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', '/content/ab/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt', '/content/ab/Hristo Botev -  - . Zashto ne sym_... - 3241.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt', '/content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/Friedrich-Nietzsche_-_Choveshko_tvyrde_choveshko_-_Kniga_za_svobodnite_duhove_tom_1_-_1474-b.txt', '/content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_2_-_6752-b.txt', '/content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', '/content/ab/Hristo Botev - V - . Shte se nameri li_ - 3686.txt', '/content/ab/Hristo Botev - Vnbe - . Petrushan - 3638.txt', '/content/ab/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt', '/content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt', '/content/ab/hristo-bozhinov.txt', '/content/ab/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt', '/content/ab/voyna.txt', '/content/ab/razumir-br-4.txt', '/content/ab/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt', '/content/ab/Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt', '/content/ab/yunaka-i-taz.txt', '/content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', '/content/ab/Hristo Botev -  - . Hajduti - 3232.txt', '/content/ab/George-Orwell_-_1984_-_53-b.txt', '/content/ab/John-R-R-Tolkien - LnSz - 5. Hobit - 17234.txt', '/content/ab/Jack-London_-_Tom_5._Razkazi_i_noveli_-_930-b.txt', '/content/ab/John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', '/content/ab/Hristo Botev -  - . V mehanata - 3243.txt', '/content/ab/Jolene-Prewit-Parker_-_Ameran_-_2908-b.txt', '/content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt', '/content/ab/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt', '/content/ab/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt', '/content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt', '/content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt', '/content/ab/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt', '/content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', '/content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', '/content/ab/Brian-Sibley-John-Howe_-_Karta_na_Tolkinovata_Sredna_zemja_-_6310-b.txt', '/content/ab/mainstream_media.txt']\n",
            "Vocab size = 50255\n",
            "single_string-->\n",
            "1 /content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            "2 /content/ab/necessary_illusions.txt\n",
            "3 /content/ab/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt\n",
            "4 /content/ab/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt\n",
            "5 /content/ab/Anglijski_narodni_prikazki_-_3637-b.txt\n",
            "6 /content/ab/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt\n",
            "7 /content/ab/Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt\n",
            "8 /content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            "9 /content/ab/Platon_-_Dyrzhavata_-_9825-b.txt\n",
            "10 /content/ab/Hristo Botev - V - . Shte vidite - 3687.txt\n",
            "11 /content/ab/democratius_prilozhenie_drugi.txt\n",
            "12 /content/ab/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            "13 /content/ab/telefonni-suveti.txt\n",
            "14 /content/ab/sreshtu-chuzhdico.txt\n",
            "15 /content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            "16 /content/ab/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt\n",
            "17 /content/ab/Omir_-_Odiseja_-_6123-b.txt\n",
            "18 /content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt\n",
            "19 /content/ab/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt\n",
            "20 /content/ab/Lev_Tolstoj_-_Vojna_i_mir_-_Pyrvi_i_vtori_tom_-_4403-b.txt\n",
            "21 /content/ab/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt\n",
            "22 /content/ab/kakvomu_contents.txt\n",
            "23 /content/ab/Virginia-Henley -  - . Arabskijat prints - 13610.txt\n",
            "24 /content/ab/Hristo Botev - V - . Videno - 3676.txt\n",
            "25 /content/ab/RISC-CISC.txt\n",
            "26 /content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            "27 /content/ab/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            "28 /content/ab/pred3.txt\n",
            "29 /content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            "30 /content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            "31 /content/ab/tupanasov.txt\n",
            "32 /content/ab/Hristo Botev -  - . Poslanie - 3240.txt\n",
            "33 /content/ab/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt\n",
            "34 /content/ab/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt\n",
            "35 /content/ab/William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt\n",
            "36 /content/ab/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt\n",
            "37 /content/ab/Hristo Botev - V - . Retsept - 3678.txt\n",
            "38 /content/ab/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt\n",
            "39 /content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            "40 /content/ab/Erich-Maria-Remarque_-_Na_Zapadnija_front_nishto_novo._Obratnijat_pyt_-_4001-b.txt\n",
            "41 /content/ab/Alexandre-Jardin_-_Zebrata_-_6966-b.txt\n",
            "42 /content/ab/ezikovi-raboti.txt\n",
            "43 /content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            "44 /content/ab/Mitut.txt\n",
            "45 /content/ab/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt\n",
            "46 /content/ab/Mircea-Eliade_-_Sakralnoto_i_profannoto_-_1736-b.txt\n",
            "47 /content/ab/Aleksander_-_Synovnik_-_1566-b.txt\n",
            "48 /content/ab/Hristo Botev -  - . Borba - 3236.txt\n",
            "49 /content/ab/ritum.txt\n",
            "50 /content/ab/Arthur-Hailey_-_Hotel_-_2134-b.txt\n",
            "51 /content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            "52 /content/ab/Hristo Botev -  - . Kym brata si - 3228.txt\n",
            "53 /content/ab/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt\n",
            "54 /content/ab/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt\n",
            "55 /content/ab/Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt\n",
            "56 /content/ab/Jack-London_-_Belija_zyb._Divoto_zove_-_2135-b.txt\n",
            "57 /content/ab/Hristo Botev - V - . Tova vi chaka! - 3668.txt\n",
            "58 /content/ab/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt\n",
            "59 /content/ab/Anton_Pavlovich_Chehov_-_Damata_s_kuchentseto_-_685-b.txt\n",
            "60 /content/ab/prehvur-vidyo.txt\n",
            "61 /content/ab/Skandinavska_mitologija_-_8141-b.txt\n",
            "62 /content/ab/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt\n",
            "63 /content/ab/zaklyuchena-poetry.txt\n",
            "64 /content/ab/Hiljada_i_edna_nosht_-_Tom_II_-_6538-b.txt\n",
            "65 /content/ab/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt\n",
            "66 /content/ab/john-1.txt\n",
            "67 /content/ab/Jack-London_-_Tom_4._Razkazi_i_noveli_-_331-b.txt\n",
            "68 /content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            "69 /content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt\n",
            "70 /content/ab/Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            "71 /content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            "72 /content/ab/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt\n",
            "73 /content/ab/Jaroslav-HaYek_-_Prikljuchenijata_na_dobrija_vojnik_Shvejk_prez_Svetovnata_vojna_-_300-b.txt\n",
            "74 /content/ab/AMD-Intel.txt\n",
            "75 /content/ab/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt\n",
            "76 /content/ab/Hristo Botev -  - . Patriot - 3239.txt\n",
            "77 /content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "78 /content/ab/kakvomu_notes.txt\n",
            "79 /content/ab/John-R-R-Tolkien_-_Hobit_-_Bilbo_Begins_ili_dotam_i_obratno_-_3435-b.txt\n",
            "80 /content/ab/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt\n",
            "81 /content/ab/Eshil_-_Tragedii_-_334-b.txt\n",
            "82 /content/ab/Jamamoto_Tsunetomo_-_Hagakure_-_Zapiskite_na_samuraja_-_8839-b.txt\n",
            "83 /content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            "84 /content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            "85 /content/ab/Hristo Botev -  - . Delba - 3230.txt\n",
            "86 /content/ab/J-D-Salinger_-_Spasiteljat_v_ryzhta_-_1389-b.txt\n",
            "87 /content/ab/chlen.txt\n",
            "88 /content/ab/kakvomu_notes_e.txt\n",
            "89 /content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            "90 /content/ab/belezhki-istinata.txt\n",
            "91 /content/ab/vyarata_i_svobodata.txt\n",
            "92 /content/ab/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt\n",
            "93 /content/ab/alyosha1.txt\n",
            "94 /content/ab/goden.txt\n",
            "95 /content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            "96 /content/ab/john-2.txt\n",
            "97 /content/ab/terminator-i-herkules.txt\n",
            "98 /content/ab/ -  - 2. Biblija - Nov zavet - 2256.txt\n",
            "99 /content/ab/podaruk.txt\n",
            "100 /content/ab/Hristo Botev - V - . Dolapyt - 3670.txt\n",
            "101 /content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            "102 /content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            "103 /content/ab/Hristo Botev - V - . Vazhno neshto - 3681.txt\n",
            "104 /content/ab/Hristo Botev -  - . Nej - 3234.txt\n",
            "105 /content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            "106 /content/ab/Hristo Botev - V - . Fabrika za popove - 3673.txt\n",
            "107 /content/ab/kakvomu.txt\n",
            "108 /content/ab/toj-vyara.txt\n",
            "109 /content/ab/Fxodor_Dostoevski_-_Bratja_Karamazovi_-_306-b.txt\n",
            "110 /content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            "111 /content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            "112 /content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt\n",
            "113 /content/ab/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            "114 /content/ab/Hristo Botev -  - . Mojata molitva - 3244.txt\n",
            "115 /content/ab/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt\n",
            "116 /content/ab/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt\n",
            "117 /content/ab/Hristo Botev -  - . Strannik - 3237.txt\n",
            "118 /content/ab/Hristo Botev -  - . Pristanala - 3233.txt\n",
            "119 /content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            "120 /content/ab/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt\n",
            "121 /content/ab/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt\n",
            "122 /content/ab/Hristo Botev - V - . Podslushano - 3675.txt\n",
            "123 /content/ab/Hristo Botev - V - . Gatanki - 3685.txt\n",
            "124 /content/ab/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt\n",
            "125 /content/ab/Lev_Tolstoj_-_Ana_Karenina_-_3066-b.txt\n",
            "126 /content/ab/zmee.txt\n",
            "127 /content/ab/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt\n",
            "128 /content/ab/Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt\n",
            "129 /content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            "130 /content/ab/John-Steinbeck_-_Na_iztok_ot_Raja_-_1953-b.txt\n",
            "131 /content/ab/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt\n",
            "132 /content/ab/DZBE_ponyatiya_7-2020_eim-33.txt\n",
            "133 /content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt\n",
            "134 /content/ab/democratius_stamboliyski.txt\n",
            "135 /content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            "136 /content/ab/Hristo Botev -  - . Gergxovden - 3238.txt\n",
            "137 /content/ab/Ernest-Hemingway_-_Za_kogo_bie_kambanata_-_5942-b.txt\n",
            "138 /content/ab/Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt\n",
            "139 /content/ab/kreativen.txt\n",
            "140 /content/ab/William-Shakespeare_-_Soneti_-_9888-b.txt\n",
            "141 /content/ab/comment-nerds-2-nerds-128-clean1.txt\n",
            "142 /content/ab/Jack-London_-_Zheljaznata_peta_-_2042-b.txt\n",
            "143 /content/ab/Hristo Botev -  - . Zadade se oblak temen - 3245.txt\n",
            "144 /content/ab/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt\n",
            "145 /content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            "146 /content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            "147 /content/ab/Hristo Botev -  - . Majtse si - 3227.txt\n",
            "148 /content/ab/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt\n",
            "149 /content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            "150 /content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            "151 /content/ab/Biblija_-_1435-b.txt\n",
            "152 /content/ab/Terry-Pratchett - Snd - 27. Poslednijat geroj - 1801.txt\n",
            "153 /content/ab/upaduk_dzbe.txt\n",
            "154 /content/ab/seks-glavna.txt\n",
            "155 /content/ab/Hristo Botev - V - . Epitafii - 3679.txt\n",
            "156 /content/ab/pred4.txt\n",
            "157 /content/ab/Hristo Botev - V - . Synovnik - 3680.txt\n",
            "158 /content/ab/Hristo Botev - V - . Politicheska zima - 3666.txt\n",
            "159 /content/ab/135-godini.txt\n",
            "160 /content/ab/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt\n",
            "161 /content/ab/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt\n",
            "162 /content/ab/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt\n",
            "163 /content/ab/kopiraneto-EIM-1-bez-imena.txt\n",
            "164 /content/ab/Hristo Botev - V - . Kazano - 3677.txt\n",
            "165 /content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            "166 /content/ab/alyosha2.txt\n",
            "167 /content/ab/Hristo Botev -  - . Elegija - 3229.txt\n",
            "168 /content/ab/Hristo Botev - Vnbe - . Namesto programa - 3635.txt\n",
            "169 /content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            "170 /content/ab/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            "171 /content/ab/Pesen_za_nibelungite_-_6576-b.txt\n",
            "172 /content/ab/doktoratut.txt\n",
            "173 /content/ab/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt\n",
            "174 /content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            "175 /content/ab/Hristo Botev -  - . Na proshtavane - 3235.txt\n",
            "176 /content/ab/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "177 /content/ab/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt\n",
            "178 /content/ab/Hristo Botev -  - . Zashto ne sym_... - 3241.txt\n",
            "179 /content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt\n",
            "180 /content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            "181 /content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            "182 /content/ab/Friedrich-Nietzsche_-_Choveshko_tvyrde_choveshko_-_Kniga_za_svobodnite_duhove_tom_1_-_1474-b.txt\n",
            "183 /content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_2_-_6752-b.txt\n",
            "184 /content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            "185 /content/ab/Hristo Botev - V - . Shte se nameri li_ - 3686.txt\n",
            "186 /content/ab/Hristo Botev - Vnbe - . Petrushan - 3638.txt\n",
            "187 /content/ab/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt\n",
            "188 /content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            "189 /content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            "190 /content/ab/hristo-bozhinov.txt\n",
            "191 /content/ab/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt\n",
            "192 /content/ab/voyna.txt\n",
            "193 /content/ab/razumir-br-4.txt\n",
            "194 /content/ab/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt\n",
            "195 /content/ab/Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt\n",
            "196 /content/ab/yunaka-i-taz.txt\n",
            "197 /content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            "198 /content/ab/Hristo Botev -  - . Hajduti - 3232.txt\n",
            "199 /content/ab/George-Orwell_-_1984_-_53-b.txt\n",
            "200 /content/ab/John-R-R-Tolkien - LnSz - 5. Hobit - 17234.txt\n",
            "201 /content/ab/Jack-London_-_Tom_5._Razkazi_i_noveli_-_930-b.txt\n",
            "202 /content/ab/John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt\n",
            "203 /content/ab/Hristo Botev -  - . V mehanata - 3243.txt\n",
            "204 /content/ab/Jolene-Prewit-Parker_-_Ameran_-_2908-b.txt\n",
            "205 /content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt\n",
            "206 /content/ab/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt\n",
            "207 /content/ab/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt\n",
            "208 /content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            "209 /content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            "210 /content/ab/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt\n",
            "211 /content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            "212 /content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            "213 /content/ab/Brian-Sibley-John-Howe_-_Karta_na_Tolkinovata_Sredna_zemja_-_6310-b.txt\n",
            "214 /content/ab/mainstream_media.txt\n",
            "81488848\n",
            "19752522\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-35-cc232e57b15d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m    150\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    151\u001b[0m \u001b[0;31m#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 152\u001b[0;31m \u001b[0msmaller\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstring_tokenized\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstring_tokenized\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    153\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstring_tokenized\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mblock_size\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mblock_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    154\u001b[0m \u001b[0;31m#  examples.append(string_tokenized[i:i + block_size])\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: slice indices must be integers or None or have an __index__ method"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "19KXiSeRYF9B"
      },
      "source": [
        "#PICKLE!\n",
        "\n",
        "import pickle\n",
        "f = open(\"string_tokenized.pickle\", \"wb\")\n",
        "pickle.Pickler(f,protocol=4).dump(string_tokenized)  # pickle.HIGHEST_PROTOCOL\n",
        "f.close()\n",
        "#f2 = open(\"dataset.pickle\", \"wb\")\n",
        "#pickle.Pickler(f,protocol=4).dump(dataset)  # pickle.HIGHEST_PROTOCOL\n",
        "#f.close()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ua64gf__g-hw"
      },
      "source": [
        "from pathlib import Path\n",
        "import os\n",
        "# the folder 'text' contains all the files\n",
        "#paths = [str(x) for x in Path(\"./text/\").glob(\"**/*.txt\")]\n",
        "#paths = [str(x) for x in Path(\"./en/\").glob(\"**/*.txt\")]\n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")]  #In one file for all, otherwise errors with the shapes?\n",
        "paths = [str(x) for x in Path(\"/content/ab2/\").glob(\"*.txt\")]\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4fRFGPlHgjKl"
      },
      "source": [
        "from pathlib import Path\n",
        "import os\n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")] "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 302
        },
        "id": "l2E3vT2WYFLD",
        "outputId": "509af3e3-7563-4aa2-fcb9-10a9d5cc6541"
      },
      "source": [
        "#sdfsdofjsdoifj ERRROR SPRI!\n",
        "#dfdfd\n",
        "examples = []\n",
        "#block_size = 100\n",
        "#BATCH_SIZE = 12\n",
        "#BUFFER_SIZE = 1000\n",
        "#block_size = 32\n",
        "#BATCH_SIZE = 4\n",
        "#BUFFER_SIZE = 64\n",
        "#... msg BatchDataset: dataset BatchDataset with 1101 items ?\n",
        "# shape [10,99],  [2, 10, 12, 99]  10,12 should be equal? ... 1101919 tokens\n",
        "#if batch_size = 100--> [100,99], [2,100, 12, 99] - last is block_size-1?\n",
        "#!! OK: with: block_size=100; batch_size=12; BUFFER_SIZE=2000\n",
        "\n",
        "block_size = 120 #100 #256 - resources exhausted, block 256, bs=20 #100\n",
        "BATCH_SIZE = num_heads  # # 20 # #24 #for 20 layers - model 1280,36,20; 12 for small model 768 ... #12\n",
        "BUFFER_SIZE = 3000 #  4000 - interrupts connection after one minute; 5000 #3072 #9216 #1000\n",
        "\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "maxsize = int(len(string_tokenized)*0.8)\n",
        "smaller = string_tokenized[0:len(string_tokenized)] #//2]\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "#  examples.append(string_tokenized[i:i + block_size])\n",
        "bReverse = True #try\n",
        "if not bReverse:\n",
        "  for i in range(0, len(smaller) - block_size + 1, block_size):\n",
        "    examples.append(smaller[i:i + block_size])\n",
        "else if bReverse: #Very long epochs, 3:35 h -- sometimes run in half, but reverse (it's randomized also later; rather generate the dataset again)\n",
        "  for i in range(len(smaller) - block_size + 1, 0, -block_size):\n",
        "    examples.append(smaller[i:i + block_size])\n",
        "\n",
        " \n",
        "inputs, labels = [], []\n",
        "k = 0\n",
        "for ex in examples:\n",
        "  inputs.append(ex[:-1]) #inefficient\n",
        "  labels.append(ex[1:])\n",
        "  if k==0:\n",
        "    print(inputs[:100])\n",
        "    print(labels[:100])\n",
        "  k+=1\n",
        "  if (k%1000==0): print(\"EX: \"+str(k),end=' ')\n",
        "  #print(inputs)\n",
        "  #print(labels)\n",
        "print(k)\n",
        "print(len(inputs))\n",
        "print(len(labels))\n",
        "i1 = len(inputs); i2 = len(labels)\n",
        "i3 = i1*BATCH_SIZE\n",
        "i4 = i1*block_size\n",
        "print(i3)\n",
        "print(i4)\n",
        "datasetBIG = tf.data.Dataset.from_tensor_slices((inputs, labels))\n",
        "datasetBIG = datasetBIG.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\n",
        "print(len(string_tokenized))\n",
        "\n",
        "# defining our optimizer\n",
        "optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)\n",
        "# definining our loss function\n",
        "loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "# defining our metric which we want to observe\n",
        "metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')\n",
        "# compiling the model\n",
        "modelBIG.compile(optimizer=optimizer, loss=[loss, *[None] * modelBIG.config.n_layer], metrics=[metric])\n",
        "print(\"MODEL COMPILED\")\n",
        "\n",
        "#PICKLE!\n",
        "\n",
        "import pickle\n",
        "f = open(\"string_tokenized.pickle\", \"wb\")\n",
        "pickle.Pickler(f,protocol=4).dump(string_tokenized)  # pickle.HIGHEST_PROTOCOL\n",
        "f.close()\n",
        "#f2 = open(\"dataset.pickle\", \"wb\")\n",
        "#pickle.Pickler(f2,protocol=pickle.HIGHEST_PROTOCOL).dump(datasetBIG)  # pickle.HIGHEST_PROTOCOL\n",
        "#f2.close()\n",
        "#print(\"dataset = \", dataset)\n",
        "#\n",
        "#InternalError: Tensorflow type 21 not convertible to numpy dtype.\n",
        "\n",
        "!cp -r /content/dataset.pickle '/content/drive/MyDrive/dataset.pickle\n",
        "print(\"Copied Dataset to drive\")\n",
        "\n",
        "num_epoch = 1 #1\n",
        "historyBIG = modelBIG.fit(datasetBIG, epochs=num_epoch)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[16632, 140, 538, 4579, 1579, 8849, 3814, 141, 140, 32016, 211, 15605, 4326, 13, 3224, 497, 141, 140, 2288, 11968, 1057, 49899, 141, 141, 141, 140, 1470, 30890, 3463, 228, 9838, 224, 625, 20165, 222, 39387, 3327, 13, 26530, 546, 11, 273, 28464, 8499, 11, 4672, 695, 233, 2974, 9648, 11, 31283, 203, 4005, 29358, 198, 1508, 1761, 1007, 11, 239, 1864, 11, 584, 1616, 202, 3529, 211, 32515, 211, 923, 197, 2390, 13719, 202, 205, 44631, 211, 6853, 11, 2701, 198, 1289, 13, 836, 16905, 2663, 15153, 248, 13540, 345, 13, 2828, 700, 273, 546, 6988, 211, 612, 9678, 198, 1645, 1461, 3386, 228, 2297, 874, 850, 6073, 11, 3527, 799, 4323, 258, 13, 141, 140, 15040, 3960]]\n",
            "[[140, 538, 4579, 1579, 8849, 3814, 141, 140, 32016, 211, 15605, 4326, 13, 3224, 497, 141, 140, 2288, 11968, 1057, 49899, 141, 141, 141, 140, 1470, 30890, 3463, 228, 9838, 224, 625, 20165, 222, 39387, 3327, 13, 26530, 546, 11, 273, 28464, 8499, 11, 4672, 695, 233, 2974, 9648, 11, 31283, 203, 4005, 29358, 198, 1508, 1761, 1007, 11, 239, 1864, 11, 584, 1616, 202, 3529, 211, 32515, 211, 923, 197, 2390, 13719, 202, 205, 44631, 211, 6853, 11, 2701, 198, 1289, 13, 836, 16905, 2663, 15153, 248, 13540, 345, 13, 2828, 700, 273, 546, 6988, 211, 612, 9678, 198, 1645, 1461, 3386, 228, 2297, 874, 850, 6073, 11, 3527, 799, 4323, 258, 13, 141, 140, 15040, 3960, 141]]\n",
            "EX: 1000 EX: 2000 EX: 3000 EX: 4000 EX: 5000 EX: 6000 EX: 7000 EX: 8000 EX: 9000 EX: 10000 EX: 11000 EX: 12000 EX: 13000 EX: 14000 EX: 15000 EX: 16000 EX: 17000 EX: 18000 EX: 19000 EX: 20000 EX: 21000 EX: 22000 EX: 23000 EX: 24000 EX: 25000 EX: 26000 EX: 27000 EX: 28000 EX: 29000 EX: 30000 EX: 31000 EX: 32000 EX: 33000 EX: 34000 EX: 35000 EX: 36000 EX: 37000 "
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-59-2cb9dc40f380>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     27\u001b[0m \u001b[0mk\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mex\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mexamples\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m   \u001b[0minputs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mex\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m#inefficient\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     30\u001b[0m   \u001b[0mlabels\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mex\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     31\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0mk\u001b[0m\u001b[0;34m==\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RG7IwociHiQ_"
      },
      "source": [
        "# AFTER LOADING\n",
        "\n",
        "examples = []\n",
        "\n",
        "block_size = 120 #100 #256 - resources exhausted, block 256, bs=20 #100\n",
        "BATCH_SIZE = num_heads  # # 20 # #24 #for 20 layers - model 1280,36,20; 12 for small model 768 ... #12\n",
        "BUFFER_SIZE = 3000 #  4000 - interrupts connection after one minute; 5000 #3072 #9216 #1000\n",
        "\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "maxsize = int(len(string_tokenized)*0.8)\n",
        "smaller = string_tokenized[0:len(string_tokenized)] #//2]\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "#  examples.append(string_tokenized[i:i + block_size])\n",
        "bReverse = True #try\n",
        "if not bReverse:\n",
        "  for i in range(0, len(smaller) - block_size + 1, block_size):\n",
        "    examples.append(smaller[i:i + block_size])\n",
        "else if bReverse: #Very long epochs, 3:35 h -- sometimes run in half, but reverse (it's randomized also later; rather generate the dataset again)\n",
        "  for i in range(len(smaller) - block_size + 1, 0, -block_size):\n",
        "    examples.append(smaller[i:i + block_size])\n",
        "\n",
        "inputs, labels = [], []\n",
        "k = 0\n",
        "for ex in examples:\n",
        "  inputs.append(ex[:-1]) #inefficient\n",
        "  labels.append(ex[1:])\n",
        "  if k==0:\n",
        "    print(inputs[:100])\n",
        "    print(labels[:100])\n",
        "  k+=1\n",
        "  if (k%1000==0): print(\"EX: \"+str(k),end=' ')\n",
        "  #print(inputs)\n",
        "  #print(labels)\n",
        "print(k)\n",
        "print(len(inputs))\n",
        "print(len(labels))\n",
        "i1 = len(inputs); i2 = len(labels)\n",
        "i3 = i1*BATCH_SIZE\n",
        "i4 = i1*block_size\n",
        "print(i3)\n",
        "print(i4)\n",
        "datasetBIG = tf.data.Dataset.from_tensor_slices((inputs, labels))\n",
        "datasetBIG = datasetBIG.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\n",
        "print(len(string_tokenized))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v5R4Y5xyjRfg",
        "outputId": "58c6e2ed-f18e-4b09-9672-15e584dfc788"
      },
      "source": [
        "num_epoch = 1 #1\n",
        "historyBIG = modelBIG.fit(datasetBIG, epochs=num_epoch)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " 749/8482 [=>............................] - ETA: 3:08:10 - loss: 5.3441 - logits_loss: 5.3441 - logits_accuracy: 0.2087 - past_key_values_1_accuracy: 0.0018 - past_key_values_2_accuracy: 0.0021 - past_key_values_3_accuracy: 0.0016 - past_key_values_4_accuracy: 0.0019 - past_key_values_5_accuracy: 0.0019 - past_key_values_6_accuracy: 0.0017 - past_key_values_7_accuracy: 0.0021 - past_key_values_8_accuracy: 0.0018 - past_key_values_9_accuracy: 0.0021 - past_key_values_10_accuracy: 0.0020 - past_key_values_11_accuracy: 0.0016 - past_key_values_12_accuracy: 0.0022 - past_key_values_13_accuracy: 0.0023 - past_key_values_14_accuracy: 0.0019 - past_key_values_15_accuracy: 0.0019 - past_key_values_16_accuracy: 0.0018 - past_key_values_17_accuracy: 0.0017 - past_key_values_18_accuracy: 0.0017 - past_key_values_19_accuracy: 0.0020 - past_key_values_20_accuracy: 0.0022 - past_key_values_21_accuracy: 0.0021 - past_key_values_22_accuracy: 0.0016 - past_key_values_23_accuracy: 0.0018 - past_key_values_24_accuracy: 0.0018"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nCT5Ka9hcJan",
        "outputId": "fe5a6755-aa23-46db-88e4-2f89b2430c72"
      },
      "source": [
        "print(len(string_tokenized))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "19752522\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 394
        },
        "id": "_hc27FgMbDQC",
        "outputId": "e87db823-60d8-4dca-a2f5-503527b1857b"
      },
      "source": [
        "\n",
        "num_epoch = 1\n",
        "historyBIG = modelBIG.fit(datasetBIG, epochs=num_epoch)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4415/8482 [==============>...............] - ETA: 1:38:51 - loss: 5.8585 - logits_loss: 5.8585 - logits_accuracy: 0.1884 - past_key_values_1_accuracy: 0.0018 - past_key_values_2_accuracy: 0.0023 - past_key_values_3_accuracy: 0.0016 - past_key_values_4_accuracy: 0.0018 - past_key_values_5_accuracy: 0.0018 - past_key_values_6_accuracy: 0.0019 - past_key_values_7_accuracy: 0.0022 - past_key_values_8_accuracy: 0.0017 - past_key_values_9_accuracy: 0.0023 - past_key_values_10_accuracy: 0.0023 - past_key_values_11_accuracy: 0.0018 - past_key_values_12_accuracy: 0.0022 - past_key_values_13_accuracy: 0.0024 - past_key_values_14_accuracy: 0.0020 - past_key_values_15_accuracy: 0.0020 - past_key_values_16_accuracy: 0.0017 - past_key_values_17_accuracy: 0.0018 - past_key_values_18_accuracy: 0.0019 - past_key_values_19_accuracy: 0.0020 - past_key_values_20_accuracy: 0.0024 - past_key_values_21_accuracy: 0.0022 - past_key_values_22_accuracy: 0.0016 - past_key_values_23_accuracy: 0.0014 - past_key_values_24_accuracy: 0.0020"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-64-38c167143510>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mnum_epoch\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mhistoryBIG\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodelBIG\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdatasetBIG\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnum_epoch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[1;32m   1181\u001b[0m                 _r=1):\n\u001b[1;32m   1182\u001b[0m               \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1183\u001b[0;31m               \u001b[0mtmp_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1184\u001b[0m               \u001b[0;32mif\u001b[0m \u001b[0mdata_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshould_sync\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1185\u001b[0m                 \u001b[0mcontext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masync_wait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    887\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    888\u001b[0m       \u001b[0;32mwith\u001b[0m \u001b[0mOptionalXlaContext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_jit_compile\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 889\u001b[0;31m         \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    890\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    891\u001b[0m       \u001b[0mnew_tracing_count\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexperimental_get_tracing_count\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    915\u001b[0m       \u001b[0;31m# In this case we have created variables on the first call, so we run the\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    916\u001b[0m       \u001b[0;31m# defunned version which is guaranteed to never create variables.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 917\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateless_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=not-callable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    918\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateful_fn\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    919\u001b[0m       \u001b[0;31m# Release the lock early so that multiple threads can perform the call\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   3022\u001b[0m        filtered_flat_args) = self._maybe_define_function(args, kwargs)\n\u001b[1;32m   3023\u001b[0m     return graph_function._call_flat(\n\u001b[0;32m-> 3024\u001b[0;31m         filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access\n\u001b[0m\u001b[1;32m   3025\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3026\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[0;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[1;32m   1959\u001b[0m       \u001b[0;31m# No tape is watching; skip to running the function.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1960\u001b[0m       return self._build_call_outputs(self._inference_function.call(\n\u001b[0;32m-> 1961\u001b[0;31m           ctx, args, cancellation_manager=cancellation_manager))\n\u001b[0m\u001b[1;32m   1962\u001b[0m     forward_backward = self._select_forward_and_backward_functions(\n\u001b[1;32m   1963\u001b[0m         \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36mcall\u001b[0;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[1;32m    594\u001b[0m               \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    595\u001b[0m               \u001b[0mattrs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mattrs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 596\u001b[0;31m               ctx=ctx)\n\u001b[0m\u001b[1;32m    597\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    598\u001b[0m           outputs = execute.execute_with_cancellation(\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[1;32m     58\u001b[0m     \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     59\u001b[0m     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n\u001b[0;32m---> 60\u001b[0;31m                                         inputs, attrs, num_outputs)\n\u001b[0m\u001b[1;32m     61\u001b[0m   \u001b[0;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     62\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lCZKHKI6vV_s"
      },
      "source": [
        "# matplotlib interprets images in RGB format, but OpenCV uses BGR format\n",
        "# so to convert the image so that it's properly loaded, convert it before loading\n",
        "#TEST 20-6-2021\n",
        "import cv2\n",
        "import matplotlib as plt\n",
        "\n",
        "img = cv2.imread('1.jpg')\t\t# this is read in BGR format\n",
        "rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\t\t# this converts it into RGB\n",
        "plt.imshow(rgb_img)\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HQFhf9PqPh8X",
        "outputId": "82ce2d96-6e4c-4bdc-fd6a-3d39e0d9b646"
      },
      "source": [
        "print(len(string_tokenized))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "21581913\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bHgpSM9ECAW2"
      },
      "source": [
        "#PICKLE!\n",
        "\n",
        "import pickle\n",
        "f = open(\"stringg_tokenized.pickle\", \"wb\")\n",
        "pickle.Pickler(f,protocol=4).dump(string_tokenized)  # pickle.HIGHEST_PROTOCOL\n",
        "f.close()\n",
        "f2 = open(\"dataset.pickle\", \"wb\")\n",
        "pickle.Pickler(f,protocol=4).dump(dataset)  # pickle.HIGHEST_PROTOCOL\n",
        "f.close()\n",
        "print(\"dataset = \", dataset)\n",
        "\n",
        "!cp -r /content/dataset.pickle '/content/drive/MyDrive/dataset.pickle\n",
        "\n",
        "infile = open(dataset.pickle,'rb')\n",
        "dataset_load = pickle.load(infile)\n",
        "infile.close()\n",
        "\n",
        "#!cp -r /content/dataset.pickle '/content/drive/MyDrive/dataset.pickle'\n",
        "\n",
        "print(\"dataset_load = \", dataset_load)\n",
        "#dataset2 = pickle."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X_kjbHCKBD_J",
        "outputId": "f9d7bfad-d562-486d-b726-4defc26855e3"
      },
      "source": [
        "# BIG GENERATE ...\n",
        "# BBBB IIIII GGGGG \n",
        "text = \"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът Емил.\"\"\"\n",
        "text = \"Имало едно време един принц, който имал златно сърце и се запознал с баба Яга. Тя била любовница на юнака и спяла със Снежанка и Терминатора, който програмирал Intel процесори.\"\"\"\n",
        "text = \"Какви са минималните изисквания за играта? Попитал Тодор и се обадил на изкуствения интелект.\"\n",
        "text = \"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът Емил.\"\n",
        "#text = \"Универсалният изкуствен разум и какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Демокрацията е мит, всичко е тоталитарна свобода на словото и любовта. Емил се усмихна на процесора.\"\n",
        "text = \"Не съм съгласен! Той отвори вратата и напусна стаята, в която бяха останали само димящата цигара на Мария и нейните сълзи.\"\n",
        "text = \"Не мога, скъпа! - Защо? - попита тя. - Ти си толкова красива, но искам да гледам мача довечера. Ще играят Ботев и Локомотив. - О, скъпи, няма проблем, тъкмо ще се видя с майка ми.\"\n",
        "number = 1\n",
        "# encoding the input text\n",
        "input_ids = tokenizerBIG.encode(text, return_tensors='tf')\n",
        "# getting out output\n",
        "beam_output = modelBIG.generate(\n",
        "  input_ids,\n",
        "  max_length = 200, # 400, #80\n",
        "  num_beams = 16, #12, 6 # 5,\n",
        "  temperature = 1, #1, #1, #1.0, #1,\n",
        "  top_k = 40, #, 40, #40\n",
        "  #top_p = 0.95,\n",
        "  no_repeat_ngram_size=2, #4, #2\n",
        "  num_return_sequences=number\n",
        ")\n",
        "for i in range(0,number):\n",
        "  print(tokenizerBIG.decode(beam_output[i]))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to 50250 (first `eos_token_id`) to generate sequence\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Не мога, скъпа! - Защо? - попита тя. - Ти си толкова красива, но искам да гледам мача довечера. Ще играят Ботев и Локомотив. - О, скъпи, няма проблем, тъкмо ще се видя с майка ми.\n",
            "\n",
            "    Не знам, че не е по-малко от мен, за да не мога да се случи, защото не се е в състояние, а не може да си кажа какво е, и да е така?\n",
            "\tИ така, ако не би могъл да го е случило, аз не ми се струва. Но аз, ти не знам какво да ме е вярно, ще трябва да ви кажа, та да ти кажа. И аз ще ти казвам, не ти се казвам аз съм го казвам. А аз да ми кажа аз. Ако не съм си казал какво ще ви казвам: „Какво ще ме казвам — казвам ти го ти, да те казвам?“ — отвърнал. Не мога и ще ми казвам какво ти е това, мисля, тя,\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eQvHcLpoGfMx",
        "outputId": "efab353f-990d-42e5-e585-e1e52e87b05d"
      },
      "source": [
        "!du -sh /content/ab/UTF8/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "140M\t/content/ab/UTF8/\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7qdComanHCKo"
      },
      "source": [
        "def ensure_utf(path = \".\"):\n",
        "    #if path = None:\n",
        "    #path = r'Z:/knigi/'\n",
        "    import os\n",
        "    import glob\n",
        "    ls = os.listdir(path)\n",
        "    print(ls)\n",
        "    os.chdir(path)    \n",
        "    print(\"========\")\n",
        "    print(\"Unzip\")\n",
        "    '''\n",
        "    for i in ls:\n",
        "      #cmd = \"unzip + i + \" \" + \"-d \" + i+\".txt\"  #-q quite?  \"-d \" - directory specify\n",
        "      #cmd = \"unzip \" + i  #+ i+\".txt\"  #-q quite?      \n",
        "      #cmd = \"unzip -l \" + \"\\\"\" + i + \"\\\"\"\n",
        "      cmd = \"unzip -j \" + \"\\\"\" + i + \"\\\"\"  #without creating dir\n",
        "      print(cmd)\n",
        "      os.system(cmd)\n",
        "    '''\n",
        "    ls = glob.glob(\"*.txt\")\n",
        "    print(ls)\n",
        "    try:\n",
        "      os.mkdir('UTF8')\n",
        "    except:\n",
        "           print(\"Dir already exists\")\n",
        "    for i in ls:\n",
        "      print(\"TO UTF8:\", i)\n",
        "      f = open(i, \"rt\", encoding='utf-8')\n",
        "      r = f.read()\n",
        "      f2 = open(path+\"/UTF8/\"+i, \"wt\", encoding='utf-8')\n",
        "      f2.write(r)\n",
        "      f.close()\n",
        "      f2.close()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c17osER1HZFl",
        "outputId": "321ce6ce-851f-434f-a947-a9b04d9a78b3"
      },
      "source": [
        "ensure_utf('/content/ab/')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', 'ivo_hristov_svruh-obobshteniya_6-2021.txt', 'alyosha2.txt', 'Lev_Tolstoj_-_Ana_Karenina_-_3066-b.txt.zip', 'Vladimir_Nabokov_-_Lolita_-_275-b.txt.zip', 'yunaka-i-taz.txt', 'Ernest-Hemingway_-_Za_kogo_bie_kambanata_-_5942-b.txt.zip', 'tupanasov.txt', 'Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt.zip', 'Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', 'EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', 'terminator-i-herkules.txt', 'John-Steinbeck_-_Na_iztok_ot_Raja_-_1953-b.txt.zip', 'Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt.zip', 'Hiljada_i_edna_nosht_-_Tom_II_-_6538-b.txt.zip', 'Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt.zip', 'Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt', 'kakvomu_notes.txt', 'Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', 'Brian-Sibley-John-Howe_-_Karta_na_Tolkinovata_Sredna_zemja_-_6310-b.txt', 'talev-svetilnik-nachalo.txt', 'Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', 'upaduk_dzbe.txt', 'Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt.zip', 'zmee.txt', 'Hristo_Botev_-_Nej_-_3234.txt.zip', 'voyna.txt', 'kakvomu.txt', 'Virginia-Henley_-_Arabskijat_prints_-_13610.txt.zip', 'Alexandre-Jardin_-_Zebrata_-_6966-b.txt.zip', 'ritum.txt', 'Petja_Dubarova_-_Az_i_moreto_-_524-b.txt.zip', 'AMD-Intel.txt', 'Hristo_Botev_-_Elegija_-_3229.txt.zip', 'Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', 'democratius_stamboliyski.txt', 'Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', 'Giovanni-Boccaccio_-_Dekameron_-_29-b.txt.zip', 'Platon_-_Dyrzhavata_-_9825-b.txt.zip', 'Fxodor_Dostoevski_-_Bratja_Karamazovi_-_306-b.txt.zip', 'Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt.zip', 'Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt', 'Hristo_Botev_-_Kym_brata_si_-_3228.txt.zip', 'hristo-bozhinov.txt', 'belezhki-istinata.txt', 'Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt.zip', 'Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt.zip', 'Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt.zip', 'Hristo_Botev_-_Patriot_-_3239.txt.zip', 'John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt.zip', 'Jolene-Prewit-Parker_-_Ameran_-_2908-b.txt.zip', 'Pesen_za_nibelungite_-_6576-b.txt.zip', 'toj-vyara.txt', 'Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', 'RISC-CISC.txt', 'Jack-London_-_Tom_4._Razkazi_i_noveli_-_331-b.txt.zip', 'Clean-Kaldata-razdel-bez-html.txt', 'Mircea-Eliade_-_Sakralnoto_i_profannoto_-_1736-b.txt.zip', 'Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt.zip', 'Mitut.txt', 'Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', 'Jack-London_-_Belija_zyb._Divoto_zove_-_2135-b.txt.zip', 'Hristo_Botev_-_Gergxovden_-_3238.txt.zip', 'Hristo_Botev_-_Zashto_ne_sym..._-_3241.txt.zip', 'Irvine-Welsh_-_Porno_-_2467-b.txt.zip', 'Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', 'Srednovekovni_kitajski_noveli_-_8142-b.txt', 'Jaroslav-HaYek_-_Prikljuchenijata_na_dobrija_vojnik_Shvejk_prez_Svetovnata_vojna_-_300-b.txt.zip', 'Jamamoto_Tsunetomo_-_Hagakure_-_Zapiskite_na_samuraja_-_8839-b.txt.zip', 'Hristo_Botev_-_Hadzhi_Dimityr_-_3242.txt.zip', 'Hristo_Botev_-_Djakon_Vasil_Levski_-_3246.txt.zip', 'Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', 'Hristo_Botev_-_Pristanala_-_3233.txt.zip', 'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt', 'Karl-May_-_Vinetu_II_-_1326-b.txt.zip', 'George-Orwell_-_1984_-_53-b.txt.zip', 'john-2.txt', 'Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt', 'sreshtu-chuzhdico.txt', 'Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt.zip', 'chitanka-hristo-botev_vestnik-duma-na-bylgarskite-emigranti-txt.zip', 'John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt.zip', 'kopiraneto-EIM-1-bez-imena.txt', '.ipynb_checkpoints', 'dimov-tyutyun.txt', 'kreativen.txt', 'Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', 'kakvomu_contents.txt', 'Arthur-Hailey_-_Hotel_-_2134-b.txt.zip', 'Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', 'Jack-London_-_Zheljaznata_peta_-_2042-b.txt.zip', 'Arthur-Hailey_-_Letishte_-_32-b.txt.zip', '135-godini.txt', 'democratius_prilozhenie_drugi.txt', 'Hristo_Botev_-_Majtse_si_-_3227.txt.zip', 'seks-glavna.txt', 'Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', 'comment-nerds-2-nerds-128-clean1.txt', 'kakvomu_notes_e.txt', 'Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt.zip', 'Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt.zip', 'Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', 'Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt.zip', 'necessary_illusions.txt', 'Untitled Folder', 'Biblija_-_1435-b.txt.zip', 'Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt.zip', 'chitanka-hristo-botev_vestnik-budilnik-txt.zip', 'Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt.zip', 'Lev_Tolstoj_-_Vojna_i_mir_-_Pyrvi_i_vtori_tom_-_4403-b.txt.zip', 'Jack-London_-_Tom_5._Razkazi_i_noveli_-_930-b.txt.zip', 'Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt.zip', 'sel.zip', 'prehvur-vidyo.txt', 'Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', 'Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt', 'Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', 'Anton_Pavlovich_Chehov_-_Damata_s_kuchentseto_-_685-b.txt.zip', 'John-R-R-Tolkien_-_LnSz_-_5._Hobit_-_17234.txt.zip', 'Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', 'Hristo_Botev_-_V_mehanata_-_3243.txt.zip', 'Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt.zip', 'John-R-R-Tolkien_-_Hobit_-_Bilbo_Begins_ili_dotam_i_obratno_-_3435-b.txt.zip', 'Izkustven_Razum_UIR_bez-http-https_1.txt', 'Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt', 'pred4.txt', 'Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt.zip', 'chitanka-hristo-botev_vestnik-zname-txt.zip', 'Terry-Pratchett_-_Snd_-_27._Poslednijat_geroj_-_1801.txt.zip', 'pred3.txt', 'Eshil_-_Tragedii_-_334-b.txt.zip', 'ezikovi-raboti.txt', 'vyarata_i_svobodata.txt', 'mainstream_media.txt', 'goden.txt', 'Hristo_Botev_-_Do_moeto_pyrvo_libe_-_3231.txt.zip', 'alyosha1.txt', 'podaruk.txt', 'John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', 'Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt.zip', 'Skandinavska_mitologija_-_8141-b.txt.zip', 'Anglijski_narodni_prikazki_-_3637-b.txt.zip', 'doktoratut.txt', 'Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt.zip', 'Hristo_Botev_-_Borba_-_3236.txt.zip', '2._Biblija_-_Nov_zavet_-_2256.txt.zip', 'Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', 'Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', 'Balzak-chast.txt', 'Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt.zip', 'zaklyuchena-poetry.txt', 'talev-svetilnik-2-chast.txt', 'Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_2_-_6752-b.txt', 'Friedrich-Nietzsche_-_Choveshko_tvyrde_choveshko_-_Kniga_za_svobodnite_duhove_tom_1_-_1474-b.txt.zip', 'telefonni-suveti.txt', 'Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', 'Hristo_Botev_-_Strannik_-_3237.txt.zip', 'Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt.zip', 'John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', 'John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', 'Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt.zip', 'Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt.zip', 'Musketari-chast.txt', 'Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt.zip', 'Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt.zip', 'Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt', 'Fxodor_Dostoevski_-_Idiot_-_3180-b.txt.zip', 'William-Shakespeare_-_Soneti_-_9888-b.txt.zip', 'William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt.zip', 'Hristo_Botev_-_Poslanie_-_3240.txt.zip', 'Karl-May_-_Vinetu_I_-_1325-b.txt.zip', 'Hristo_Botev_-_Hajduti_-_3232.txt.zip', 'Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt', 'Hristo_Botev_-_Obesvaneto_na_Vasil_Levski_-_3247.txt.zip', 'Hristo_Botev_-_Delba_-_3230.txt.zip', 'Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt.zip', 'chlen.txt', 'john-1.txt', 'Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt.zip', 'Erich-Maria-Remarque_-_Na_Zapadnija_front_nishto_novo._Obratnijat_pyt_-_4001-b.txt.zip', 'Hristo_Botev_-_Mojata_molitva_-_3244.txt.zip', 'corpus', 'Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', 'Hristo_Botev_-_Na_proshtavane_-_3235.txt.zip', 'Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt', 'Hristo_Botev_-_Zadade_se_oblak_temen_-_3245.txt.zip', 'Omir_-_Odiseja_-_6123-b.txt.zip', 'razumir-br-4.txt', 'DZBE_ponyatiya_7-2020_eim-33.txt', 'J-D-Salinger_-_Spasiteljat_v_ryzhta_-_1389-b.txt.zip']\n",
            "========\n",
            "Unzip\n",
            "['Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', 'ivo_hristov_svruh-obobshteniya_6-2021.txt', 'alyosha2.txt', 'yunaka-i-taz.txt', 'tupanasov.txt', 'Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', 'EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', 'terminator-i-herkules.txt', 'Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt', 'kakvomu_notes.txt', 'Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', 'Brian-Sibley-John-Howe_-_Karta_na_Tolkinovata_Sredna_zemja_-_6310-b.txt', 'talev-svetilnik-nachalo.txt', 'Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', 'upaduk_dzbe.txt', 'zmee.txt', 'voyna.txt', 'kakvomu.txt', 'ritum.txt', 'AMD-Intel.txt', 'Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', 'democratius_stamboliyski.txt', 'Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', 'Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt', 'hristo-bozhinov.txt', 'belezhki-istinata.txt', 'toj-vyara.txt', 'Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', 'RISC-CISC.txt', 'Clean-Kaldata-razdel-bez-html.txt', 'Mitut.txt', 'Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', 'Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', 'Srednovekovni_kitajski_noveli_-_8142-b.txt', 'Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', 'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt', 'john-2.txt', 'Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt', 'sreshtu-chuzhdico.txt', 'kopiraneto-EIM-1-bez-imena.txt', 'dimov-tyutyun.txt', 'kreativen.txt', 'Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', 'kakvomu_contents.txt', 'Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', '135-godini.txt', 'democratius_prilozhenie_drugi.txt', 'seks-glavna.txt', 'Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', 'comment-nerds-2-nerds-128-clean1.txt', 'kakvomu_notes_e.txt', 'Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', 'necessary_illusions.txt', 'prehvur-vidyo.txt', 'Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', 'Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt', 'Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', 'Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', 'Izkustven_Razum_UIR_bez-http-https_1.txt', 'Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt', 'pred4.txt', 'pred3.txt', 'ezikovi-raboti.txt', 'vyarata_i_svobodata.txt', 'mainstream_media.txt', 'goden.txt', 'alyosha1.txt', 'podaruk.txt', 'John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', 'doktoratut.txt', 'Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', 'Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', 'Balzak-chast.txt', 'zaklyuchena-poetry.txt', 'talev-svetilnik-2-chast.txt', 'Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_2_-_6752-b.txt', 'telefonni-suveti.txt', 'Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', 'John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', 'John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', 'Musketari-chast.txt', 'Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt', 'Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt', 'chlen.txt', 'john-1.txt', 'Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', 'Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt', 'razumir-br-4.txt', 'DZBE_ponyatiya_7-2020_eim-33.txt']\n",
            "TO UTF8: Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            "TO UTF8: ivo_hristov_svruh-obobshteniya_6-2021.txt\n",
            "TO UTF8: alyosha2.txt\n",
            "TO UTF8: yunaka-i-taz.txt\n",
            "TO UTF8: tupanasov.txt\n",
            "TO UTF8: Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            "TO UTF8: EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            "TO UTF8: terminator-i-herkules.txt\n",
            "TO UTF8: Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt\n",
            "TO UTF8: kakvomu_notes.txt\n",
            "TO UTF8: Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            "TO UTF8: Brian-Sibley-John-Howe_-_Karta_na_Tolkinovata_Sredna_zemja_-_6310-b.txt\n",
            "TO UTF8: talev-svetilnik-nachalo.txt\n",
            "TO UTF8: Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            "TO UTF8: upaduk_dzbe.txt\n",
            "TO UTF8: zmee.txt\n",
            "TO UTF8: voyna.txt\n",
            "TO UTF8: kakvomu.txt\n",
            "TO UTF8: ritum.txt\n",
            "TO UTF8: AMD-Intel.txt\n",
            "TO UTF8: Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            "TO UTF8: democratius_stamboliyski.txt\n",
            "TO UTF8: Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            "TO UTF8: Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt\n",
            "TO UTF8: hristo-bozhinov.txt\n",
            "TO UTF8: belezhki-istinata.txt\n",
            "TO UTF8: toj-vyara.txt\n",
            "TO UTF8: Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            "TO UTF8: RISC-CISC.txt\n",
            "TO UTF8: Clean-Kaldata-razdel-bez-html.txt\n",
            "TO UTF8: Mitut.txt\n",
            "TO UTF8: Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            "TO UTF8: Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            "TO UTF8: Srednovekovni_kitajski_noveli_-_8142-b.txt\n",
            "TO UTF8: Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            "TO UTF8: Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt\n",
            "TO UTF8: john-2.txt\n",
            "TO UTF8: Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt\n",
            "TO UTF8: sreshtu-chuzhdico.txt\n",
            "TO UTF8: kopiraneto-EIM-1-bez-imena.txt\n",
            "TO UTF8: dimov-tyutyun.txt\n",
            "TO UTF8: kreativen.txt\n",
            "TO UTF8: Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            "TO UTF8: kakvomu_contents.txt\n",
            "TO UTF8: Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            "TO UTF8: 135-godini.txt\n",
            "TO UTF8: democratius_prilozhenie_drugi.txt\n",
            "TO UTF8: seks-glavna.txt\n",
            "TO UTF8: Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            "TO UTF8: comment-nerds-2-nerds-128-clean1.txt\n",
            "TO UTF8: kakvomu_notes_e.txt\n",
            "TO UTF8: Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "TO UTF8: necessary_illusions.txt\n",
            "TO UTF8: prehvur-vidyo.txt\n",
            "TO UTF8: Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            "TO UTF8: Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            "TO UTF8: Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            "TO UTF8: Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            "TO UTF8: Izkustven_Razum_UIR_bez-http-https_1.txt\n",
            "TO UTF8: Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt\n",
            "TO UTF8: pred4.txt\n",
            "TO UTF8: pred3.txt\n",
            "TO UTF8: ezikovi-raboti.txt\n",
            "TO UTF8: vyarata_i_svobodata.txt\n",
            "TO UTF8: mainstream_media.txt\n",
            "TO UTF8: goden.txt\n",
            "TO UTF8: alyosha1.txt\n",
            "TO UTF8: podaruk.txt\n",
            "TO UTF8: John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            "TO UTF8: doktoratut.txt\n",
            "TO UTF8: Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            "TO UTF8: Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            "TO UTF8: Balzak-chast.txt\n",
            "TO UTF8: zaklyuchena-poetry.txt\n",
            "TO UTF8: talev-svetilnik-2-chast.txt\n",
            "TO UTF8: Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_2_-_6752-b.txt\n",
            "TO UTF8: telefonni-suveti.txt\n",
            "TO UTF8: Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            "TO UTF8: John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt\n",
            "TO UTF8: John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            "TO UTF8: Musketari-chast.txt\n",
            "TO UTF8: Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt\n",
            "TO UTF8: Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt\n",
            "TO UTF8: chlen.txt\n",
            "TO UTF8: john-1.txt\n",
            "TO UTF8: Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt\n",
            "TO UTF8: Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt\n",
            "TO UTF8: razumir-br-4.txt\n",
            "TO UTF8: DZBE_ponyatiya_7-2020_eim-33.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_3_nGUDyW4-J",
        "outputId": "0011595b-1c39-4e82-aa60-c16a882e5e58"
      },
      "source": [
        "ensure_utf('/content/sample_data/ab/')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Hristo Botev - Vnbe - . Namesto programa - 3635.txt', 'Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', 'George-Orwell_-_1984_-_53-b.txt', 'Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', 'Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', 'igrachi-yunaci.txt', 'igrach-mekichar.txt', 'Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt', 'Hristo Botev - V - . Podslushano - 3675.txt', 'Giovanni-Boccaccio_-_Dekameron_-_29-b.txt', 'Hristo Botev - V - . Vazhno neshto - 3681.txt', 'Pavel_Vezhinov_-_Vezni_-_10103-b.txt', 'bivalica.txt', 'Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt', 'podvizi-akro-mezo-stih.txt', 'Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt', 'Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt', 'Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', 'Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', 'Hristo Botev - V - . Obrezanie na zhenite - 3683.txt', 'Srednovekovni_kitajski_noveli_-_8142-b.txt', 'EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', 'Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt', 'seksut.txt', 'Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', 'Alexandre-Jardin_-_Zebrata_-_6966-b.txt', 'igrachut.txt', 'Hristo Botev -  - . Hadzhi Dimityr - 3242.txt', 'rechnik.txt', 'Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt', 'Hristo Botev - V - . Retsept - 3678.txt', 'Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', 'Aleko-Schweizariya.txt', 'Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt', 'Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt', 'John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', 'Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt', 'Hristo Botev -  - . Patriot - 3239.txt', 'Gore-Vidal_-_Julian_-_7517-b.txt', 'Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', 'Hristo Botev -  - . Zashto ne sym_... - 3241.txt', 'Eshil_-_Tragedii_-_334-b.txt', 'Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt', 'Anglijski_narodni_prikazki_-_3637-b.txt', 'Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt', 'Isaac-Asimov_-_Fondatsijata_-_1021-b.txt', 'Hristo Botev -  - . V mehanata - 3243.txt', 'Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', 'Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt', 'Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', 'Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt', 'Hristo Botev - V - . Kalyp za uchiteli - 3674.txt', 'Hristo Botev - V - . Shte vidite - 3687.txt', 'Karl-May_-_Vinetu_II_-_1326-b.txt', 'Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', 'Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', 'Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', 'Hristo Botev -  - . Borba - 3236.txt', 'StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', 'Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt', 'Hristo Botev -  - . Elegija - 3229.txt', 'Hristo Botev - V - . Videno - 3676.txt', 'Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt', 'Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt', 'Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', 'Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt', 'Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt', 'trus.txt', 'Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt', 'Platon_-_Dyrzhavata_-_9825-b.txt', 'Hristo Botev -  - . Delba - 3230.txt', 'Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt', 'Hristo Botev -  - . Majtse si - 3227.txt', 'Hristo Botev - V - . Dolapyt - 3670.txt', 'Gore-Vidal_-_Sytvorenieto_-_987-b.txt', 'Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt', 'Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt', 'William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt', 'John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt', 'StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', 'Talev-svetilnik-2-chast.txt', 'Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', 'Pavel_Vezhinov-Duh-na-bademi.txt', 'Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt', 'Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt', 'Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt', 'Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt', 'Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', 'igri-za.txt', 'Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', 'Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', 'Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', 'Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt', 'dimov-tyutyun.txt', 'John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', 'Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt', 'Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', 'Hristo Botev - V - . Synovnik - 3680.txt', 'Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt', 'Hristo Botev - V - . Taja nosht synuvah... - 3672.txt', 'Hristo Botev - V - . Politicheska zima - 3666.txt', 'Hristo Botev -  - . Gergxovden - 3238.txt', 'Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt', 'igrach-na.txt', 'Karl-May_-_Vinetu_I_-_1325-b.txt', 'Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt', 'Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt', 'Hristo Botev -  - . Kym brata si - 3228.txt', 'Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', 'Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt', 'John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', 'Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt', 'John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', 'radost.txt', 'Hristo Botev - V - . Epitafii - 3679.txt', 'Skandinavska_mitologija_-_8141-b.txt', 'Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt', 'IT_Kaldata-razdel-bez-html.txt', 'Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt', 'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt', 'gnyav.txt', 'Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt', 'Hristo Botev -  - . Hajduti - 3232.txt', 'Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt', 'Arthur-Hailey_-_Letishte_-_32-b.txt', 'Hristo Botev -  - . Strannik - 3237.txt', 'Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt', 'Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt', 'Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt', 'Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt', 'Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt', 'Omir_-_Odiseja_-_6123-b.txt', 'dve-narechiya.txt', 'Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt', 'Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', 'Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt', 'Talev-svetilnik-nachalo.txt', '_-  - 2. Biblija - Nov zavet - 2256.txt', 'Hristo Botev -  - . Nej - 3234.txt', 'Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt', 'Hristo Botev - Vnbe - . Smeshen plach - 3637.txt', 'darenie.txt', 'Balzak-chast.txt', 'Hristo Botev - V - . Shte se nameri li_ - 3686.txt', 'Andre-Maurois_-_Bajron_-_3615-b.txt', 'Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', 'Hristo Botev - V - . Tova vi chaka! - 3668.txt', 'Bruder-Grimm_-_Prikazki_-_648-b.txt', 'StanisYaw-Lem_-_Edem_-_1368-b.txt', 'Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt', 'Hristo Botev - V - . O, tempora! O, mores! - 3669.txt', 'Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt', 'Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt', 'Pesen_za_nibelungite_-_6576-b.txt', 'Hristo Botev - Vnbe - . Petrushan - 3638.txt', 'Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', 'Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt', 'Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', 'Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', 'Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', 'Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt', 'Biblija_-_1435-b.txt', 'stranichni.txt', 'Hristo Botev - V - . Gatanki - 3685.txt', 'Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', 'Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt', 'Hristo Botev -  - . Na proshtavane - 3235.txt', 'Hristo Botev -  - . Poslanie - 3240.txt', 'Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt', 'Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', 'Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', 'igrachut-i-lyubovta.txt', 'Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt', 'John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', 'Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', 'Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', 'Vladimir_Nabokov_-_Lolita_-_275-b.txt', 'Petja_Dubarova_-_Az_i_moreto_-_524-b.txt', 'Hristo Botev - V - . Fabrika za popove - 3673.txt', 'Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt', 'Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', 'Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', 'Fxodor_Dostoevski_-_Idiot_-_3180-b.txt', 'Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt', 'Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt', 'Hristo Botev -  - . Pristanala - 3233.txt', 'Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', 'Hristo Botev -  - . Zadade se oblak temen - 3245.txt', 'Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt', 'baladi.txt', 'Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt', 'Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt', 'Hristo Botev -  - . Mojata molitva - 3244.txt', 'Voyna_Kum_Vechnata_Voyna.txt', 'Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt', 'Hristo Botev - V - . Kazano - 3677.txt', 'Hristo Botev -  - . Djakon Vasil Levski - 3246.txt', 'Musketari-chast.txt', 'Tosh_Chovekut-i-Misleshtata_Mashina.txt', 'Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt', 'Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', 'Irvine-Welsh_-_Porno_-_2467-b.txt', 'Isaac-Asimov_-_Samite_bogove_-_1197-b.txt', 'Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt', 'Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', 'Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt', 'stari.txt', 'StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', 'Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt', 'Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', 'Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt']\n",
            "========\n",
            "Unzip\n",
            "['Hristo Botev - Vnbe - . Namesto programa - 3635.txt', 'Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', 'George-Orwell_-_1984_-_53-b.txt', 'Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', 'Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', 'igrachi-yunaci.txt', 'igrach-mekichar.txt', 'Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt', 'Hristo Botev - V - . Podslushano - 3675.txt', 'Giovanni-Boccaccio_-_Dekameron_-_29-b.txt', 'Hristo Botev - V - . Vazhno neshto - 3681.txt', 'Pavel_Vezhinov_-_Vezni_-_10103-b.txt', 'bivalica.txt', 'Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt', 'podvizi-akro-mezo-stih.txt', 'Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt', 'Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt', 'Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', 'Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', 'Hristo Botev - V - . Obrezanie na zhenite - 3683.txt', 'Srednovekovni_kitajski_noveli_-_8142-b.txt', 'EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt', 'Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt', 'seksut.txt', 'Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', 'Alexandre-Jardin_-_Zebrata_-_6966-b.txt', 'igrachut.txt', 'Hristo Botev -  - . Hadzhi Dimityr - 3242.txt', 'rechnik.txt', 'Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt', 'Hristo Botev - V - . Retsept - 3678.txt', 'Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', 'Aleko-Schweizariya.txt', 'Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt', 'Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt', 'John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', 'Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt', 'Hristo Botev -  - . Patriot - 3239.txt', 'Gore-Vidal_-_Julian_-_7517-b.txt', 'Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', 'Hristo Botev -  - . Zashto ne sym_... - 3241.txt', 'Eshil_-_Tragedii_-_334-b.txt', 'Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt', 'Anglijski_narodni_prikazki_-_3637-b.txt', 'Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt', 'Isaac-Asimov_-_Fondatsijata_-_1021-b.txt', 'Hristo Botev -  - . V mehanata - 3243.txt', 'Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', 'Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt', 'Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', 'Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt', 'Hristo Botev - V - . Kalyp za uchiteli - 3674.txt', 'Hristo Botev - V - . Shte vidite - 3687.txt', 'Karl-May_-_Vinetu_II_-_1326-b.txt', 'Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', 'Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', 'Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', 'Hristo Botev -  - . Borba - 3236.txt', 'StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', 'Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt', 'Hristo Botev -  - . Elegija - 3229.txt', 'Hristo Botev - V - . Videno - 3676.txt', 'Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt', 'Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt', 'Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', 'Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt', 'Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt', 'trus.txt', 'Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt', 'Platon_-_Dyrzhavata_-_9825-b.txt', 'Hristo Botev -  - . Delba - 3230.txt', 'Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt', 'Hristo Botev -  - . Majtse si - 3227.txt', 'Hristo Botev - V - . Dolapyt - 3670.txt', 'Gore-Vidal_-_Sytvorenieto_-_987-b.txt', 'Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt', 'Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt', 'William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt', 'John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt', 'StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', 'Talev-svetilnik-2-chast.txt', 'Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', 'Pavel_Vezhinov-Duh-na-bademi.txt', 'Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt', 'Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt', 'Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt', 'Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt', 'Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', 'igri-za.txt', 'Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', 'Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', 'Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', 'Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt', 'dimov-tyutyun.txt', 'John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', 'Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt', 'Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', 'Hristo Botev - V - . Synovnik - 3680.txt', 'Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt', 'Hristo Botev - V - . Taja nosht synuvah... - 3672.txt', 'Hristo Botev - V - . Politicheska zima - 3666.txt', 'Hristo Botev -  - . Gergxovden - 3238.txt', 'Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt', 'igrach-na.txt', 'Karl-May_-_Vinetu_I_-_1325-b.txt', 'Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt', 'Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt', 'Hristo Botev -  - . Kym brata si - 3228.txt', 'Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', 'Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt', 'John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', 'Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt', 'John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', 'radost.txt', 'Hristo Botev - V - . Epitafii - 3679.txt', 'Skandinavska_mitologija_-_8141-b.txt', 'Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt', 'IT_Kaldata-razdel-bez-html.txt', 'Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt', 'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt', 'gnyav.txt', 'Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt', 'Hristo Botev -  - . Hajduti - 3232.txt', 'Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt', 'Arthur-Hailey_-_Letishte_-_32-b.txt', 'Hristo Botev -  - . Strannik - 3237.txt', 'Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt', 'Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt', 'Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt', 'Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt', 'Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt', 'Omir_-_Odiseja_-_6123-b.txt', 'dve-narechiya.txt', 'Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt', 'Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', 'Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt', 'Talev-svetilnik-nachalo.txt', '_-  - 2. Biblija - Nov zavet - 2256.txt', 'Hristo Botev -  - . Nej - 3234.txt', 'Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt', 'Hristo Botev - Vnbe - . Smeshen plach - 3637.txt', 'darenie.txt', 'Balzak-chast.txt', 'Hristo Botev - V - . Shte se nameri li_ - 3686.txt', 'Andre-Maurois_-_Bajron_-_3615-b.txt', 'Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', 'Hristo Botev - V - . Tova vi chaka! - 3668.txt', 'Bruder-Grimm_-_Prikazki_-_648-b.txt', 'StanisYaw-Lem_-_Edem_-_1368-b.txt', 'Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt', 'Hristo Botev - V - . O, tempora! O, mores! - 3669.txt', 'Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt', 'Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt', 'Pesen_za_nibelungite_-_6576-b.txt', 'Hristo Botev - Vnbe - . Petrushan - 3638.txt', 'Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', 'Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt', 'Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', 'Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', 'Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', 'Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt', 'Biblija_-_1435-b.txt', 'stranichni.txt', 'Hristo Botev - V - . Gatanki - 3685.txt', 'Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', 'Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt', 'Hristo Botev -  - . Na proshtavane - 3235.txt', 'Hristo Botev -  - . Poslanie - 3240.txt', 'Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt', 'Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', 'Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', 'igrachut-i-lyubovta.txt', 'Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt', 'John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', 'Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', 'Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', 'Vladimir_Nabokov_-_Lolita_-_275-b.txt', 'Petja_Dubarova_-_Az_i_moreto_-_524-b.txt', 'Hristo Botev - V - . Fabrika za popove - 3673.txt', 'Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt', 'Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', 'Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', 'Fxodor_Dostoevski_-_Idiot_-_3180-b.txt', 'Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt', 'Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt', 'Hristo Botev -  - . Pristanala - 3233.txt', 'Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', 'Hristo Botev -  - . Zadade se oblak temen - 3245.txt', 'Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt', 'baladi.txt', 'Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt', 'Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt', 'Hristo Botev -  - . Mojata molitva - 3244.txt', 'Voyna_Kum_Vechnata_Voyna.txt', 'Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt', 'Hristo Botev - V - . Kazano - 3677.txt', 'Hristo Botev -  - . Djakon Vasil Levski - 3246.txt', 'Musketari-chast.txt', 'Tosh_Chovekut-i-Misleshtata_Mashina.txt', 'Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt', 'Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', 'Irvine-Welsh_-_Porno_-_2467-b.txt', 'Isaac-Asimov_-_Samite_bogove_-_1197-b.txt', 'Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt', 'Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', 'Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt', 'stari.txt', 'StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', 'Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt', 'Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', 'Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt']\n",
            "TO UTF8: Hristo Botev - Vnbe - . Namesto programa - 3635.txt\n",
            "TO UTF8: Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt\n",
            "TO UTF8: George-Orwell_-_1984_-_53-b.txt\n",
            "TO UTF8: Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            "TO UTF8: Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            "TO UTF8: igrachi-yunaci.txt\n",
            "TO UTF8: igrach-mekichar.txt\n",
            "TO UTF8: Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt\n",
            "TO UTF8: Hristo Botev - V - . Podslushano - 3675.txt\n",
            "TO UTF8: Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Vazhno neshto - 3681.txt\n",
            "TO UTF8: Pavel_Vezhinov_-_Vezni_-_10103-b.txt\n",
            "TO UTF8: bivalica.txt\n",
            "TO UTF8: Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt\n",
            "TO UTF8: podvizi-akro-mezo-stih.txt\n",
            "TO UTF8: Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt\n",
            "TO UTF8: Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt\n",
            "TO UTF8: Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt\n",
            "TO UTF8: Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Obrezanie na zhenite - 3683.txt\n",
            "TO UTF8: Srednovekovni_kitajski_noveli_-_8142-b.txt\n",
            "TO UTF8: EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            "TO UTF8: Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            "TO UTF8: seksut.txt\n",
            "TO UTF8: Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt\n",
            "TO UTF8: Alexandre-Jardin_-_Zebrata_-_6966-b.txt\n",
            "TO UTF8: igrachut.txt\n",
            "TO UTF8: Hristo Botev -  - . Hadzhi Dimityr - 3242.txt\n",
            "TO UTF8: rechnik.txt\n",
            "TO UTF8: Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Retsept - 3678.txt\n",
            "TO UTF8: Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            "TO UTF8: Aleko-Schweizariya.txt\n",
            "TO UTF8: Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt\n",
            "TO UTF8: John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            "TO UTF8: Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Patriot - 3239.txt\n",
            "TO UTF8: Gore-Vidal_-_Julian_-_7517-b.txt\n",
            "TO UTF8: Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Zashto ne sym_... - 3241.txt\n",
            "TO UTF8: Eshil_-_Tragedii_-_334-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt\n",
            "TO UTF8: Anglijski_narodni_prikazki_-_3637-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt\n",
            "TO UTF8: Isaac-Asimov_-_Fondatsijata_-_1021-b.txt\n",
            "TO UTF8: Hristo Botev -  - . V mehanata - 3243.txt\n",
            "TO UTF8: Ivan Vazov - Tc - 3. Nova zemja - 3754.txt\n",
            "TO UTF8: Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt\n",
            "TO UTF8: Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt\n",
            "TO UTF8: Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Kalyp za uchiteli - 3674.txt\n",
            "TO UTF8: Hristo Botev - V - . Shte vidite - 3687.txt\n",
            "TO UTF8: Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            "TO UTF8: Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt\n",
            "TO UTF8: Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            "TO UTF8: Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Borba - 3236.txt\n",
            "TO UTF8: StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt\n",
            "TO UTF8: Hristo Botev -  - . Elegija - 3229.txt\n",
            "TO UTF8: Hristo Botev - V - . Videno - 3676.txt\n",
            "TO UTF8: Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt\n",
            "TO UTF8: Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            "TO UTF8: Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt\n",
            "TO UTF8: Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt\n",
            "TO UTF8: trus.txt\n",
            "TO UTF8: Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt\n",
            "TO UTF8: Platon_-_Dyrzhavata_-_9825-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Delba - 3230.txt\n",
            "TO UTF8: Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Majtse si - 3227.txt\n",
            "TO UTF8: Hristo Botev - V - . Dolapyt - 3670.txt\n",
            "TO UTF8: Gore-Vidal_-_Sytvorenieto_-_987-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt\n",
            "TO UTF8: Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt\n",
            "TO UTF8: William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt\n",
            "TO UTF8: John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            "TO UTF8: StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt\n",
            "TO UTF8: Talev-svetilnik-2-chast.txt\n",
            "TO UTF8: Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            "TO UTF8: Pavel_Vezhinov-Duh-na-bademi.txt\n",
            "TO UTF8: Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt\n",
            "TO UTF8: Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt\n",
            "TO UTF8: Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt\n",
            "TO UTF8: Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt\n",
            "TO UTF8: Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt\n",
            "TO UTF8: igri-za.txt\n",
            "TO UTF8: Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            "TO UTF8: Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            "TO UTF8: Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt\n",
            "TO UTF8: dimov-tyutyun.txt\n",
            "TO UTF8: John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt\n",
            "TO UTF8: Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            "TO UTF8: Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Synovnik - 3680.txt\n",
            "TO UTF8: Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Taja nosht synuvah... - 3672.txt\n",
            "TO UTF8: Hristo Botev - V - . Politicheska zima - 3666.txt\n",
            "TO UTF8: Hristo Botev -  - . Gergxovden - 3238.txt\n",
            "TO UTF8: Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            "TO UTF8: igrach-na.txt\n",
            "TO UTF8: Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            "TO UTF8: Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt\n",
            "TO UTF8: Hristo Botev -  - . Kym brata si - 3228.txt\n",
            "TO UTF8: Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            "TO UTF8: Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt\n",
            "TO UTF8: John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt\n",
            "TO UTF8: John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt\n",
            "TO UTF8: radost.txt\n",
            "TO UTF8: Hristo Botev - V - . Epitafii - 3679.txt\n",
            "TO UTF8: Skandinavska_mitologija_-_8141-b.txt\n",
            "TO UTF8: Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            "TO UTF8: IT_Kaldata-razdel-bez-html.txt\n",
            "TO UTF8: Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            "TO UTF8: Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt\n",
            "TO UTF8: gnyav.txt\n",
            "TO UTF8: Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Hajduti - 3232.txt\n",
            "TO UTF8: Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt\n",
            "TO UTF8: Arthur-Hailey_-_Letishte_-_32-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Strannik - 3237.txt\n",
            "TO UTF8: Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt\n",
            "TO UTF8: Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt\n",
            "TO UTF8: Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            "TO UTF8: Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "TO UTF8: Omir_-_Odiseja_-_6123-b.txt\n",
            "TO UTF8: dve-narechiya.txt\n",
            "TO UTF8: Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt\n",
            "TO UTF8: Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt\n",
            "TO UTF8: Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt\n",
            "TO UTF8: Talev-svetilnik-nachalo.txt\n",
            "TO UTF8: _-  - 2. Biblija - Nov zavet - 2256.txt\n",
            "TO UTF8: Hristo Botev -  - . Nej - 3234.txt\n",
            "TO UTF8: Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt\n",
            "TO UTF8: Hristo Botev - Vnbe - . Smeshen plach - 3637.txt\n",
            "TO UTF8: darenie.txt\n",
            "TO UTF8: Balzak-chast.txt\n",
            "TO UTF8: Hristo Botev - V - . Shte se nameri li_ - 3686.txt\n",
            "TO UTF8: Andre-Maurois_-_Bajron_-_3615-b.txt\n",
            "TO UTF8: Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Tova vi chaka! - 3668.txt\n",
            "TO UTF8: Bruder-Grimm_-_Prikazki_-_648-b.txt\n",
            "TO UTF8: StanisYaw-Lem_-_Edem_-_1368-b.txt\n",
            "TO UTF8: Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt\n",
            "TO UTF8: Hristo Botev - V - . O, tempora! O, mores! - 3669.txt\n",
            "TO UTF8: Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt\n",
            "TO UTF8: Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            "TO UTF8: Pesen_za_nibelungite_-_6576-b.txt\n",
            "TO UTF8: Hristo Botev - Vnbe - . Petrushan - 3638.txt\n",
            "TO UTF8: Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt\n",
            "TO UTF8: Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "TO UTF8: Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            "TO UTF8: Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt\n",
            "TO UTF8: Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt\n",
            "TO UTF8: Biblija_-_1435-b.txt\n",
            "TO UTF8: stranichni.txt\n",
            "TO UTF8: Hristo Botev - V - . Gatanki - 3685.txt\n",
            "TO UTF8: Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt\n",
            "TO UTF8: Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Na proshtavane - 3235.txt\n",
            "TO UTF8: Hristo Botev -  - . Poslanie - 3240.txt\n",
            "TO UTF8: Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt\n",
            "TO UTF8: Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt\n",
            "TO UTF8: Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            "TO UTF8: igrachut-i-lyubovta.txt\n",
            "TO UTF8: Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt\n",
            "TO UTF8: John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt\n",
            "TO UTF8: Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            "TO UTF8: Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            "TO UTF8: Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            "TO UTF8: Petja_Dubarova_-_Az_i_moreto_-_524-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Fabrika za popove - 3673.txt\n",
            "TO UTF8: Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt\n",
            "TO UTF8: Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt\n",
            "TO UTF8: Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt\n",
            "TO UTF8: Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            "TO UTF8: Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt\n",
            "TO UTF8: Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Pristanala - 3233.txt\n",
            "TO UTF8: Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Zadade se oblak temen - 3245.txt\n",
            "TO UTF8: Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt\n",
            "TO UTF8: baladi.txt\n",
            "TO UTF8: Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            "TO UTF8: Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            "TO UTF8: Hristo Botev -  - . Mojata molitva - 3244.txt\n",
            "TO UTF8: Voyna_Kum_Vechnata_Voyna.txt\n",
            "TO UTF8: Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt\n",
            "TO UTF8: Hristo Botev - V - . Kazano - 3677.txt\n",
            "TO UTF8: Hristo Botev -  - . Djakon Vasil Levski - 3246.txt\n",
            "TO UTF8: Musketari-chast.txt\n",
            "TO UTF8: Tosh_Chovekut-i-Misleshtata_Mashina.txt\n",
            "TO UTF8: Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            "TO UTF8: Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            "TO UTF8: Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            "TO UTF8: Isaac-Asimov_-_Samite_bogove_-_1197-b.txt\n",
            "TO UTF8: Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt\n",
            "TO UTF8: Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt\n",
            "TO UTF8: Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt\n",
            "TO UTF8: stari.txt\n",
            "TO UTF8: StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt\n",
            "TO UTF8: Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt\n",
            "TO UTF8: Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            "TO UTF8: Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ICYGoSFmFsiB",
        "outputId": "de4b7000-f554-4ca6-e098-66f683779e70"
      },
      "source": [
        "!ls /content/ab/\n",
        "!du -sh /content/ab/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'_-  - 2. Biblija - Nov zavet - 2256.txt'\n",
            " Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt\n",
            " Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt\n",
            " Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt\n",
            " Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt\n",
            " Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt'\n",
            " Aleko-Schweizariya.txt\n",
            " Alexandre-Jardin_-_Zebrata_-_6966-b.txt\n",
            " Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            " Andre-Maurois_-_Bajron_-_3615-b.txt\n",
            " Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt\n",
            " Anglijski_narodni_prikazki_-_3637-b.txt\n",
            " Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            " Arthur-Hailey_-_Letishte_-_32-b.txt\n",
            " Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt\n",
            " Balzak-chast.txt\n",
            " Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            " Biblija_-_1435-b.txt\n",
            " Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt\n",
            " Bruder-Grimm_-_Prikazki_-_648-b.txt\n",
            " Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt\n",
            " Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt\n",
            " dimov-tyutyun.txt\n",
            " Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            " DZBE_ponyatiya_7-2020_eim-33.txt\n",
            " EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            " Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            " Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            " Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            " Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt\n",
            " Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            " Eshil_-_Tragedii_-_334-b.txt\n",
            " Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            " Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            " Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            " George-Orwell_-_1984_-_53-b.txt\n",
            " Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            " Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            " Gore-Vidal_-_Julian_-_7517-b.txt\n",
            " Gore-Vidal_-_Sytvorenieto_-_987-b.txt\n",
            " Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            " Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt\n",
            " Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "'Hristo Botev -  - . Borba - 3236.txt'\n",
            "'Hristo Botev -  - . Delba - 3230.txt'\n",
            "'Hristo Botev -  - . Djakon Vasil Levski - 3246.txt'\n",
            "'Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt'\n",
            "'Hristo Botev -  - . Elegija - 3229.txt'\n",
            "'Hristo Botev -  - . Gergxovden - 3238.txt'\n",
            "'Hristo Botev -  - . Hadzhi Dimityr - 3242.txt'\n",
            "'Hristo Botev -  - . Hajduti - 3232.txt'\n",
            "'Hristo Botev -  - . Kym brata si - 3228.txt'\n",
            "'Hristo Botev -  - . Majtse si - 3227.txt'\n",
            "'Hristo Botev -  - . Mojata molitva - 3244.txt'\n",
            "'Hristo Botev -  - . Na proshtavane - 3235.txt'\n",
            "'Hristo Botev -  - . Nej - 3234.txt'\n",
            "'Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt'\n",
            "'Hristo Botev -  - . Patriot - 3239.txt'\n",
            "'Hristo Botev -  - . Poslanie - 3240.txt'\n",
            "'Hristo Botev -  - . Pristanala - 3233.txt'\n",
            "'Hristo Botev -  - . Strannik - 3237.txt'\n",
            "'Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt'\n",
            "'Hristo Botev - V - . Dolapyt - 3670.txt'\n",
            "'Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt'\n",
            "'Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt'\n",
            "'Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt'\n",
            "'Hristo Botev - V - . Epitafii - 3679.txt'\n",
            "'Hristo Botev - V - . Fabrika za popove - 3673.txt'\n",
            "'Hristo Botev - V - . Gatanki - 3685.txt'\n",
            "'Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt'\n",
            "'Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt'\n",
            "'Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt'\n",
            "'Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt'\n",
            "'Hristo Botev - V - . Kalyp za uchiteli - 3674.txt'\n",
            "'Hristo Botev - V - . Kazano - 3677.txt'\n",
            "'Hristo Botev -  - . V mehanata - 3243.txt'\n",
            "'Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt'\n",
            "'Hristo Botev - Vnbe - . Namesto programa - 3635.txt'\n",
            "'Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt'\n",
            "'Hristo Botev - Vnbe - . Petrushan - 3638.txt'\n",
            "'Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt'\n",
            "'Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt'\n",
            "'Hristo Botev - Vnbe - . Smeshen plach - 3637.txt'\n",
            "'Hristo Botev - V - . Obrezanie na zhenite - 3683.txt'\n",
            "'Hristo Botev - V - . O, tempora! O, mores! - 3669.txt'\n",
            "'Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt'\n",
            "'Hristo Botev - V - . Podslushano - 3675.txt'\n",
            "'Hristo Botev - V - . Politicheska zima - 3666.txt'\n",
            "'Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt'\n",
            "'Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt'\n",
            "'Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt'\n",
            "'Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt'\n",
            "'Hristo Botev - V - . Retsept - 3678.txt'\n",
            "'Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt'\n",
            "'Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt'\n",
            "'Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt'\n",
            "'Hristo Botev - V - . Shte se nameri li_ - 3686.txt'\n",
            "'Hristo Botev - V - . Shte vidite - 3687.txt'\n",
            "'Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt'\n",
            "'Hristo Botev - V - . Synovnik - 3680.txt'\n",
            "'Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt'\n",
            "'Hristo Botev - V - . Taja nosht synuvah... - 3672.txt'\n",
            "'Hristo Botev - V - . Tova vi chaka! - 3668.txt'\n",
            "'Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt'\n",
            "'Hristo Botev - V - . Vazhno neshto - 3681.txt'\n",
            "'Hristo Botev - V - . Videno - 3676.txt'\n",
            "'Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt'\n",
            "'Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt'\n",
            "'Hristo Botev -  - . Zadade se oblak temen - 3245.txt'\n",
            "'Hristo Botev -  - . Zashto ne sym_... - 3241.txt'\n",
            " Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            " Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            " Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt\n",
            " Isaac-Asimov_-_Fondatsijata_-_1021-b.txt\n",
            " Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt\n",
            " Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt\n",
            " Istinata.txt\n",
            " Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            " Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            " Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            " Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt\n",
            " Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt\n",
            "'Ivan Vazov - Tc - 3. Nova zemja - 3754.txt'\n",
            " Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            " Izkustven_RAZUM_UIR_bez-http.txt\n",
            " Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            " Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            " Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            " John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt\n",
            " John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt\n",
            " John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt\n",
            " John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            " John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            " John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            " Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            " Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt\n",
            " Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            " Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            " Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            " Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt\n",
            " Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            " Lev_Tolstoj_Vojna_i_Mir-KRAYA_NARODITE.txt\n",
            " Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            " Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            " Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt\n",
            " Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt\n",
            " Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            " Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt\n",
            " Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            " Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            " Musketari-chast.txt\n",
            " Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            " Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt\n",
            " Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            " Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            " Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            " Omir_-_Odiseja_-_6123-b.txt\n",
            " Pavel_Vezhinov-Duh-na-bademi.txt\n",
            " Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt\n",
            " Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt\n",
            " Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt\n",
            " Pavel_Vezhinov_-_Vezni_-_10103-b.txt\n",
            " Pesen_za_nibelungite_-_6576-b.txt\n",
            " Petja_Dubarova_-_Az_i_moreto_-_524-b.txt\n",
            " Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            " Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt\n",
            " Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt\n",
            " Platon_-_Dyrzhavata_-_9825-b.txt\n",
            " Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt\n",
            " Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            " Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            " Skandinavska_mitologija_-_8141-b.txt\n",
            " Srednovekovni_kitajski_noveli_-_8142-b.txt\n",
            " StanisYaw-Lem_-_Edem_-_1368-b.txt\n",
            " StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt\n",
            " StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt\n",
            " StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt\n",
            " Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            " Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            " Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt\n",
            " Talev-svetilnik-2-chast.txt\n",
            " Talev-svetilnik-nachalo.txt\n",
            " Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt\n",
            " Tosh_Chovekut-i-Misleshtata_Mashina.txt\n",
            " Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt\n",
            " Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            " Voyna_Kum_Vechnata_Voyna.txt\n",
            " William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt\n",
            " Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt\n",
            " Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt\n",
            " Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt\n",
            "133M\t/content/ab/\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tNA1hhGzBOjf"
      },
      "source": [
        "text = \"Компютрите са бързи, но хората обичат да летят покрай космически кораби.\"\n",
        "text = \"Членът отишъл в космическия кораб и излетял.\"\n",
        "text = \"What is Todor?\"\n",
        "#text = \"Ада отвори вратата и Емил я последва. На земята лежаха златни бижута.\"\n",
        "text = \"Ада отвори вратата и Емил я последва. Коридорът светеше до бижутера, когато Тичко се уплаши. Чернокосата дръпна метлата.\"\n",
        "text = \"Чернокосата ритна Тичко в топките и той излетя с метлата. Ана го погледна и се разсмя, а Емил падна с лазера.\"\n",
        "text = \"Обичам книги и Ана. Асансьорът потегли и срещнаха Широкоплещестият обаче метлата излетя от коридора и падна при картините в асансьора.\"\n",
        "#text = \"Ана, ти си единственото заради което момче като мен има сили да живее.\"\n",
        "#text = \"Компютрите са бързи, но хората обичат да летят покрай космически кораби.\"\n",
        "#Ада отвори вратата и Емил я последва в Англия. Коридорът светеше до бижутера, когато той се страхуваше за свободата си. Какво му трябва на човек\n",
        "text = \"Най-бързият процесор е 1 ГХц и има 500 милиона транзистора, но не е по-добър от Жигули, най-хубавата кола според песните на Наско Ментата. Какво му трябва на човек, освен едно ферари?\"\n",
        "#text = \"\"\"Да бъдеш или да не бъдеш - попита проститутката? Това е въпросът, задник! - отговори Терминаторът - застреля БМВ-то с пушката си и колата с Джон Конър се преобърна. Тогава момичето засмука хия му, докато той не свърши в устата и тя рече: О, толкова ти е голям!\"\"\"\n",
        "text = \"\"\"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът - застреля президента с пушката си и колата с Емил се преобърна. Тогава момичето го целуна и те умряха.\"\"\"\n",
        "text = \"\"\"Програмирането и изкуствения интелект са тъпотии, с които не се занимават феминистки. Демокрацията е мит.\"\"\"\n",
        "text = \"\"\"Универсалният изкуствен разум и какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Демокрацията е мит, всичко е тоталитарна свобода на словото и любовта. Емил се усмихна на процесора.\"\"\"\n",
        "text = \"\"\"Какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Рап. Демокрацията е мит. Любовта и свещеният сметач са най-добрите - рече Тош и се обади на държавата.\"\"\"\n",
        "text = \"\"\"Какво му трябва на човек? Парите, колите, жените и песните!\"\"\"\n",
        "text = \"\"\"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът - застреля президента с пушката си и колата с Емил се преобърна. Тогава момичето го целуна и те умряха.\"\"\"\n",
        "text = \"Членът на ЦК на БКП отишъл в космическия кораб и излетял при лордовете господари от Англия, когато нацистите кацнали в България със сметачите си Правец-8М и се видели с Ангел и Тош пред телевизора. Тогава Фейз се обадил на Коко и отишли да ритат пред блока.\"\n",
        "text = \"Най-бързият процесор е 1 ГХц и има 500 милиона транзистора, но не е по-добър от Жигули, най-хубавата кола според песните на Наско Ментата. Какво му трябва на човек, освен едно ферари?\"\n",
        "text = \"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът Емил.\"\"\"\n",
        "number = 1\n",
        "# encoding the input text\n",
        "input_ids = tokenizer.encode(text, return_tensors='tf')\n",
        "# getting out output\n",
        "beam_output = model.generate(\n",
        "  input_ids,\n",
        "  max_length = 200, # 400, #80\n",
        "  num_beams = 12, #12, 6 # 5,\n",
        "  temperature = 1, #1.0, #1,\n",
        "  top_k = 10, #, 40, #40\n",
        "  top_p = 0.95,\n",
        "  no_repeat_ngram_size=2, #4, #2\n",
        "  num_return_sequences=number\n",
        ")\n",
        "for i in range(0,number):\n",
        "  print(tokenizer.decode(beam_output[i]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "suy5vnwFBwSU",
        "outputId": "82dd9ae2-30e3-4f2d-b85f-9714033d9c70"
      },
      "source": [
        "#SAVE BIG   \n",
        "   \n",
        "from transformers import WEIGHTS_NAME, CONFIG_NAME\n",
        "import os\n",
        "output_dir = '/content/1024/'\n",
        "output_dir = '/content/gpt2/' #FOR tosh.bg ...\n",
        "output_dir = '/content/drive/MyDrive/gpt2-medium/'\n",
        "#or drive\n",
        "#output_dir = '/content/drive/My Drive/gpt-bg/'  #for emil\n",
        "\n",
        "# creating directory if it is not present\n",
        "if not os.path.exists(output_dir):\n",
        "  os.mkdir(output_dir)\n",
        "model_to_save = modelBIG.module if hasattr(modelBIG, 'module') else modelBIG\n",
        "output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\n",
        "output_config_file = os.path.join(output_dir, CONFIG_NAME)\n",
        "# save model and model configs\n",
        "modelBIG.save_pretrained(output_dir)\n",
        "model_to_save.config.to_json_file(output_config_file)\n",
        "# save tokenizer\n",
        "tokenizerBIG.save_pretrained(output_dir)\n",
        "\n",
        "# 1.31 GB model for\n",
        "'''\n",
        "~ MEDIUM \n",
        " n_embd= 1024,\n",
        " n_head = 16\n",
        " n_layer = 24 \n",
        "'''\n",
        "'''\n",
        "The default small model: 474 MB\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'\\nThe default small model: 474 MB\\n'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y5GGbHM2BRGP"
      },
      "source": [
        "#SAVE SMALL\n",
        "\n",
        "from transformers import WEIGHTS_NAME, CONFIG_NAME\n",
        "import os\n",
        "output_dir = './model_bn_custom/'\n",
        "#or drive\n",
        "output_dir = '/content/drive/My Drive/gpt-bg/'\n",
        "\n",
        "# creating directory if it is not present\n",
        "if not os.path.exists(output_dir):\n",
        "  os.mkdir(output_dir)\n",
        "model_to_save = model.module if hasattr(model, 'module') else model\n",
        "output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\n",
        "output_config_file = os.path.join(output_dir, CONFIG_NAME)\n",
        "# save model and model configs\n",
        "model.save_pretrained(output_dir)\n",
        "model_to_save.config.to_json_file(output_config_file)\n",
        "# save tokenizer\n",
        "tokenizer.save_pretrained(output_dir)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WFM1DLd2j75D"
      },
      "source": [
        "# **LOAD MODEL**\n",
        "\n",
        "## Run after you have a saved model in your Drive."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-TOb-jg7Bhi2",
        "outputId": "c7d7590e-afe1-4c99-fd23-4baf7c28380b"
      },
      "source": [
        "#LOADMODEL  adjust which tokenizer etc.\n",
        "#SAVE ALSO THE DATASET! PICKLE - Can't pickle it, incompatible\n",
        "import tensorflow as tf\n",
        "from transformers import GPT2Config, TFGPT2LMHeadModel, GPT2Tokenizer\n",
        "\n",
        "load_dir = \"/content/1024/\"\n",
        "load_dir = '/content/drive/MyDrive/gpt2/' #@ TWENKID\n",
        "tokenizer2 = GPT2Tokenizer.from_pretrained(load_dir)\n",
        "model2 = TFGPT2LMHeadModel.from_pretrained(load_dir)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "All model checkpoint layers were used when initializing TFGPT2LMHeadModel.\n",
            "\n",
            "All the layers of TFGPT2LMHeadModel were initialized from the model checkpoint at /content/drive/MyDrive/gpt2/.\n",
            "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFGPT2LMHeadModel for predictions without further training.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PoAv8-IDxlYh"
      },
      "source": [
        "!ls /content/ab/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VCp-eIunlHcy",
        "outputId": "e548496e-18e5-40e8-8596-3ceebd34cef8"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<transformers.models.gpt2.modeling_tf_gpt2.TFGPT2LMHeadModel at 0x7f1d8a218760>"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iB3IB8P3w-b_"
      },
      "source": [
        "tokenizer2 = GPT2Tokenizer.from_pretrained(load_dir)"
      ],
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MNpD_GyEYjSo",
        "outputId": "5b5b8554-2596-4ef0-fcf0-49800d0ecc2c"
      },
      "source": [
        "ls '/content/drive/MyDrive/gpt2-medium/'"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "ls: cannot access '/content/drive/MyDrive/gpt2-medium/': No such file or directory\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZGw3M_Cht5vq",
        "outputId": "4b127e29-11c0-4bbb-b596-88ff37d6bfa1"
      },
      "source": [
        "from pathlib import Path\n",
        "import os\n",
        "import random\n",
        "\n",
        "# the folder 'text' contains all the files\n",
        "#paths = [str(x) for x in Path(\"./text/\").glob(\"**/*.txt\")]\n",
        "#paths = [str(x) for x in Path(\"./en/\").glob(\"**/*.txt\")]\n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/ab/UTF8/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/ab/2/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/sample_data/ab/UTF8/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")] \n",
        "print(len(paths))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "319\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kH-lpzHenLSi"
      },
      "source": [
        "# **TOKENIZE - ТОКЕНИЗЕ**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5SfqlMaUH-iX",
        "outputId": "29a4543c-dcb0-4e2b-9572-e5a4901c0c7c"
      },
      "source": [
        "#TOKENIZE - Only part of the Dataset for now - after loading AFTER LOADING  ... tokenizer2\n",
        "#set devider to allow for faster epoch iterations\n",
        "divider = 1 #2 #1 #4 #1 # #2 #1 #5, 3 \n",
        "shifter = 0 #0 #0  #start not from the first\n",
        "iReverse = 0 #1 #0 #1 # 1 # reverse the dir list\n",
        "windowmax = 150000\n",
        "location = 0.0 #beginning, 1.0 - ending\n",
        "iFromTheEnd = 0\n",
        "iPaddingEnd = 0 #500 # 1500 #skip final notes, book signature etc.\n",
        "\n",
        "windowmax = 200000 # 100000 #120000 #80000 #40000 #50000 #100000 #100000 #smaller window, reverse\n",
        "#windowmax = 100000 #from 10.7.2021 --> more iterations at random places\n",
        "#windowmax = 100000 #22.7.2021\n",
        "#windowmax = 200000 \n",
        "#windowmax = 100000 \n",
        "windowmax = 200000 #75000 \n",
        "iReverse = 0 #1 #0 #1 #0 #1\n",
        "bRandomLocation = False #True \n",
        "startPos = 0\n",
        "start = 0\n",
        "bShiftOneStep = True #start from windowmax # 4.7.2021\n",
        "\n",
        "from pathlib import Path\n",
        "import os\n",
        "import random\n",
        "\n",
        "# the folder 'text' contains all the files\n",
        "#paths = [str(x) for x in Path(\"./text/\").glob(\"**/*.txt\")]\n",
        "#paths = [str(x) for x in Path(\"./en/\").glob(\"**/*.txt\")]\n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/ab/UTF8/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/ab/2/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/sample_data/ab/UTF8/\").glob(\"*.txt\")] \n",
        "paths = [str(x) for x in Path(\"/content/ab/\").glob(\"*.txt\")] \n",
        "#paths = [str(x) for x in Path(\"/content/a\").glob(\"*.txt\")]  #check one by one\n",
        "\n",
        "#ADD big twice: for second sampling, #22-7-2021\n",
        "additional = []\n",
        "for filename in paths:\n",
        "   with open(filename, \"r\", encoding='utf-8') as f:   \n",
        "     x = f.read()\n",
        "     #if (len(x)>windowmax*8): #if (len(x)>windowmax):\n",
        "     #if (len(x)>windowmax*1.1) and (len(x)<windowmax*6): # and (len(x)<windowmax*1): #if (len(x)>windowmax):\n",
        "     #if ((len(x)>windowmax*1.1) and (len(x)<windowmax*6)) or (len(x)>1100000): # and (len(x)<windowmax*1): #if (len(x)>windowmax):\n",
        "     if ((len(x)>windowmax*1.5)): # and (len(x)<windowmax*6)) or (len(x)>1100000): # and (len(x)<windowmax*1): #if (len(x)>windowmax):\n",
        "        additional.append(filename)\n",
        "\n",
        "defLen = len(paths)\n",
        "paths += additional\n",
        "\n",
        "print(paths)\n",
        "selected_paths = []\n",
        "items = []\n",
        "k = shifter # 0 - 1.0\n",
        "pathslen = len(paths)\n",
        "#if reverse...\n",
        "if iReverse!=0: paths.reverse()\n",
        "maxfiles = 500*divider #change >\n",
        "top = 0\n",
        "files = {}\n",
        "step = 0\n",
        "#for filename in paths: \n",
        "for filename in paths: \n",
        "  if k >= pathslen: break\n",
        "  if k > maxfiles: break\n",
        "  with open(filename, \"r\", encoding='utf-8') as f:        \n",
        "    if k%divider != 0: k+=1; print(\"CONTINUE\",k); continue #1/divider\n",
        "    #if k%divider >1: continue #1/divider\n",
        "    x = f.read()\n",
        "    if (len(x) > windowmax):\n",
        "        #if len(x)>=2*windowmax: #4.7.2021\n",
        "        if len(x)>=windowmax: #6.7.2021\n",
        "          if bShiftOneStep:\n",
        "            rat = windowmax/len(x)\n",
        "            if (rat < 0): rat = 0.0            \n",
        "            #fl = max(random.random(),rat) #- rat) #, len(x)-windowmax) #>0.0 0.9 # - len(x)*0.1            \n",
        "            #start = windowmax + fl*len(x)\n",
        "            #start = windowmax//2 + fl*len(x) 5.7\n",
        "            fl = random.random()\n",
        "            if (fl > (1.0 - rat)): fl = 1.0-rat #to fill the windowmax - check the debug output\n",
        "            start = fl*len(x) #6.7\n",
        "            print(\"bShiftOneStep!\")\n",
        "            print(\"start=\",start)\n",
        "          else: start = startPos; print(\"start(default)=\",start)  #default 0           \n",
        "        if bRandomLocation:\n",
        "          fl = random.random()\n",
        "          start = int(fl)*len(x)\n",
        "          items.append(x[start:min(start+windowmax-iPaddingEnd, len(x)-iPaddingEnd)])\n",
        "          step = min(start+windowmax-iPaddingEnd, len(x)-iPaddingEnd - start)\n",
        "        else:\n",
        "             #items.append(x[0:windowmax]) #that appends both the beginning and the end (not that's bad)\n",
        "            if iFromTheEnd == 0:\n",
        "                                start = startPos #may be different than 0\n",
        "                                #items.append(x[0:min(windowmax, len(x))])\n",
        "                                items.append(x[start:min(windowmax, len(x))])\n",
        "                                step = min(windowmax, len(x))\n",
        "            else: #last ...\n",
        "                 items.append(x[-windowmax:-iPaddingEnd]) #last windowmax - padding (skip technical etc.)\n",
        "                 #items.append(x[0:min(windowmax,len(x))]) #last windowmax - padding (skip technical etc.)\n",
        "                 step = windowmax - iPaddingEnd #DEBUG sizes etc.!\n",
        "    else:\n",
        "         items.append(x)   # the whole\n",
        "         step = len(x)\n",
        "        \n",
        "    files[paths[k]]=top\n",
        "    top+=step\n",
        "    selected_paths.append(filename)\n",
        "    print(str(k) + \" /150=\" + str(top/150) + str(top) + \"ends @ \" + paths[k]) # + \" ends at \", )\n",
        "    k+=1\n",
        "    #print(x)\n",
        "  #single_string += x + tokenizerBIG.eos_token\n",
        "  items.append(tokenizer2.eos_token)\n",
        "single_string = ''.join(items)\n",
        "print(\"LEN: single_string: \", len(single_string))\n",
        "\n",
        "if (len(single_string)<999): print(\"\\n===============\\n\"+single_string[:100])\n",
        "string_tokenized = tokenizer2.encode(single_string) #, truncation=True) #, max_length=100)\n",
        "if (len(string_tokenized)<200000): print(string_tokenized[:10])\n",
        "print(\"LEN: tokenized: \", len(string_tokenized))\n",
        "print(\"Files without doubling\", defLen)\n",
        "print(\"Files With doubling:\", len(paths))\n",
        "#**DATASET**\n",
        "\n",
        "#ENSURE there's no NONE elements! Caused Typespec etc. error? #25.6.2021\n",
        "#[tokenizer2.eos_token_id if x==None else x for x in string_tokenized]\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['/content/ab/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt', '/content/ab/gender.txt', '/content/ab/John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', '/content/ab/Aleko-Schweizariya.txt', '/content/ab/Lev_Tolstoj_Vojna_i_Mir-KRAYA_NARODITE.txt', '/content/ab/Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', '/content/ab/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt', '/content/ab/predstava.txt', '/content/ab/kakvomu_notes_e.txt', '/content/ab/kreativen.txt', '/content/ab/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt', '/content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', '/content/ab/chlen.txt', '/content/ab/yunaka-i-taz.txt', '/content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', '/content/ab/Eshil_1.txt', '/content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', '/content/ab/Volkswagen.txt', '/content/ab/Biblija_-_1435-b.txt', '/content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt', '/content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', '/content/ab/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', '/content/ab/Robin-S-Sharma_-_Monahyt_kojto_prodade_svoeto_ferari_-_Duhovna_pritcha_za_tova_kak_da_osyshtestvite_mechtite_si_i_da_postignete_sydbata_si_-_5236-b.txt', '/content/ab/Hristo Botev -  - . Strannik - 3237.txt', '/content/ab/Hristo Botev -  - . Borba - 3236.txt', '/content/ab/Anglijski_narodni_prikazki_-_3637-b.txt', '/content/ab/fursov-schwab.txt', '/content/ab/robstvo.txt', '/content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt', '/content/ab/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt', '/content/ab/comment-nerds-2-nerds-128-clean1.txt', '/content/ab/Hristo Botev -  - . Majtse si - 3227.txt', '/content/ab/hardware2.txt', '/content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt', '/content/ab/mainstream_media.txt', '/content/ab/Charles-Bukowski_-_Goreshta_muzika_-_6175-b.txt', '/content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', '/content/ab/proc.txt', '/content/ab/Platon_-_Dyrzhavata_-_9825-b.txt', '/content/ab/Ivan Vazov -  - . Avtobiografija - 3756.txt', '/content/ab/polit-komentari-1.txt', '/content/ab/negativen-otbor.txt', '/content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt', '/content/ab/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt', '/content/ab/Obekti-11-7-2021.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', '/content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt', '/content/ab/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt', '/content/ab/Talev-svetilnik-2-chast.txt', '/content/ab/Matthew-Quick_-_Narychnik_na_optimista_-_9307-b.txt', '/content/ab/stari.txt', '/content/ab/computers-Linus-gt-1b.txt', '/content/ab/Hristo Botev - V - . Retsept - 3678.txt', '/content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', '/content/ab/Hristo Botev -  - . Hajduti - 3232.txt', '/content/ab/fursov-chechoslovakiya-1968-i-dr-rusia.txt', '/content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', '/content/ab/10-noemvri.txt', '/content/ab/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt', '/content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt', '/content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', '/content/ab/tupanasov.txt', '/content/ab/baladi.txt', '/content/ab/podvizi-akro-mezo-stih.txt', '/content/ab/evro-put.txt', '/content/ab/sreshtu-chuzhdico.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt', '/content/ab/Alexa-Riley_-_Devitsata_kauboj_-_9151-b.txt', '/content/ab/terminator-i-herkules.txt', '/content/ab/Izkustven_RAZUM_UIR_bez-http_bez_Sudurjanie.txt', '/content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', '/content/ab/hardware1.txt', '/content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt', '/content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', '/content/ab/krivorazbranata.txt', '/content/ab/upaduk_dzbe.txt', '/content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt', '/content/ab/politika-26-7-2021.txt', '/content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt', '/content/ab/igri-za.txt', '/content/ab/vyarata_i_svobodata.txt', '/content/ab/Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt', '/content/ab/Voyna_Kum_Vechnata_Voyna.txt', '/content/ab/EIM-B_br-8-16-li_bez-Chovekut-i-Misleshtata-mashina.txt', '/content/ab/Mitut-Ot-Kontrapunkti-200K.txt', '/content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt', '/content/ab/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt', '/content/ab/Hristo Botev -  - . Zadade se oblak temen - 3245.txt', '/content/ab/fursov-SSSR-dnesh-bezkulturn.txt', '/content/ab/Terminator-2_6-7-2021.txt', '/content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt', '/content/ab/Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', '/content/ab/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt', '/content/ab/vizata-nachalo.txt', '/content/ab/fursov-korona-apokalipsis.txt', '/content/ab/istoriya-germaniya-26-7.2021.txt', '/content/ab/Hristo Botev -  - . Mojata molitva - 3244.txt', '/content/ab/bezdushen.txt', '/content/ab/sistema-narodnoto.txt', '/content/ab/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt', '/content/ab/Hristo Botev - Vnbe - . Petrushan - 3638.txt', '/content/ab/George-Orwell_-_1984_-_53-b.txt', '/content/ab/hardware0.txt', '/content/ab/computers-Linus-gtplus-2b.txt', '/content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', '/content/ab/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', '/content/ab/Hristo Botev -  - . Na proshtavane - 3235.txt', '/content/ab/belezhki-istinata.txt', '/content/ab/Hristo Botev -  - . Delba - 3230.txt', '/content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt', '/content/ab/pred3.txt', '/content/ab/bgmama-computers.txt', '/content/ab/fursov-myusulmani.txt', '/content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', '/content/ab/fursov-iztochna-evropa-silnite-na-denya.txt', '/content/ab/ivo-hristov-pepelishte.txt', '/content/ab/Mark-Twain_-_Pisma_ot_Zemjata._Avtobiografija_-_4984-b.txt', '/content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', '/content/ab/fursov-vododel.txt', '/content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt', '/content/ab/manipulaciya.txt', '/content/ab/stranichni.txt', '/content/ab/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', '/content/ab/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt', '/content/ab/DZBE_ponyatiya_7-2020_eim-33.txt', '/content/ab/igrach-mekichar.txt', '/content/ab/fursov-nato-1938-Myunhen.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt', '/content/ab/ezikovi-raboti.txt', '/content/ab/fursov-fefelov-tronove-kultura-na-smeha.txt', '/content/ab/pisma-27-7-2021.txt', '/content/ab/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt', '/content/ab/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt', '/content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt', '/content/ab/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt', '/content/ab/radost.txt', '/content/ab/agi-7-2019.txt', '/content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', '/content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', '/content/ab/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt', '/content/ab/Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', '/content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt', '/content/ab/democratius_stamboliyski.txt', '/content/ab/Mitut.txt', '/content/ab/Srednovekovni_kitajski_noveli_-_8142-b.txt', '/content/ab/computers-razni-26-7-2021.txt', '/content/ab/eseto-sasho.txt', '/content/ab/750ti.txt', '/content/ab/AMD-Intel.txt', '/content/ab/zaklyuchena-poetry.txt', '/content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt', '/content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', '/content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', '/content/ab/igrachi-yunaci.txt', '/content/ab/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt', '/content/ab/Sergej_Minaev_-_Duhless_-_9430-b.txt', '/content/ab/zmee.txt', '/content/ab/Ada_12-10_E--ot-GLAVA-23.txt', '/content/ab/trus.txt', '/content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', '/content/ab/svobodna-evropa.txt', '/content/ab/Hristo Botev -  - . Zashto ne sym_... - 3241.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/Hristo Botev - V - . Gatanki - 3685.txt', '/content/ab/Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt', '/content/ab/Hristo Botev -  - . V mehanata - 3243.txt', '/content/ab/seksut.txt', '/content/ab/pisma-do-irina-malko-red.txt', '/content/ab/Pesen_za_nibelungite_-_6576-b.txt', '/content/ab/fursov-stalin-nisheta.txt', '/content/ab/fursov-skriti-prichini-vtora-svetovna.txt', '/content/ab/Hristo Botev - V - . Vazhno neshto - 3681.txt', '/content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt', '/content/ab/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt', '/content/ab/toj-vyara.txt', '/content/ab/Talev-svetilnik-nachalo.txt', '/content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt', '/content/ab/dve-narechiya.txt', '/content/ab/Hristo Botev - V - . Kazano - 3677.txt', '/content/ab/podaruk.txt', '/content/ab/Istinata.txt', '/content/ab/Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt', '/content/ab/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt', '/content/ab/bivalica.txt', '/content/ab/Hristo Botev -  - . Nej - 3234.txt', '/content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt', '/content/ab/SmartCen.txt', '/content/ab/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt', '/content/ab/RISC-CISC.txt', '/content/ab/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt', '/content/ab/zapadnichestvo.txt', '/content/ab/Skandinavska_mitologija_-_8141-b.txt', '/content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt', '/content/ab/galina_tx_slavina_memory_2004_edit-9-10-2016_Clean-11-7-2021.txt', '/content/ab/fursov-neoburjoaziya.txt', '/content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt', '/content/ab/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt', '/content/ab/Gore-Vidal_-_Julian_-_7517-b.txt', '/content/ab/youtube-26-7-2021.txt', '/content/ab/Hristo Botev - V - . Shte se nameri li_ - 3686.txt', '/content/ab/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt', '/content/ab/Hristo Botev - V - . Shte vidite - 3687.txt', '/content/ab/hristo-bozhinov.txt', '/content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', '/content/ab/voyna.txt', '/content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', '/content/ab/darenie.txt', '/content/ab/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt', '/content/ab/Hristo Botev - V - . Dolapyt - 3670.txt', '/content/ab/Hristo Botev - V - . Videno - 3676.txt', '/content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt', '/content/ab/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', '/content/ab/dimov-tyutyun.txt', '/content/ab/fursov-otrepki-ubiici-gorbachev.txt', '/content/ab/alyosha2.txt', '/content/ab/Hristo Botev -  - . Kym brata si - 3228.txt', '/content/ab/doktoratut.txt', '/content/ab/goden.txt', '/content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt', '/content/ab/eksploataiya.txt', '/content/ab/pechalba.txt', '/content/ab/Hristo Botev - V - . Synovnik - 3680.txt', '/content/ab/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt', '/content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt', '/content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt', '/content/ab/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', '/content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', '/content/ab/Bogomil_Rajnov_-_Umiraj_samo_v_kraen_sluchaj.txt', '/content/ab/fursov-multikulturalizum.txt', '/content/ab/Protesti2020.txt', '/content/ab/john-1.txt', '/content/ab/Piers-Anthony_-_Zov_za_zavryshtane_-_1236-b.txt', '/content/ab/EIM-smetach-B-BEZ-ZAGLAVIYA-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/dod.txt', '/content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt', '/content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', '/content/ab/William-Saroyan_-_Neshto_kato_nozh_neshto_kato_tsvete_izobshto_kato_nishto_na_sveta_-_razkazi_-_186-b.txt', '/content/ab/igrach-na.txt', '/content/ab/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt', '/content/ab/Hristo Botev - V - . Epitafii - 3679.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', '/content/ab/ritum-solo-some-blog-26-7-2021.txt', '/content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', '/content/ab/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', '/content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt', '/content/ab/fursov-stalin-smurt.txt', '/content/ab/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt', '/content/ab/smyana-na-sistemata.txt', '/content/ab/komunalen.txt', '/content/ab/polagane-na-trud.txt', '/content/ab/Branislav_Nushich_-_Avtobiografija_-_1713-b.txt', '/content/ab/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt', '/content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', '/content/ab/Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt', '/content/ab/iskam-si-dvata-paunda.txt', '/content/ab/vreme-prognoza-26-7-2021.txt', '/content/ab/igrachut.txt', '/content/ab/Tosh_Chovekut-i-Misleshtata_Mashina.txt', '/content/ab/Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt', '/content/ab/ritum.txt', '/content/ab/Hristo Botev -  - . Elegija - 3229.txt', '/content/ab/fursov-evropa-nichiy-dom-marx.txt', '/content/ab/Musketari-chast.txt', '/content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt', '/content/ab/alyosha1.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/fursov-USA-SASHT-nishteta-bednost.txt', '/content/ab/Hristo Botev -  - . Poslanie - 3240.txt', '/content/ab/fursov-soros.txt', '/content/ab/Andre-Agassi_-_AUGMENTED-Otkrito_-_Avtobiografija_-_10291-b-EDIT-2.txt', '/content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt', '/content/ab/Alexandre-Jardin_-_Zebrata_-_6966-b.txt', '/content/ab/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt', '/content/ab/Hristo Botev -  - . Patriot - 3239.txt', '/content/ab/AI-ivo-hristov.txt', '/content/ab/Analiz-na-smisula.txt', '/content/ab/john-2.txt', '/content/ab/Pavel_Vezhinov-Duh-na-bademi.txt', '/content/ab/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt', '/content/ab/computers-Ardes-7-7-2021.txt', '/content/ab/Hristo Botev - V - . Tova vi chaka! - 3668.txt', '/content/ab/fursov-kitay.txt', '/content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', '/content/ab/GamersMeld-2020-g-news-3080-Ti-AMD-Big-Navi-edited.txt', '/content/ab/kakvomu.txt', '/content/ab/dari-dobro-eko-soc.txt', '/content/ab/Balzak-chast.txt', '/content/ab/Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt', '/content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', '/content/ab/PCs-1.txt', '/content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', '/content/ab/uir-pisma.txt', '/content/ab/_-  - 2. Biblija - Nov zavet - 2256.txt', '/content/ab/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt', '/content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', '/content/ab/Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt', '/content/ab/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt', '/content/ab/Terminator-1_6-7-2021.txt', '/content/ab/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt', '/content/ab/kopiraneto-EIM-1-bez-imena.txt', '/content/ab/necessary_illusions.txt', '/content/ab/tennis-1.txt', '/content/ab/Mitut-ot-Zashto-mujkiyat-chlen.txt', '/content/ab/Hristo Botev - V - . Podslushano - 3675.txt', '/content/ab/igrachut-i-lyubovta.txt', '/content/ab/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt', '/content/ab/fursov-brexit-trump.txt', '/content/ab/fursov-korona.txt', '/content/ab/rechnik.txt', '/content/ab/Hristo Botev -  - . Pristanala - 3233.txt', '/content/ab/gnyav.txt', '/content/ab/Hristo Botev - Vnbe - . Namesto programa - 3635.txt', '/content/ab/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt', '/content/ab/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', '/content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt', '/content/ab/prehvur-vidyo.txt', '/content/ab/Hristo Botev - V - . Fabrika za popove - 3673.txt', '/content/ab/Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', '/content/ab/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt', '/content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', '/content/ab/Omir_-_Odiseja_-_6123-b.txt', '/content/ab/dogonvashto.txt', '/content/ab/Hristo Botev - V - . Politicheska zima - 3666.txt', '/content/ab/seks-glavna.txt', '/content/ab/pred4.txt', '/content/ab/135-godini.txt', '/content/ab/telefonni-suveti.txt', '/content/ab/cinichna.txt', '/content/ab/razumir-br-4.txt', '/content/ab/Hristo Botev -  - . Gergxovden - 3238.txt', '/content/ab/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt', '/content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', '/content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt', '/content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', '/content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', '/content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', '/content/ab/Biblija_-_1435-b.txt', '/content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt', '/content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', '/content/ab/Robin-S-Sharma_-_Monahyt_kojto_prodade_svoeto_ferari_-_Duhovna_pritcha_za_tova_kak_da_osyshtestvite_mechtite_si_i_da_postignete_sydbata_si_-_5236-b.txt', '/content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt', '/content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt', '/content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', '/content/ab/Platon_-_Dyrzhavata_-_9825-b.txt', '/content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', '/content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt', '/content/ab/Matthew-Quick_-_Narychnik_na_optimista_-_9307-b.txt', '/content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', '/content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', '/content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt', '/content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', '/content/ab/Izkustven_RAZUM_UIR_bez-http_bez_Sudurjanie.txt', '/content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', '/content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt', '/content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', '/content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt', '/content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt', '/content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt', '/content/ab/Terminator-2_6-7-2021.txt', '/content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt', '/content/ab/Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', '/content/ab/George-Orwell_-_1984_-_53-b.txt', '/content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', '/content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt', '/content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', '/content/ab/Mark-Twain_-_Pisma_ot_Zemjata._Avtobiografija_-_4984-b.txt', '/content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', '/content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt', '/content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt', '/content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', '/content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', '/content/ab/Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', '/content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt', '/content/ab/Mitut.txt', '/content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt', '/content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', '/content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', '/content/ab/Sergej_Minaev_-_Duhless_-_9430-b.txt', '/content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/Pesen_za_nibelungite_-_6576-b.txt', '/content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt', '/content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt', '/content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt', '/content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt', '/content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt', '/content/ab/Gore-Vidal_-_Julian_-_7517-b.txt', '/content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', '/content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt', '/content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt', '/content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt', '/content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', '/content/ab/Bogomil_Rajnov_-_Umiraj_samo_v_kraen_sluchaj.txt', '/content/ab/Piers-Anthony_-_Zov_za_zavryshtane_-_1236-b.txt', '/content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/William-Saroyan_-_Neshto_kato_nozh_neshto_kato_tsvete_izobshto_kato_nishto_na_sveta_-_razkazi_-_186-b.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', '/content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', '/content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt', '/content/ab/Branislav_Nushich_-_Avtobiografija_-_1713-b.txt', '/content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', '/content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/Andre-Agassi_-_AUGMENTED-Otkrito_-_Avtobiografija_-_10291-b-EDIT-2.txt', '/content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt', '/content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', '/content/ab/kakvomu.txt', '/content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', '/content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', '/content/ab/_-  - 2. Biblija - Nov zavet - 2256.txt', '/content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt', '/content/ab/Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt', '/content/ab/Terminator-1_6-7-2021.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', '/content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt', '/content/ab/Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', '/content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', '/content/ab/Omir_-_Odiseja_-_6123-b.txt', '/content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', '/content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt']\n",
            "0 /150=117.1133333333333317567ends @ /content/ab/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt\n",
            "1 /150=152.7066666666666822906ends @ /content/ab/gender.txt\n",
            "bShiftOneStep!\n",
            "start= 4103.000000000011\n",
            "2 /150=1486.04222906ends @ /content/ab/John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt\n",
            "3 /150=1604.68240702ends @ /content/ab/Aleko-Schweizariya.txt\n",
            "4 /150=2281.0866666666666342163ends @ /content/ab/Lev_Tolstoj_Vojna_i_Mir-KRAYA_NARODITE.txt\n",
            "5 /150=3218.8133333333335482822ends @ /content/ab/Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt\n",
            "6 /150=3226.1266666666666483919ends @ /content/ab/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt\n",
            "7 /150=3385.4733333333334507821ends @ /content/ab/predstava.txt\n",
            "8 /150=3932.1866666666665589828ends @ /content/ab/kakvomu_notes_e.txt\n",
            "9 /150=4493.546666666667674032ends @ /content/ab/kreativen.txt\n",
            "10 /150=4570.673333333333685601ends @ /content/ab/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt\n",
            "bShiftOneStep!\n",
            "start= 213814.0\n",
            "11 /150=5904.006666666667885601ends @ /content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt\n",
            "12 /150=5989.953333333333898493ends @ /content/ab/chlen.txt\n",
            "13 /150=6144.886666666666921733ends @ /content/ab/yunaka-i-taz.txt\n",
            "bShiftOneStep!\n",
            "start= 1062666.2734661398\n",
            "14 /150=7478.221121733ends @ /content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            "bShiftOneStep!\n",
            "start= 554847.6384868245\n",
            "15 /150=8811.5533333333331321733ends @ /content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt\n",
            "16 /150=10125.2533333333341518788ends @ /content/ab/Eshil_1.txt\n",
            "bShiftOneStep!\n",
            "start= 576526.0\n",
            "17 /150=11458.5866666666661718788ends @ /content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt\n",
            "18 /150=11515.5066666666661727326ends @ /content/ab/Volkswagen.txt\n",
            "bShiftOneStep!\n",
            "start= 2745364.6591726267\n",
            "19 /150=12848.841927326ends @ /content/ab/Biblija_-_1435-b.txt\n",
            "bShiftOneStep!\n",
            "start= 139515.2192316534\n",
            "20 /150=14182.1733333333342127326ends @ /content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            "bShiftOneStep!\n",
            "start= 222205.0\n",
            "21 /150=15515.5066666666662327326ends @ /content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt\n",
            "22 /150=16565.5733333333342484836ends @ /content/ab/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            "bShiftOneStep!\n",
            "start= 128613.00000000001\n",
            "23 /150=17898.9066666666662684836ends @ /content/ab/Robin-S-Sharma_-_Monahyt_kojto_prodade_svoeto_ferari_-_Duhovna_pritcha_za_tova_kak_da_osyshtestvite_mechtite_si_i_da_postignete_sydbata_si_-_5236-b.txt\n",
            "24 /150=17907.62686140ends @ /content/ab/Hristo Botev -  - . Strannik - 3237.txt\n",
            "25 /150=17919.473333333332687921ends @ /content/ab/Hristo Botev -  - . Borba - 3236.txt\n",
            "bShiftOneStep!\n",
            "start= 16561.000000000007\n",
            "26 /150=19252.8066666666672887921ends @ /content/ab/Anglijski_narodni_prikazki_-_3637-b.txt\n",
            "27 /150=19536.0066666666682930401ends @ /content/ab/fursov-schwab.txt\n",
            "28 /150=19607.7933333333352941169ends @ /content/ab/robstvo.txt\n",
            "bShiftOneStep!\n",
            "start= 275791.9672639088\n",
            "29 /150=20941.1266666666673141169ends @ /content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            "30 /150=21000.7666666666663150115ends @ /content/ab/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt\n",
            "31 /150=21139.0333333333333170855ends @ /content/ab/comment-nerds-2-nerds-128-clean1.txt\n",
            "32 /150=21146.2466666666663171937ends @ /content/ab/Hristo Botev -  - . Majtse si - 3227.txt\n",
            "33 /150=21911.4533333333353286718ends @ /content/ab/hardware2.txt\n",
            "bShiftOneStep!\n",
            "start= 384211.0\n",
            "34 /150=23244.7866666666673486718ends @ /content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            "35 /150=23430.4466666666673514567ends @ /content/ab/mainstream_media.txt\n",
            "bShiftOneStep!\n",
            "start= 54440.999999999985\n",
            "36 /150=24763.783714567ends @ /content/ab/Charles-Bukowski_-_Goreshta_muzika_-_6175-b.txt\n",
            "bShiftOneStep!\n",
            "start= 165985.21005896197\n",
            "37 /150=26097.1133333333353914567ends @ /content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            "38 /150=26491.7066666666653973756ends @ /content/ab/proc.txt\n",
            "bShiftOneStep!\n",
            "start= 286448.24732633715\n",
            "39 /150=27825.044173756ends @ /content/ab/Platon_-_Dyrzhavata_-_9825-b.txt\n",
            "40 /150=27835.0533333333334175258ends @ /content/ab/Ivan Vazov -  - . Avtobiografija - 3756.txt\n",
            "41 /150=27894.2933333333354184144ends @ /content/ab/polit-komentari-1.txt\n",
            "42 /150=28163.3933333333334224509ends @ /content/ab/negativen-otbor.txt\n",
            "bShiftOneStep!\n",
            "start= 257304.0\n",
            "43 /150=29496.7266666666664424509ends @ /content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            "44 /150=29571.584435737ends @ /content/ab/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt\n",
            "45 /150=30162.3066666666674524346ends @ /content/ab/Obekti-11-7-2021.txt\n",
            "bShiftOneStep!\n",
            "start= 528575.0\n",
            "46 /150=31495.644724346ends @ /content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt\n",
            "bShiftOneStep!\n",
            "start= 885842.0\n",
            "47 /150=32828.9733333333354924346ends @ /content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            "48 /150=32874.1133333333354931117ends @ /content/ab/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt\n",
            "49 /150=33717.345057601ends @ /content/ab/Talev-svetilnik-2-chast.txt\n",
            "bShiftOneStep!\n",
            "start= 181376.00000000003\n",
            "50 /150=35050.673333333335257601ends @ /content/ab/Matthew-Quick_-_Narychnik_na_optimista_-_9307-b.txt\n",
            "51 /150=35069.285260392ends @ /content/ab/stari.txt\n",
            "52 /150=35142.326666666675271349ends @ /content/ab/computers-Linus-gt-1b.txt\n",
            "53 /150=35144.553333333345271683ends @ /content/ab/Hristo Botev - V - . Retsept - 3678.txt\n",
            "bShiftOneStep!\n",
            "start= 160706.61624999886\n",
            "54 /150=36477.8866666666655471683ends @ /content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            "55 /150=36507.233333333335476085ends @ /content/ab/Hristo Botev -  - . Hajduti - 3232.txt\n",
            "56 /150=36631.873333333345494781ends @ /content/ab/fursov-chechoslovakiya-1968-i-dr-rusia.txt\n",
            "bShiftOneStep!\n",
            "start= 600187.0\n",
            "57 /150=37965.2066666666655694781ends @ /content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt\n",
            "58 /150=38121.3466666666655718202ends @ /content/ab/10-noemvri.txt\n",
            "59 /150=38131.745719761ends @ /content/ab/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt\n",
            "bShiftOneStep!\n",
            "start= 366073.67790595634\n",
            "60 /150=39465.0733333333345919761ends @ /content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "bShiftOneStep!\n",
            "start= 124124.99999999999\n",
            "61 /150=40798.406666666676119761ends @ /content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            "62 /150=41070.506666666676160576ends @ /content/ab/tupanasov.txt\n",
            "63 /150=41091.226163683ends @ /content/ab/baladi.txt\n",
            "64 /150=41097.7933333333356164669ends @ /content/ab/podvizi-akro-mezo-stih.txt\n",
            "65 /150=41105.2533333333346165788ends @ /content/ab/evro-put.txt\n",
            "66 /150=41521.4333333333346228215ends @ /content/ab/sreshtu-chuzhdico.txt\n",
            "bShiftOneStep!\n",
            "start= 14844.000000000011\n",
            "67 /150=42854.766666666676428215ends @ /content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt\n",
            "68 /150=43557.493333333336533624ends @ /content/ab/Alexa-Riley_-_Devitsata_kauboj_-_9151-b.txt\n",
            "69 /150=43585.06537750ends @ /content/ab/terminator-i-herkules.txt\n",
            "bShiftOneStep!\n",
            "start= 858155.349498305\n",
            "70 /150=44918.3333333333366737750ends @ /content/ab/Izkustven_RAZUM_UIR_bez-http_bez_Sudurjanie.txt\n",
            "bShiftOneStep!\n",
            "start= 267805.0\n",
            "71 /150=46251.6666666666646937750ends @ /content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt\n",
            "72 /150=46957.006666666677043551ends @ /content/ab/hardware1.txt\n",
            "bShiftOneStep!\n",
            "start= 15513.221729543953\n",
            "73 /150=48290.347243551ends @ /content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            "bShiftOneStep!\n",
            "start= 97542.72208113465\n",
            "74 /150=49623.673333333337443551ends @ /content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            "75 /150=49958.7466666666667493812ends @ /content/ab/krivorazbranata.txt\n",
            "76 /150=50421.633333333337563245ends @ /content/ab/upaduk_dzbe.txt\n",
            "bShiftOneStep!\n",
            "start= 111174.1024121926\n",
            "77 /150=51754.966666666677763245ends @ /content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            "78 /150=52248.827837323ends @ /content/ab/politika-26-7-2021.txt\n",
            "bShiftOneStep!\n",
            "start= 669291.017199297\n",
            "79 /150=53582.1533333333358037323ends @ /content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            "80 /150=53640.713333333338046107ends @ /content/ab/igri-za.txt\n",
            "81 /150=53872.2466666666668080837ends @ /content/ab/vyarata_i_svobodata.txt\n",
            "82 /150=54163.38124495ends @ /content/ab/Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt\n",
            "83 /150=54819.3066666666648222896ends @ /content/ab/Voyna_Kum_Vechnata_Voyna.txt\n",
            "bShiftOneStep!\n",
            "start= 92168.00000000001\n",
            "84 /150=56152.648422896ends @ /content/ab/EIM-B_br-8-16-li_bez-Chovekut-i-Misleshtata-mashina.txt\n",
            "85 /150=57334.626666666668600194ends @ /content/ab/Mitut-Ot-Kontrapunkti-200K.txt\n",
            "bShiftOneStep!\n",
            "start= 420754.0\n",
            "86 /150=58667.968800194ends @ /content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            "87 /150=58700.506666666678805076ends @ /content/ab/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt\n",
            "88 /150=58711.133333333338806670ends @ /content/ab/Hristo Botev -  - . Zadade se oblak temen - 3245.txt\n",
            "89 /150=58913.0133333333368836952ends @ /content/ab/fursov-SSSR-dnesh-bezkulturn.txt\n",
            "bShiftOneStep!\n",
            "start= 163144.42086454196\n",
            "90 /150=60246.3466666666659036952ends @ /content/ab/Terminator-2_6-7-2021.txt\n",
            "bShiftOneStep!\n",
            "start= 136239.2917059022\n",
            "91 /150=61579.689236952ends @ /content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt\n",
            "bShiftOneStep!\n",
            "start= 274739.0\n",
            "92 /150=62913.0133333333369436952ends @ /content/ab/Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            "93 /150=62984.786666666679447718ends @ /content/ab/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt\n",
            "94 /150=63014.773333333339452216ends @ /content/ab/vizata-nachalo.txt\n",
            "95 /150=63074.593333333339461189ends @ /content/ab/fursov-korona-apokalipsis.txt\n",
            "96 /150=63113.069466959ends @ /content/ab/istoriya-germaniya-26-7.2021.txt\n",
            "97 /150=63120.053333333349468008ends @ /content/ab/Hristo Botev -  - . Mojata molitva - 3244.txt\n",
            "98 /150=63162.466666666679474370ends @ /content/ab/bezdushen.txt\n",
            "99 /150=63170.429475563ends @ /content/ab/sistema-narodnoto.txt\n",
            "100 /150=63180.533333333339477080ends @ /content/ab/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt\n",
            "101 /150=63218.289482742ends @ /content/ab/Hristo Botev - Vnbe - . Petrushan - 3638.txt\n",
            "bShiftOneStep!\n",
            "start= 352767.48340408347\n",
            "102 /150=64551.6133333333359682742ends @ /content/ab/George-Orwell_-_1984_-_53-b.txt\n",
            "103 /150=65513.646666666679827047ends @ /content/ab/hardware0.txt\n",
            "104 /150=65548.926666666679832339ends @ /content/ab/computers-Linus-gtplus-2b.txt\n",
            "bShiftOneStep!\n",
            "start= 109638.27659349416\n",
            "105 /150=66882.2610032339ends @ /content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            "bShiftOneStep!\n",
            "start= 48329.00000000001\n",
            "106 /150=68215.5933333333410232339ends @ /content/ab/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "107 /150=68238.5666666666710235785ends @ /content/ab/Hristo Botev -  - . Na proshtavane - 3235.txt\n",
            "108 /150=68320.7266666666710248109ends @ /content/ab/belezhki-istinata.txt\n",
            "109 /150=68325.5266666666710248829ends @ /content/ab/Hristo Botev -  - . Delba - 3230.txt\n",
            "bShiftOneStep!\n",
            "start= 98868.67162903711\n",
            "110 /150=69658.8610448829ends @ /content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            "111 /150=70256.6933333333310538504ends @ /content/ab/pred3.txt\n",
            "112 /150=70339.2533333333310550888ends @ /content/ab/bgmama-computers.txt\n",
            "113 /150=70463.8133333333410569572ends @ /content/ab/fursov-myusulmani.txt\n",
            "bShiftOneStep!\n",
            "start= 1125629.8393166605\n",
            "114 /150=71797.1466666666710769572ends @ /content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt\n",
            "115 /150=71834.0133333333410775102ends @ /content/ab/fursov-iztochna-evropa-silnite-na-denya.txt\n",
            "116 /150=72377.5666666666710856635ends @ /content/ab/ivo-hristov-pepelishte.txt\n",
            "bShiftOneStep!\n",
            "start= 809823.0\n",
            "117 /150=73710.911056635ends @ /content/ab/Mark-Twain_-_Pisma_ot_Zemjata._Avtobiografija_-_4984-b.txt\n",
            "bShiftOneStep!\n",
            "start= 180051.0\n",
            "118 /150=75044.2333333333411256635ends @ /content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            "119 /150=75320.5733333333311298086ends @ /content/ab/fursov-vododel.txt\n",
            "bShiftOneStep!\n",
            "start= 61739.74614780663\n",
            "120 /150=76653.9066666666611498086ends @ /content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            "121 /150=76784.9933333333311517749ends @ /content/ab/manipulaciya.txt\n",
            "122 /150=76789.3411518401ends @ /content/ab/stranichni.txt\n",
            "123 /150=76841.1733333333411526176ends @ /content/ab/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt\n",
            "bShiftOneStep!\n",
            "start= 290453.0484671401\n",
            "124 /150=78174.5066666666711726176ends @ /content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt\n",
            "125 /150=79077.4533333333411861618ends @ /content/ab/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt\n",
            "bShiftOneStep!\n",
            "start= 35426.00213454155\n",
            "126 /150=80410.7866666666712061618ends @ /content/ab/DZBE_ponyatiya_7-2020_eim-33.txt\n",
            "127 /150=80428.2733333333312064241ends @ /content/ab/igrach-mekichar.txt\n",
            "128 /150=80442.0812066312ends @ /content/ab/fursov-nato-1938-Myunhen.txt\n",
            "bShiftOneStep!\n",
            "start= 23211.329473942707\n",
            "129 /150=81775.4133333333312266312ends @ /content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            "130 /150=81863.0666666666712279460ends @ /content/ab/ezikovi-raboti.txt\n",
            "131 /150=82316.7466666666712347512ends @ /content/ab/fursov-fefelov-tronove-kultura-na-smeha.txt\n",
            "132 /150=82672.3212400848ends @ /content/ab/pisma-27-7-2021.txt\n",
            "133 /150=82713.7066666666712407056ends @ /content/ab/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt\n",
            "134 /150=82764.0533333333312414608ends @ /content/ab/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt\n",
            "bShiftOneStep!\n",
            "start= 33382.50807524014\n",
            "135 /150=84097.3866666666712614608ends @ /content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt\n",
            "136 /150=84142.6333333333312621395ends @ /content/ab/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt\n",
            "137 /150=84162.4933333333312624374ends @ /content/ab/radost.txt\n",
            "138 /150=84200.6733333333412630101ends @ /content/ab/agi-7-2019.txt\n",
            "bShiftOneStep!\n",
            "start= 314577.0855886436\n",
            "139 /150=85534.0066666666712830101ends @ /content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            "bShiftOneStep!\n",
            "start= 1037666.0\n",
            "140 /150=86867.3413030101ends @ /content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            "141 /150=86901.1413035171ends @ /content/ab/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt\n",
            "bShiftOneStep!\n",
            "start= 45659.05335892609\n",
            "142 /150=88234.4733333333313235171ends @ /content/ab/Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            "bShiftOneStep!\n",
            "start= 521795.2767833621\n",
            "143 /150=89567.8066666666713435171ends @ /content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt\n",
            "144 /150=89779.9813466997ends @ /content/ab/democratius_stamboliyski.txt\n",
            "bShiftOneStep!\n",
            "start= 465024.0\n",
            "145 /150=91113.3133333333413666997ends @ /content/ab/Mitut.txt\n",
            "146 /150=91860.5733333333313779086ends @ /content/ab/Srednovekovni_kitajski_noveli_-_8142-b.txt\n",
            "147 /150=92648.9666666666613897345ends @ /content/ab/computers-razni-26-7-2021.txt\n",
            "148 /150=92660.3866666666713899058ends @ /content/ab/eseto-sasho.txt\n",
            "149 /150=92673.3333333333313901000ends @ /content/ab/750ti.txt\n",
            "150 /150=92807.7413921161ends @ /content/ab/AMD-Intel.txt\n",
            "151 /150=92914.013937100ends @ /content/ab/zaklyuchena-poetry.txt\n",
            "bShiftOneStep!\n",
            "start= 713741.0\n",
            "152 /150=94247.3333333333314137100ends @ /content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            "bShiftOneStep!\n",
            "start= 154896.00000000003\n",
            "153 /150=95580.6666666666714337100ends @ /content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            "bShiftOneStep!\n",
            "start= 64392.20753879546\n",
            "154 /150=96914.014537100ends @ /content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            "155 /150=96936.8133333333414540522ends @ /content/ab/igrachi-yunaci.txt\n",
            "156 /150=96983.7814547567ends @ /content/ab/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt\n",
            "bShiftOneStep!\n",
            "start= 42898.543298665354\n",
            "157 /150=98317.1133333333314747567ends @ /content/ab/Sergej_Minaev_-_Duhless_-_9430-b.txt\n",
            "158 /150=98497.0666666666714774560ends @ /content/ab/zmee.txt\n",
            "159 /150=99542.4533333333414931368ends @ /content/ab/Ada_12-10_E--ot-GLAVA-23.txt\n",
            "160 /150=99551.7933333333314932769ends @ /content/ab/trus.txt\n",
            "bShiftOneStep!\n",
            "start= 31917.434680072016\n",
            "161 /150=100885.1266666666615132769ends @ /content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            "162 /150=101087.6133333333315163142ends @ /content/ab/svobodna-evropa.txt\n",
            "163 /150=101092.5215163878ends @ /content/ab/Hristo Botev -  - . Zashto ne sym_... - 3241.txt\n",
            "bShiftOneStep!\n",
            "start= 117527.0\n",
            "164 /150=102425.8533333333315363878ends @ /content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            "165 /150=102434.015365100ends @ /content/ab/Hristo Botev - V - . Gatanki - 3685.txt\n",
            "bShiftOneStep!\n",
            "start= 64389.00000000001\n",
            "166 /150=103767.3333333333315565100ends @ /content/ab/Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt\n",
            "167 /150=103774.515566175ends @ /content/ab/Hristo Botev -  - . V mehanata - 3243.txt\n",
            "168 /150=103817.0933333333415572564ends @ /content/ab/seksut.txt\n",
            "169 /150=104114.7215617208ends @ /content/ab/pisma-do-irina-malko-red.txt\n",
            "bShiftOneStep!\n",
            "start= 231570.00000000003\n",
            "170 /150=105448.0533333333315817208ends @ /content/ab/Pesen_za_nibelungite_-_6576-b.txt\n",
            "171 /150=105524.3933333333315828659ends @ /content/ab/fursov-stalin-nisheta.txt\n",
            "172 /150=105629.3215844398ends @ /content/ab/fursov-skriti-prichini-vtora-svetovna.txt\n",
            "173 /150=105638.5533333333315845783ends @ /content/ab/Hristo Botev - V - . Vazhno neshto - 3681.txt\n",
            "bShiftOneStep!\n",
            "start= 490252.7338569038\n",
            "174 /150=106971.8866666666716045783ends @ /content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt\n",
            "175 /150=106989.716048455ends @ /content/ab/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt\n",
            "176 /150=107084.1533333333416062623ends @ /content/ab/toj-vyara.txt\n",
            "177 /150=107927.4616189119ends @ /content/ab/Talev-svetilnik-nachalo.txt\n",
            "bShiftOneStep!\n",
            "start= 647814.0\n",
            "178 /150=109260.7933333333316389119ends @ /content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt\n",
            "179 /150=109269.9133333333316390487ends @ /content/ab/dve-narechiya.txt\n",
            "180 /150=109272.7333333333416390910ends @ /content/ab/Hristo Botev - V - . Kazano - 3677.txt\n",
            "181 /150=109466.1733333333416419926ends @ /content/ab/podaruk.txt\n",
            "182 /150=110207.4816531122ends @ /content/ab/Istinata.txt\n",
            "183 /150=111460.516719075ends @ /content/ab/Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt\n",
            "184 /150=111514.0266666666716727104ends @ /content/ab/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt\n",
            "185 /150=111607.5466666666616741132ends @ /content/ab/bivalica.txt\n",
            "186 /150=111613.8616742079ends @ /content/ab/Hristo Botev -  - . Nej - 3234.txt\n",
            "bShiftOneStep!\n",
            "start= 107026.45422435626\n",
            "187 /150=112947.1933333333316942079ends @ /content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt\n",
            "188 /150=113009.5816951437ends @ /content/ab/SmartCen.txt\n",
            "189 /150=113089.8216963473ends @ /content/ab/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt\n",
            "190 /150=113131.1733333333416969676ends @ /content/ab/RISC-CISC.txt\n",
            "191 /150=113518.3133333333417027747ends @ /content/ab/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt\n",
            "192 /150=113605.3617040804ends @ /content/ab/zapadnichestvo.txt\n",
            "193 /150=114782.0617217309ends @ /content/ab/Skandinavska_mitologija_-_8141-b.txt\n",
            "bShiftOneStep!\n",
            "start= 1118980.0\n",
            "194 /150=116115.3933333333317417309ends @ /content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt\n",
            "195 /150=116191.5533333333317428733ends @ /content/ab/galina_tx_slavina_memory_2004_edit-9-10-2016_Clean-11-7-2021.txt\n",
            "196 /150=116370.0733333333317455511ends @ /content/ab/fursov-neoburjoaziya.txt\n",
            "bShiftOneStep!\n",
            "start= 694535.0\n",
            "197 /150=117703.4066666666617655511ends @ /content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            "198 /150=118032.4217704863ends @ /content/ab/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt\n",
            "bShiftOneStep!\n",
            "start= 412524.1579628108\n",
            "199 /150=119365.7533333333317904863ends @ /content/ab/Gore-Vidal_-_Julian_-_7517-b.txt\n",
            "200 /150=119389.5933333333417908439ends @ /content/ab/youtube-26-7-2021.txt\n",
            "201 /150=119394.8817909232ends @ /content/ab/Hristo Botev - V - . Shte se nameri li_ - 3686.txt\n",
            "202 /150=119442.4417916366ends @ /content/ab/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt\n",
            "203 /150=119444.6733333333417916701ends @ /content/ab/Hristo Botev - V - . Shte vidite - 3687.txt\n",
            "204 /150=119500.917925135ends @ /content/ab/hristo-bozhinov.txt\n",
            "bShiftOneStep!\n",
            "start= 3390.999999999995\n",
            "205 /150=120834.2333333333418125135ends @ /content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt\n",
            "206 /150=121584.9333333333318237740ends @ /content/ab/voyna.txt\n",
            "bShiftOneStep!\n",
            "start= 20463.524192597895\n",
            "207 /150=122918.2666666666618437740ends @ /content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt\n",
            "208 /150=122930.5466666666618439582ends @ /content/ab/darenie.txt\n",
            "209 /150=122935.6818440352ends @ /content/ab/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt\n",
            "210 /150=122952.7333333333418442910ends @ /content/ab/Hristo Botev - V - . Dolapyt - 3670.txt\n",
            "211 /150=122955.4733333333318443321ends @ /content/ab/Hristo Botev - V - . Videno - 3676.txt\n",
            "bShiftOneStep!\n",
            "start= 211735.75019864168\n",
            "212 /150=124288.8066666666718643321ends @ /content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            "213 /150=124883.6666666666718732550ends @ /content/ab/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            "bShiftOneStep!\n",
            "start= 42251.59166578247\n",
            "214 /150=126217.018932550ends @ /content/ab/dimov-tyutyun.txt\n",
            "215 /150=126247.6618937149ends @ /content/ab/fursov-otrepki-ubiici-gorbachev.txt\n",
            "216 /150=126591.1418988671ends @ /content/ab/alyosha2.txt\n",
            "217 /150=126595.5418989331ends @ /content/ab/Hristo Botev -  - . Kym brata si - 3228.txt\n",
            "218 /150=126719.5419007931ends @ /content/ab/doktoratut.txt\n",
            "219 /150=126788.3066666666719018246ends @ /content/ab/goden.txt\n",
            "220 /150=126854.5533333333319028183ends @ /content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt\n",
            "221 /150=126959.2533333333319043888ends @ /content/ab/eksploataiya.txt\n",
            "222 /150=127029.3219054398ends @ /content/ab/pechalba.txt\n",
            "223 /150=127036.4666666666619055470ends @ /content/ab/Hristo Botev - V - . Synovnik - 3680.txt\n",
            "224 /150=127048.2419057236ends @ /content/ab/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt\n",
            "bShiftOneStep!\n",
            "start= 370441.3025988133\n",
            "225 /150=128381.5733333333319257236ends @ /content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            "bShiftOneStep!\n",
            "start= 105590.42252233812\n",
            "226 /150=129714.9066666666619457236ends @ /content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            "227 /150=130279.8066666666719541971ends @ /content/ab/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            "bShiftOneStep!\n",
            "start= 439598.0\n",
            "228 /150=131613.1419741971ends @ /content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            "bShiftOneStep!\n",
            "start= 89505.44615161378\n",
            "229 /150=132946.4733333333319941971ends @ /content/ab/Bogomil_Rajnov_-_Umiraj_samo_v_kraen_sluchaj.txt\n",
            "230 /150=133153.4266666666719973014ends @ /content/ab/fursov-multikulturalizum.txt\n",
            "231 /150=133309.4933333333519996424ends @ /content/ab/Protesti2020.txt\n",
            "232 /150=133332.8066666666719999921ends @ /content/ab/john-1.txt\n",
            "bShiftOneStep!\n",
            "start= 256170.0\n",
            "233 /150=134666.1420199921ends @ /content/ab/Piers-Anthony_-_Zov_za_zavryshtane_-_1236-b.txt\n",
            "234 /150=135746.2066666666720361931ends @ /content/ab/EIM-smetach-B-BEZ-ZAGLAVIYA-do-br-6-moje-da-e-nepulno-bg.txt\n",
            "235 /150=135759.2533333333320363888ends @ /content/ab/dod.txt\n",
            "bShiftOneStep!\n",
            "start= 52253.161620537\n",
            "236 /150=137092.5866666666720563888ends @ /content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            "bShiftOneStep!\n",
            "start= 5162.105946019707\n",
            "237 /150=138425.9220763888ends @ /content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            "238 /150=138494.4466666666620774167ends @ /content/ab/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt\n",
            "bShiftOneStep!\n",
            "start= 16231.919054297266\n",
            "239 /150=139827.7820974167ends @ /content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt\n",
            "bShiftOneStep!\n",
            "start= 303896.0\n",
            "240 /150=141161.1133333333421174167ends @ /content/ab/William-Saroyan_-_Neshto_kato_nozh_neshto_kato_tsvete_izobshto_kato_nishto_na_sveta_-_razkazi_-_186-b.txt\n",
            "241 /150=141174.2066666666721176131ends @ /content/ab/igrach-na.txt\n",
            "bShiftOneStep!\n",
            "start= 14521.220400377753\n",
            "242 /150=142507.5421376131ends @ /content/ab/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt\n",
            "243 /150=142521.4421378216ends @ /content/ab/Hristo Botev - V - . Epitafii - 3679.txt\n",
            "bShiftOneStep!\n",
            "start= 208242.3843756006\n",
            "244 /150=143854.7733333333521578216ends @ /content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt\n",
            "245 /150=143938.4733333333321590771ends @ /content/ab/ritum-solo-some-blog-26-7-2021.txt\n",
            "bShiftOneStep!\n",
            "start= 754510.259554563\n",
            "246 /150=145271.8066666666721790771ends @ /content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            "247 /150=145331.4421799716ends @ /content/ab/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt\n",
            "bShiftOneStep!\n",
            "start= 173353.4474904355\n",
            "248 /150=146664.7733333333521999716ends @ /content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt\n",
            "bShiftOneStep!\n",
            "start= 533750.0\n",
            "249 /150=147998.1066666666622199716ends @ /content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            "250 /150=148056.2133333333522208432ends @ /content/ab/fursov-stalin-smurt.txt\n",
            "251 /150=148115.6422217346ends @ /content/ab/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt\n",
            "252 /150=148316.5822247487ends @ /content/ab/smyana-na-sistemata.txt\n",
            "253 /150=148637.7933333333322295669ends @ /content/ab/komunalen.txt\n",
            "254 /150=148814.9266666666722322239ends @ /content/ab/polagane-na-trud.txt\n",
            "bShiftOneStep!\n",
            "start= 137308.0004543977\n",
            "255 /150=150148.2622522239ends @ /content/ab/Branislav_Nushich_-_Avtobiografija_-_1713-b.txt\n",
            "256 /150=150204.8622530729ends @ /content/ab/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt\n",
            "bShiftOneStep!\n",
            "start= 210262.00000000003\n",
            "257 /150=151538.1933333333322730729ends @ /content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt\n",
            "bShiftOneStep!\n",
            "start= 13190.9739637143\n",
            "258 /150=152871.5266666666722930729ends @ /content/ab/Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt\n",
            "259 /150=153009.1133333333422951367ends @ /content/ab/iskam-si-dvata-paunda.txt\n",
            "260 /150=153013.2933333333322951994ends @ /content/ab/vreme-prognoza-26-7-2021.txt\n",
            "261 /150=153019.2622952889ends @ /content/ab/igrachut.txt\n",
            "262 /150=153432.4423014866ends @ /content/ab/Tosh_Chovekut-i-Misleshtata_Mashina.txt\n",
            "bShiftOneStep!\n",
            "start= 76144.00000000001\n",
            "263 /150=154765.7733333333523214866ends @ /content/ab/Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt\n",
            "264 /150=154789.9666666666723218495ends @ /content/ab/ritum.txt\n",
            "265 /150=154796.7666666666623219515ends @ /content/ab/Hristo Botev -  - . Elegija - 3229.txt\n",
            "266 /150=155000.4066666666823250061ends @ /content/ab/fursov-evropa-nichiy-dom-marx.txt\n",
            "267 /150=156301.1066666666623445166ends @ /content/ab/Musketari-chast.txt\n",
            "bShiftOneStep!\n",
            "start= 155494.17884545127\n",
            "268 /150=157634.4423645166ends @ /content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            "269 /150=157675.7623651364ends @ /content/ab/alyosha1.txt\n",
            "bShiftOneStep!\n",
            "start= 89124.3039371409\n",
            "270 /150=159009.0933333333223851364ends @ /content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt\n",
            "271 /150=159173.023875950ends @ /content/ab/fursov-USA-SASHT-nishteta-bednost.txt\n",
            "272 /150=159175.8266666666623876374ends @ /content/ab/Hristo Botev -  - . Poslanie - 3240.txt\n",
            "273 /150=159271.9266666666723890789ends @ /content/ab/fursov-soros.txt\n",
            "bShiftOneStep!\n",
            "start= 712423.0\n",
            "274 /150=160605.2624090789ends @ /content/ab/Andre-Agassi_-_AUGMENTED-Otkrito_-_Avtobiografija_-_10291-b-EDIT-2.txt\n",
            "bShiftOneStep!\n",
            "start= 408462.0556582224\n",
            "275 /150=161938.5933333333224290789ends @ /content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            "bShiftOneStep!\n",
            "start= 11014.057938201604\n",
            "276 /150=163271.9266666666724490789ends @ /content/ab/Alexandre-Jardin_-_Zebrata_-_6966-b.txt\n",
            "277 /150=163310.9224496638ends @ /content/ab/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt\n",
            "278 /150=163315.0666666666824497260ends @ /content/ab/Hristo Botev -  - . Patriot - 3239.txt\n",
            "279 /150=163394.3624509154ends @ /content/ab/AI-ivo-hristov.txt\n",
            "280 /150=163858.0933333333224578714ends @ /content/ab/Analiz-na-smisula.txt\n",
            "281 /150=163947.7333333333424592160ends @ /content/ab/john-2.txt\n",
            "bShiftOneStep!\n",
            "start= 28330.000000000007\n",
            "282 /150=165281.0666666666824792160ends @ /content/ab/Pavel_Vezhinov-Duh-na-bademi.txt\n",
            "283 /150=165294.3933333333424794159ends @ /content/ab/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt\n",
            "284 /150=165607.0824841062ends @ /content/ab/computers-Ardes-7-7-2021.txt\n",
            "285 /150=165669.0266666666724850354ends @ /content/ab/Hristo Botev - V - . Tova vi chaka! - 3668.txt\n",
            "286 /150=165857.0466666666624878557ends @ /content/ab/fursov-kitay.txt\n",
            "bShiftOneStep!\n",
            "start= 200537.0\n",
            "287 /150=167190.3825078557ends @ /content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            "288 /150=167228.6425084296ends @ /content/ab/GamersMeld-2020-g-news-3080-Ti-AMD-Big-Navi-edited.txt\n",
            "bShiftOneStep!\n",
            "start= 149673.0\n",
            "289 /150=168561.9733333333325284296ends @ /content/ab/kakvomu.txt\n",
            "290 /150=168588.3625288254ends @ /content/ab/dari-dobro-eko-soc.txt\n",
            "bShiftOneStep!\n",
            "start= 2140.000000000011\n",
            "291 /150=169921.6933333333325488254ends @ /content/ab/Balzak-chast.txt\n",
            "292 /150=170389.3666666666725558405ends @ /content/ab/Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt\n",
            "bShiftOneStep!\n",
            "start= 564358.0\n",
            "293 /150=171722.725758405ends @ /content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt\n",
            "294 /150=172815.2825922292ends @ /content/ab/PCs-1.txt\n",
            "bShiftOneStep!\n",
            "start= 540357.6836538508\n",
            "295 /150=174148.6133333333426122292ends @ /content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt\n",
            "296 /150=174177.6466666666726126647ends @ /content/ab/uir-pisma.txt\n",
            "bShiftOneStep!\n",
            "start= 208558.35954042396\n",
            "297 /150=175510.9826326647ends @ /content/ab/_-  - 2. Biblija - Nov zavet - 2256.txt\n",
            "298 /150=175521.326328195ends @ /content/ab/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt\n",
            "bShiftOneStep!\n",
            "start= 413363.0313865447\n",
            "299 /150=176854.6333333333326528195ends @ /content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt\n",
            "300 /150=177473.6733333333426621051ends @ /content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt\n",
            "bShiftOneStep!\n",
            "start= 113767.99999999999\n",
            "301 /150=178807.0066666666526821051ends @ /content/ab/Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt\n",
            "302 /150=178845.3466666666826826802ends @ /content/ab/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt\n",
            "bShiftOneStep!\n",
            "start= 22994.564401674026\n",
            "303 /150=180178.6827026802ends @ /content/ab/Terminator-1_6-7-2021.txt\n",
            "304 /150=180239.1827035877ends @ /content/ab/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt\n",
            "305 /150=180332.1666666666627049825ends @ /content/ab/kopiraneto-EIM-1-bez-imena.txt\n",
            "306 /150=180395.0733333333327059261ends @ /content/ab/necessary_illusions.txt\n",
            "307 /150=180917.4866666666627137623ends @ /content/ab/tennis-1.txt\n",
            "308 /150=181281.8866666666627192283ends @ /content/ab/Mitut-ot-Zashto-mujkiyat-chlen.txt\n",
            "309 /150=181286.4533333333427192968ends @ /content/ab/Hristo Botev - V - . Podslushano - 3675.txt\n",
            "310 /150=181314.4427197166ends @ /content/ab/igrachut-i-lyubovta.txt\n",
            "bShiftOneStep!\n",
            "start= 38274.278001566156\n",
            "311 /150=182647.7733333333527397166ends @ /content/ab/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt\n",
            "312 /150=182733.0866666666727409963ends @ /content/ab/fursov-brexit-trump.txt\n",
            "313 /150=183095.7866666666527464368ends @ /content/ab/fursov-korona.txt\n",
            "314 /150=183283.7933333333327492569ends @ /content/ab/rechnik.txt\n",
            "315 /150=183296.827494520ends @ /content/ab/Hristo Botev -  - . Pristanala - 3233.txt\n",
            "316 /150=183313.1333333333327496970ends @ /content/ab/gnyav.txt\n",
            "317 /150=183355.427503310ends @ /content/ab/Hristo Botev - Vnbe - . Namesto programa - 3635.txt\n",
            "318 /150=183377.027506550ends @ /content/ab/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt\n",
            "319 /150=183381.7133333333527507257ends @ /content/ab/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt\n",
            "bShiftOneStep!\n",
            "start= 216251.00000000003\n",
            "320 /150=184715.0466666666627707257ends @ /content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt\n",
            "bShiftOneStep!\n",
            "start= 531997.0\n",
            "321 /150=186048.3827907257ends @ /content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            "322 /150=186074.1066666666627911116ends @ /content/ab/prehvur-vidyo.txt\n",
            "323 /150=186086.1666666666627912925ends @ /content/ab/Hristo Botev - V - . Fabrika za popove - 3673.txt\n",
            "bShiftOneStep!\n",
            "start= 105199.0\n",
            "324 /150=187419.528112925ends @ /content/ab/Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            "325 /150=187548.7428132311ends @ /content/ab/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt\n",
            "bShiftOneStep!\n",
            "start= 157979.73585993372\n",
            "326 /150=188882.0733333333328332311ends @ /content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt\n",
            "bShiftOneStep!\n",
            "start= 484609.0\n",
            "327 /150=190215.4066666666828532311ends @ /content/ab/Omir_-_Odiseja_-_6123-b.txt\n",
            "328 /150=190229.1228534368ends @ /content/ab/dogonvashto.txt\n",
            "329 /150=190290.2066666666728543531ends @ /content/ab/Hristo Botev - V - . Politicheska zima - 3666.txt\n",
            "330 /150=190313.528547025ends @ /content/ab/seks-glavna.txt\n",
            "331 /150=190988.3133333333228648247ends @ /content/ab/pred4.txt\n",
            "332 /150=191013.2266666666528651984ends @ /content/ab/135-godini.txt\n",
            "333 /150=191029.1066666666628654366ends @ /content/ab/telefonni-suveti.txt\n",
            "334 /150=191068.2533333333328660238ends @ /content/ab/cinichna.txt\n",
            "335 /150=191144.4628671669ends @ /content/ab/razumir-br-4.txt\n",
            "336 /150=191151.7066666666728672756ends @ /content/ab/Hristo Botev -  - . Gergxovden - 3238.txt\n",
            "337 /150=191157.428673610ends @ /content/ab/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt\n",
            "bShiftOneStep!\n",
            "start= 82660.51609282257\n",
            "338 /150=192490.7333333333428873610ends @ /content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt\n",
            "bShiftOneStep!\n",
            "start= 299441.0705296264\n",
            "339 /150=193824.0666666666829073610ends @ /content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            "bShiftOneStep!\n",
            "start= 161204.24007792768\n",
            "340 /150=195157.429273610ends @ /content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt\n",
            "bShiftOneStep!\n",
            "start= 743393.0243014938\n",
            "341 /150=196490.7333333333429473610ends @ /content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            "bShiftOneStep!\n",
            "start= 557389.9465805284\n",
            "342 /150=197824.0666666666829673610ends @ /content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt\n",
            "bShiftOneStep!\n",
            "start= 576526.0\n",
            "343 /150=199157.429873610ends @ /content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt\n",
            "bShiftOneStep!\n",
            "start= 3531307.1651803576\n",
            "344 /150=200490.7333333333430073610ends @ /content/ab/Biblija_-_1435-b.txt\n",
            "bShiftOneStep!\n",
            "start= 935308.3433580272\n",
            "345 /150=201824.0666666666830273610ends @ /content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            "bShiftOneStep!\n",
            "start= 222205.0\n",
            "346 /150=203157.430473610ends @ /content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt\n",
            "bShiftOneStep!\n",
            "start= 128613.00000000001\n",
            "347 /150=204490.7333333333430673610ends @ /content/ab/Robin-S-Sharma_-_Monahyt_kojto_prodade_svoeto_ferari_-_Duhovna_pritcha_za_tova_kak_da_osyshtestvite_mechtite_si_i_da_postignete_sydbata_si_-_5236-b.txt\n",
            "bShiftOneStep!\n",
            "start= 465680.25215640984\n",
            "348 /150=205824.0666666666830873610ends @ /content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            "bShiftOneStep!\n",
            "start= 267368.09862976783\n",
            "349 /150=207157.431073610ends @ /content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            "bShiftOneStep!\n",
            "start= 139131.22145079993\n",
            "350 /150=208490.7333333333431273610ends @ /content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            "bShiftOneStep!\n",
            "start= 406197.8820979927\n",
            "351 /150=209824.0666666666831473610ends @ /content/ab/Platon_-_Dyrzhavata_-_9825-b.txt\n",
            "bShiftOneStep!\n",
            "start= 257304.0\n",
            "352 /150=211157.431673610ends @ /content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            "bShiftOneStep!\n",
            "start= 528575.0\n",
            "353 /150=212490.7333333333431873610ends @ /content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt\n",
            "bShiftOneStep!\n",
            "start= 8437.78496672081\n",
            "354 /150=213824.0666666666832073610ends @ /content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            "bShiftOneStep!\n",
            "start= 181376.00000000003\n",
            "355 /150=215157.432273610ends @ /content/ab/Matthew-Quick_-_Narychnik_na_optimista_-_9307-b.txt\n",
            "bShiftOneStep!\n",
            "start= 256346.59089435104\n",
            "356 /150=216490.7333333333432473610ends @ /content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            "bShiftOneStep!\n",
            "start= 600187.0\n",
            "357 /150=217824.0666666666832673610ends @ /content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt\n",
            "bShiftOneStep!\n",
            "start= 487523.9578062601\n",
            "358 /150=219157.432873610ends @ /content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "bShiftOneStep!\n",
            "start= 124124.99999999999\n",
            "359 /150=220490.7333333333433073610ends @ /content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            "bShiftOneStep!\n",
            "start= 987880.897841239\n",
            "360 /150=221824.0666666666833273610ends @ /content/ab/Izkustven_RAZUM_UIR_bez-http_bez_Sudurjanie.txt\n",
            "bShiftOneStep!\n",
            "start= 267805.0\n",
            "361 /150=223157.433473610ends @ /content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt\n",
            "bShiftOneStep!\n",
            "start= 324090.4909121004\n",
            "362 /150=224490.7333333333433673610ends @ /content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            "bShiftOneStep!\n",
            "start= 384554.04926600837\n",
            "363 /150=225824.0666666666833873610ends @ /content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            "bShiftOneStep!\n",
            "start= 211976.00000000003\n",
            "364 /150=227157.434073610ends @ /content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            "bShiftOneStep!\n",
            "start= 886314.4799361302\n",
            "365 /150=228490.7333333333434273610ends @ /content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            "bShiftOneStep!\n",
            "start= 420754.0\n",
            "366 /150=229824.0666666666834473610ends @ /content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            "bShiftOneStep!\n",
            "start= 171118.0\n",
            "367 /150=231157.434673610ends @ /content/ab/Terminator-2_6-7-2021.txt\n",
            "bShiftOneStep!\n",
            "start= 210859.1776876477\n",
            "368 /150=232490.7333333333434873610ends @ /content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt\n",
            "bShiftOneStep!\n",
            "start= 274739.0\n",
            "369 /150=233824.0666666666835073610ends @ /content/ab/Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            "bShiftOneStep!\n",
            "start= 248858.12206186142\n",
            "370 /150=235157.435273610ends @ /content/ab/George-Orwell_-_1984_-_53-b.txt\n",
            "bShiftOneStep!\n",
            "start= 408634.0\n",
            "371 /150=236490.7333333333435473610ends @ /content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            "bShiftOneStep!\n",
            "start= 330828.0\n",
            "372 /150=237824.0666666666835673610ends @ /content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            "bShiftOneStep!\n",
            "start= 131718.71428605\n",
            "373 /150=239157.435873610ends @ /content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt\n",
            "bShiftOneStep!\n",
            "start= 114370.0153090803\n",
            "374 /150=240490.7333333333436073610ends @ /content/ab/Mark-Twain_-_Pisma_ot_Zemjata._Avtobiografija_-_4984-b.txt\n",
            "bShiftOneStep!\n",
            "start= 180051.0\n",
            "375 /150=241824.0666666666836273610ends @ /content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            "bShiftOneStep!\n",
            "start= 106843.0\n",
            "376 /150=243157.436473610ends @ /content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            "bShiftOneStep!\n",
            "start= 151119.86878100957\n",
            "377 /150=244490.7333333333436673610ends @ /content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt\n",
            "bShiftOneStep!\n",
            "start= 182517.94763523995\n",
            "378 /150=245824.0666666666836873610ends @ /content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            "bShiftOneStep!\n",
            "start= 266497.0\n",
            "379 /150=247157.437073610ends @ /content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt\n",
            "bShiftOneStep!\n",
            "start= 387354.4655730812\n",
            "380 /150=248490.7333333333437273610ends @ /content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            "bShiftOneStep!\n",
            "start= 84348.177120843\n",
            "381 /150=249824.0666666666837473610ends @ /content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            "bShiftOneStep!\n",
            "start= 253246.00000000003\n",
            "382 /150=251157.437673610ends @ /content/ab/Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            "bShiftOneStep!\n",
            "start= 580624.0\n",
            "383 /150=252490.7333333333437873610ends @ /content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt\n",
            "bShiftOneStep!\n",
            "start= 465024.0\n",
            "384 /150=253824.0666666666838073610ends @ /content/ab/Mitut.txt\n",
            "bShiftOneStep!\n",
            "start= 69047.35738753754\n",
            "385 /150=255157.438273610ends @ /content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            "bShiftOneStep!\n",
            "start= 48318.504667834626\n",
            "386 /150=256490.7333333333438473610ends @ /content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            "bShiftOneStep!\n",
            "start= 263611.0\n",
            "387 /150=257824.0666666666838673610ends @ /content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            "bShiftOneStep!\n",
            "start= 105942.44709771563\n",
            "388 /150=259157.438873610ends @ /content/ab/Sergej_Minaev_-_Duhless_-_9430-b.txt\n",
            "bShiftOneStep!\n",
            "start= 29404.22140485562\n",
            "389 /150=260490.7333333333439073610ends @ /content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            "bShiftOneStep!\n",
            "start= 117527.0\n",
            "390 /150=261824.0666666666839273610ends @ /content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            "bShiftOneStep!\n",
            "start= 130327.04725067425\n",
            "391 /150=263157.439473610ends @ /content/ab/Pesen_za_nibelungite_-_6576-b.txt\n",
            "bShiftOneStep!\n",
            "start= 841300.2161142004\n",
            "392 /150=264490.7333333333439673610ends @ /content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt\n",
            "bShiftOneStep!\n",
            "start= 249908.94292246943\n",
            "393 /150=265824.0666666666539873610ends @ /content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt\n",
            "bShiftOneStep!\n",
            "start= 24253.9389661517\n",
            "394 /150=267157.440073610ends @ /content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt\n",
            "bShiftOneStep!\n",
            "start= 1115535.7012603288\n",
            "395 /150=268490.7333333333440273610ends @ /content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt\n",
            "bShiftOneStep!\n",
            "start= 107991.1737064662\n",
            "396 /150=269824.0666666666540473610ends @ /content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            "bShiftOneStep!\n",
            "start= 904344.0\n",
            "397 /150=271157.440673610ends @ /content/ab/Gore-Vidal_-_Julian_-_7517-b.txt\n",
            "bShiftOneStep!\n",
            "start= 539409.0\n",
            "398 /150=272490.7333333333440873610ends @ /content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt\n",
            "bShiftOneStep!\n",
            "start= 390377.0\n",
            "399 /150=273824.0666666666541073610ends @ /content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            "bShiftOneStep!\n",
            "start= 418600.9454577589\n",
            "400 /150=275157.441273610ends @ /content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            "bShiftOneStep!\n",
            "start= 159364.0\n",
            "401 /150=276490.7333333333441473610ends @ /content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            "bShiftOneStep!\n",
            "start= 439598.0\n",
            "402 /150=277824.0666666666541673610ends @ /content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            "bShiftOneStep!\n",
            "start= 40178.99630825307\n",
            "403 /150=279157.441873610ends @ /content/ab/Bogomil_Rajnov_-_Umiraj_samo_v_kraen_sluchaj.txt\n",
            "bShiftOneStep!\n",
            "start= 73971.20320545959\n",
            "404 /150=280490.7333333333442073610ends @ /content/ab/Piers-Anthony_-_Zov_za_zavryshtane_-_1236-b.txt\n",
            "bShiftOneStep!\n",
            "start= 474049.0901821156\n",
            "405 /150=281824.0666666666542273610ends @ /content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            "bShiftOneStep!\n",
            "start= 144307.0\n",
            "406 /150=283157.442473610ends @ /content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            "bShiftOneStep!\n",
            "start= 205744.67225499783\n",
            "407 /150=284490.7333333333442673610ends @ /content/ab/William-Saroyan_-_Neshto_kato_nozh_neshto_kato_tsvete_izobshto_kato_nishto_na_sveta_-_razkazi_-_186-b.txt\n",
            "bShiftOneStep!\n",
            "start= 822303.0\n",
            "408 /150=285824.0666666666542873610ends @ /content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt\n",
            "bShiftOneStep!\n",
            "start= 36953.631765110746\n",
            "409 /150=287157.443073610ends @ /content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            "bShiftOneStep!\n",
            "start= 345295.0\n",
            "410 /150=288490.7333333333443273610ends @ /content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt\n",
            "bShiftOneStep!\n",
            "start= 181378.8311130166\n",
            "411 /150=289824.0666666666543473610ends @ /content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            "bShiftOneStep!\n",
            "start= 17397.815321199214\n",
            "412 /150=291157.443673610ends @ /content/ab/Branislav_Nushich_-_Avtobiografija_-_1713-b.txt\n",
            "bShiftOneStep!\n",
            "start= 125207.19596851156\n",
            "413 /150=292490.7333333333443873610ends @ /content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt\n",
            "bShiftOneStep!\n",
            "start= 581955.1505330119\n",
            "414 /150=293824.0666666666544073610ends @ /content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            "bShiftOneStep!\n",
            "start= 220207.54447561956\n",
            "415 /150=295157.444273610ends @ /content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt\n",
            "bShiftOneStep!\n",
            "start= 65220.7554606624\n",
            "416 /150=296490.7333333333444473610ends @ /content/ab/Andre-Agassi_-_AUGMENTED-Otkrito_-_Avtobiografija_-_10291-b-EDIT-2.txt\n",
            "bShiftOneStep!\n",
            "start= 161252.41430607703\n",
            "417 /150=297824.0666666666544673610ends @ /content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            "bShiftOneStep!\n",
            "start= 200537.0\n",
            "418 /150=299157.444873610ends @ /content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            "bShiftOneStep!\n",
            "start= 149673.0\n",
            "419 /150=300490.7333333333445073610ends @ /content/ab/kakvomu.txt\n",
            "bShiftOneStep!\n",
            "start= 83900.96268212827\n",
            "420 /150=301824.0666666666545273610ends @ /content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt\n",
            "bShiftOneStep!\n",
            "start= 427924.58681327617\n",
            "421 /150=303157.445473610ends @ /content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt\n",
            "bShiftOneStep!\n",
            "start= 627001.619193586\n",
            "422 /150=304490.7333333333445673610ends @ /content/ab/_-  - 2. Biblija - Nov zavet - 2256.txt\n",
            "bShiftOneStep!\n",
            "start= 595717.683454074\n",
            "423 /150=305824.0666666666545873610ends @ /content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt\n",
            "bShiftOneStep!\n",
            "start= 113767.99999999999\n",
            "424 /150=307157.446073610ends @ /content/ab/Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt\n",
            "bShiftOneStep!\n",
            "start= 231855.99999999997\n",
            "425 /150=308490.7333333333446273610ends @ /content/ab/Terminator-1_6-7-2021.txt\n",
            "bShiftOneStep!\n",
            "start= 216251.00000000003\n",
            "426 /150=309824.0666666666546473610ends @ /content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt\n",
            "bShiftOneStep!\n",
            "start= 531997.0\n",
            "427 /150=311157.446673610ends @ /content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            "bShiftOneStep!\n",
            "start= 105199.0\n",
            "428 /150=312490.7333333333446873610ends @ /content/ab/Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            "bShiftOneStep!\n",
            "start= 334011.0\n",
            "429 /150=313824.0666666666547073610ends @ /content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt\n",
            "bShiftOneStep!\n",
            "start= 232565.85754075777\n",
            "430 /150=315157.447273610ends @ /content/ab/Omir_-_Odiseja_-_6123-b.txt\n",
            "bShiftOneStep!\n",
            "start= 253596.0\n",
            "431 /150=316490.7333333333447473610ends @ /content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt\n",
            "bShiftOneStep!\n",
            "start= 212524.1674986325\n",
            "432 /150=317824.0666666666547673610ends @ /content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            "LEN: single_string:  47675342\n",
            "LEN: tokenized:  11382385\n",
            "Files without doubling 340\n",
            "Files With doubling: 433\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zBrocaP1SwYi"
      },
      "source": [
        "#**DATASET**\n",
        "\n",
        "#ENSURE there's no NONE elements! Caused Typespec etc. error? #25.6.2021\n",
        "[tokenizer2.eos_token_id if x==None else x for x in string_tokenized]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HIaNrr266Van",
        "outputId": "9e3f02d4-5f25-4704-fabe-8fd0b73ffd42"
      },
      "source": [
        "print(selected_paths)\n",
        "print(len(selected_paths))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['/content/ab/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt', '/content/ab/Biblija_-_1435-b.txt', '/content/ab/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt', '/content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt', '/content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt', '/content/ab/Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt', '/content/ab/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt', '/content/ab/pred3.txt', '/content/ab/ezikovi-raboti.txt', '/content/ab/polagane-na-trud.txt', '/content/ab/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt', '/content/ab/igrach-mekichar.txt', '/content/ab/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt', '/content/ab/Hristo Botev - V - . Retsept - 3678.txt', '/content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt', '/content/ab/Mitut.txt', '/content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt', '/content/ab/Hristo Botev -  - . Gergxovden - 3238.txt', '/content/ab/Hristo Botev - V - . Epitafii - 3679.txt', '/content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt', '/content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt', '/content/ab/fursov-fefelov-tronove-kultura-na-smeha.txt', '/content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt', '/content/ab/Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt', '/content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt', '/content/ab/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt', '/content/ab/gnyav.txt', '/content/ab/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt', '/content/ab/kreativen.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt', '/content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt', '/content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt', '/content/ab/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt', '/content/ab/Analiz-na-smisula.txt', '/content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt', '/content/ab/Hristo Botev - V - . Kazano - 3677.txt', '/content/ab/doktoratut.txt', '/content/ab/EIM-B_br-8-16-li_bez-Chovekut-i-Misleshtata-mashina.txt', '/content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt', '/content/ab/135-godini.txt', '/content/ab/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt', '/content/ab/PCs-1.txt', '/content/ab/Hristo Botev -  - . Pristanala - 3233.txt', '/content/ab/Hristo Botev - V - . Politicheska zima - 3666.txt', '/content/ab/igrachut-i-lyubovta.txt', '/content/ab/negativen-otbor.txt', '/content/ab/igrach-na.txt', '/content/ab/Hristo Botev - V - . Vazhno neshto - 3681.txt', '/content/ab/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt', '/content/ab/fursov-iztochna-evropa-silnite-na-denya.txt', '/content/ab/Aleko-Schweizariya.txt', '/content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt', '/content/ab/telefonni-suveti.txt', '/content/ab/alyosha1.txt', '/content/ab/fursov-USA-SASHT-nishteta-bednost.txt', '/content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt', '/content/ab/Hristo Botev -  - . Borba - 3236.txt', '/content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt', '/content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt', '/content/ab/gender.txt', '/content/ab/Hristo Botev -  - . Na proshtavane - 3235.txt', '/content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt', '/content/ab/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt', '/content/ab/hardware2.txt', '/content/ab/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt', '/content/ab/Hristo Botev -  - . Nej - 3234.txt', '/content/ab/comment-nerds-2-nerds-128-clean1.txt', '/content/ab/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt', '/content/ab/Hristo Botev -  - . Elegija - 3229.txt', '/content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt', '/content/ab/ivo-hristov-pepelishte.txt', '/content/ab/hristo-bozhinov.txt', '/content/ab/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt', '/content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt', '/content/ab/seks-glavna.txt', '/content/ab/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt', '/content/ab/dogonvashto.txt', '/content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt', '/content/ab/Anglijski_narodni_prikazki_-_3637-b.txt', '/content/ab/Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt', '/content/ab/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt', '/content/ab/Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt', '/content/ab/smyana-na-sistemata.txt', '/content/ab/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt', '/content/ab/Hristo Botev -  - . Strannik - 3237.txt', '/content/ab/darenie.txt', '/content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt', '/content/ab/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt', '/content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt', '/content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt', '/content/ab/manipulaciya.txt', '/content/ab/komunalen.txt', '/content/ab/10-noemvri.txt', '/content/ab/Protesti2020.txt', '/content/ab/zmee.txt', '/content/ab/prehvur-vidyo.txt', '/content/ab/robstvo.txt', '/content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt', '/content/ab/podvizi-akro-mezo-stih.txt', '/content/ab/750ti.txt', '/content/ab/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt', '/content/ab/fursov-brexit-trump.txt', '/content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt', '/content/ab/fursov-soros.txt', '/content/ab/podaruk.txt', '/content/ab/Hristo Botev -  - . Zadade se oblak temen - 3245.txt', '/content/ab/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt', '/content/ab/igri-za.txt', '/content/ab/upaduk_dzbe.txt', '/content/ab/fursov-otrepki-ubiici-gorbachev.txt', '/content/ab/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt', '/content/ab/terminator-i-herkules.txt', '/content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt', '/content/ab/tupanasov.txt', '/content/ab/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt', '/content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt', '/content/ab/fursov-stalin-smurt.txt', '/content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt', '/content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt', '/content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt', '/content/ab/john-2.txt', '/content/ab/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt', '/content/ab/Balzak-chast.txt', '/content/ab/Mitut-ot-Zashto-mujkiyat-chlen.txt', '/content/ab/Pavel_Vezhinov-Duh-na-bademi.txt', '/content/ab/Talev-svetilnik-nachalo.txt', '/content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt', '/content/ab/vizata-nachalo.txt', '/content/ab/DZBE_ponyatiya_7-2020_eim-33.txt', '/content/ab/Gore-Vidal_-_Julian_-_7517-b.txt', '/content/ab/stari.txt', '/content/ab/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt', '/content/ab/fursov-multikulturalizum.txt', '/content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt', '/content/ab/Talev-svetilnik-2-chast.txt', '/content/ab/Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt', '/content/ab/mainstream_media.txt', '/content/ab/fursov-korona-apokalipsis.txt', '/content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt', '/content/ab/dimov-tyutyun.txt', '/content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt', '/content/ab/kakvomu.txt', '/content/ab/proc.txt', '/content/ab/Izkustven_RAZUM_UIR_bez-http_bez_Sudurjanie.txt', '/content/ab/Hristo Botev -  - . V mehanata - 3243.txt', '/content/ab/trus.txt', '/content/ab/Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt', '/content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt', '/content/ab/fursov-SSSR-dnesh-bezkulturn.txt', '/content/ab/bezdushen.txt', '/content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt', '/content/ab/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt', '/content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt', '/content/ab/GamersMeld-2020-g-news-3080-Ti-AMD-Big-Navi-edited.txt', '/content/ab/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt', '/content/ab/alyosha2.txt', '/content/ab/razumir-br-4.txt', '/content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt', '/content/ab/fursov-chechoslovakiya-1968-i-dr-rusia.txt', '/content/ab/Hristo Botev - V - . Podslushano - 3675.txt', '/content/ab/iskam-si-dvata-paunda.txt', '/content/ab/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt', '/content/ab/toj-vyara.txt', '/content/ab/radost.txt', '/content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt', '/content/ab/fursov-evropa-nichiy-dom-marx.txt', '/content/ab/Hristo Botev - V - . Shte se nameri li_ - 3686.txt', '/content/ab/Pesen_za_nibelungite_-_6576-b.txt', '/content/ab/polit-komentari-1.txt', '/content/ab/Omir_-_Odiseja_-_6123-b.txt', '/content/ab/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt', '/content/ab/fursov-neoburjoaziya.txt', '/content/ab/Hristo Botev -  - . Mojata molitva - 3244.txt', '/content/ab/Hristo Botev - V - . Gatanki - 3685.txt', '/content/ab/Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt', '/content/ab/Voyna_Kum_Vechnata_Voyna.txt', '/content/ab/rechnik.txt', '/content/ab/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt', '/content/ab/Musketari-chast.txt', '/content/ab/pechalba.txt', '/content/ab/fursov-vododel.txt', '/content/ab/Tosh_Chovekut-i-Misleshtata_Mashina.txt', '/content/ab/stranichni.txt', '/content/ab/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt', '/content/ab/fursov-myusulmani.txt', '/content/ab/fursov-korona.txt', '/content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt', '/content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt', '/content/ab/fursov-skriti-prichini-vtora-svetovna.txt', '/content/ab/Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt', '/content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt', '/content/ab/igrachi-yunaci.txt', '/content/ab/chlen.txt', '/content/ab/Hristo Botev -  - . Hajduti - 3232.txt', '/content/ab/Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt', '/content/ab/pred4.txt', '/content/ab/belezhki-istinata.txt', '/content/ab/EIM-smetach-BEZ-ZAGLAVIYA-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt', '/content/ab/Mitut-Ot-Kontrapunkti-200K.txt', '/content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt', '/content/ab/cinichna.txt', '/content/ab/dod.txt', '/content/ab/Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt', '/content/ab/krivorazbranata.txt', '/content/ab/predstava.txt', '/content/ab/zaklyuchena-poetry.txt', '/content/ab/Hristo Botev -  - . Kym brata si - 3228.txt', '/content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt', '/content/ab/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt', '/content/ab/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt', '/content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt', '/content/ab/Hristo Botev - V - . Tova vi chaka! - 3668.txt', '/content/ab/George-Orwell_-_1984_-_53-b.txt', '/content/ab/evro-put.txt', '/content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt', '/content/ab/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt', '/content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt', '/content/ab/democratius_stamboliyski.txt', '/content/ab/Skandinavska_mitologija_-_8141-b.txt', '/content/ab/Hristo Botev - V - . Videno - 3676.txt', '/content/ab/EIM-smetach-B-BEZ-ZAGLAVIYA-do-br-6-moje-da-e-nepulno-bg.txt', '/content/ab/Ada_12-10_E--ot-GLAVA-23.txt', '/content/ab/john-1.txt', '/content/ab/yunaka-i-taz.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt', '/content/ab/RISC-CISC.txt', '/content/ab/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt', '/content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt', '/content/ab/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt', '/content/ab/sistema-narodnoto.txt', '/content/ab/seksut.txt', '/content/ab/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt', '/content/ab/igrachut.txt', '/content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt', '/content/ab/voyna.txt', '/content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt', '/content/ab/SmartCen.txt', '/content/ab/dve-narechiya.txt', '/content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt', '/content/ab/John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt', '/content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt', '/content/ab/AI-ivo-hristov.txt', '/content/ab/Hristo Botev - Vnbe - . Petrushan - 3638.txt', '/content/ab/Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt', '/content/ab/goden.txt', '/content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt', '/content/ab/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt', '/content/ab/agi-7-2019.txt', '/content/ab/Istinata.txt', '/content/ab/Hristo Botev - V - . Shte vidite - 3687.txt', '/content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt', '/content/ab/AMD-Intel.txt', '/content/ab/ritum.txt', '/content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt', '/content/ab/ivo_hristov_svruh-obobshteniya_6-2021.txt', '/content/ab/baladi.txt', '/content/ab/Hristo Botev -  - . Delba - 3230.txt', '/content/ab/sreshtu-chuzhdico.txt', '/content/ab/Hristo Botev - V - . Fabrika za popove - 3673.txt', '/content/ab/vyarata_i_svobodata.txt', '/content/ab/zapadnichestvo.txt', '/content/ab/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt', '/content/ab/Hristo Botev - V - . Dolapyt - 3670.txt', '/content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt', '/content/ab/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt', '/content/ab/hardware0.txt', '/content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt', '/content/ab/Alexandre-Jardin_-_Zebrata_-_6966-b.txt', '/content/ab/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt', '/content/ab/kopiraneto-EIM-1-bez-imena.txt', '/content/ab/uir-pisma.txt', '/content/ab/hardware1.txt', '/content/ab/eksploataiya.txt', '/content/ab/Lev_Tolstoj_Vojna_i_Mir-KRAYA_NARODITE.txt', '/content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt', '/content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt', '/content/ab/Hristo Botev -  - . Zashto ne sym_... - 3241.txt', '/content/ab/Hristo Botev -  - . Poslanie - 3240.txt', '/content/ab/necessary_illusions.txt', '/content/ab/Platon_-_Dyrzhavata_-_9825-b.txt', '/content/ab/Volkswagen.txt', '/content/ab/Eshil_1.txt', '/content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt', '/content/ab/fursov-schwab.txt', '/content/ab/Srednovekovni_kitajski_noveli_-_8142-b.txt', '/content/ab/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt', '/content/ab/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt', '/content/ab/Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt', '/content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt', '/content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt', '/content/ab/kakvomu_notes_e.txt', '/content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt', '/content/ab/Hristo Botev -  - . Patriot - 3239.txt', '/content/ab/_-  - 2. Biblija - Nov zavet - 2256.txt', '/content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt', '/content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt', '/content/ab/fursov-kitay.txt', '/content/ab/bivalica.txt', '/content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt', '/content/ab/Hristo Botev - V - . Synovnik - 3680.txt', '/content/ab/Hristo Botev -  - . Majtse si - 3227.txt', '/content/ab/fursov-nato-1938-Myunhen.txt', '/content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt', '/content/ab/Hristo Botev - Vnbe - . Namesto programa - 3635.txt', '/content/ab/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt', '/content/ab/fursov-stalin-nisheta.txt', '/content/ab/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt']\n",
            "310\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MIUnNIEUgTD3",
        "outputId": "ab5d5ca9-f56d-4adf-c33c-b6ae5401d648"
      },
      "source": [
        "print(len(paths))\n",
        "print(len(single_string))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "211\n",
            "7639443\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RFjdBSVwujJf"
      },
      "source": [
        "#!rm /content/ab/*"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SjoHMuaF370m"
      },
      "source": [
        "#**Prepare Dataset** for Tensorflow"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h8LO773NJIKH",
        "outputId": "b40325d4-5c1e-482d-a2d0-3ab5187033b7"
      },
      "source": [
        "# AFTER LOADING\n",
        "\n",
        "examples = []\n",
        "num_heads = 16\n",
        "block_size = 160 # OOM ... 170, buffer 3230 #160 #200,4500 - OOM; increase size from 150 28.6 #150 #130 120 #100 #256 - resources exhausted, block 256, bs=20 #100\n",
        "BATCH_SIZE = num_heads  # # 20 # #24 #for 20 layers - model 1280,36,20; 12 for small model 768 ... #12\n",
        "BUFFER_SIZE = 3200 #3200 # increase size from 3200 28.6 #3600 #3200 #3300 #3500  4000 - interrupts connection after one minute; 5000 #3072 #9216 #1000\n",
        "\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "maxsize = int(len(string_tokenized)*0.8)\n",
        "smaller = string_tokenized[0:len(string_tokenized)] #//2]\n",
        "#for i in range(0, len(string_tokenized) - block_size + 1, block_size):\n",
        "#  examples.append(string_tokenized[i:i + block_size])\n",
        "bReverse = False # True #try .. not rectangular - because of the Reverse?\n",
        "\n",
        "if not bReverse:\n",
        "  for i in range(0, len(smaller) - block_size + 1, block_size):\n",
        "    examples.append(smaller[i:i + block_size])\n",
        "''' something is wrong when creating the dataset    \n",
        "elif bReverse: #Very long epochs, 3:35 h -- sometimes run in half, but reverse (it's randomized also later; rather generate the dataset again)\n",
        "  for i in range(len(smaller) - block_size + 1, 0, -block_size):\n",
        "    examples.append(smaller[i:i + block_size])\n",
        "'''    \n",
        "\n",
        "inputs, labels = [], []\n",
        "k = 0\n",
        "top = 0\n",
        "n = 0\n",
        "for ex in examples:\n",
        "  #print(\"top=\", top)\n",
        "  inputs.append(ex[:-1]) #inefficient\n",
        "  labels.append(ex[1:])\n",
        "  if k==0:\n",
        "    print(inputs[:100])\n",
        "    print(labels[:100])\n",
        "  k+=1\n",
        "  if (k%1000==0): print(\"EX: \"+str(k),end=' ')\n",
        "  #print(inputs)\n",
        "  #print(labels)\n",
        "print(k)\n",
        "print(len(inputs))\n",
        "print(len(labels))\n",
        "i1 = len(inputs); i2 = len(labels)\n",
        "i3 = i1*BATCH_SIZE\n",
        "i4 = i1*block_size\n",
        "print(i3)\n",
        "print(i4)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[545, 6038, 10479, 321, 281, 141, 3505, 198, 2905, 141, 141, 141, 2683, 22620, 237, 13, 1395, 13526, 207, 4556, 399, 38186, 1030, 374, 10515, 218, 382, 255, 33303, 11, 10208, 2936, 3422, 495, 426, 7703, 399, 3505, 198, 2905, 382, 211, 4673, 3497, 9176, 351, 254, 483, 692, 233, 374, 2559, 413, 11471, 10479, 321, 281, 13, 5566, 542, 556, 15308, 1781, 11, 11471, 10479, 321, 281, 251, 1097, 5974, 599, 473, 11420, 22440, 239, 556, 20899, 237, 2029, 3039, 18725, 211, 1237, 30416, 208, 11503, 198, 15936, 1679, 22262, 13, 141, 16201, 1478, 10577, 211, 399, 3505, 198, 2905, 382, 323, 1625, 2555, 211, 700, 1004, 224, 228, 13570, 188, 21801, 211, 322, 13, 10479, 321, 281, 198, 224, 228, 12923, 295, 211, 7785, 211, 760, 9744, 224, 3266, 402, 13, 141, 141, 141, 463, 12830, 141, 141, 141, 1796, 396, 1948, 363, 40176, 211, 2616, 141, 1081, 4943, 20688, 198, 10595, 11, 141, 758, 2431, 429]]\n",
            "[[6038, 10479, 321, 281, 141, 3505, 198, 2905, 141, 141, 141, 2683, 22620, 237, 13, 1395, 13526, 207, 4556, 399, 38186, 1030, 374, 10515, 218, 382, 255, 33303, 11, 10208, 2936, 3422, 495, 426, 7703, 399, 3505, 198, 2905, 382, 211, 4673, 3497, 9176, 351, 254, 483, 692, 233, 374, 2559, 413, 11471, 10479, 321, 281, 13, 5566, 542, 556, 15308, 1781, 11, 11471, 10479, 321, 281, 251, 1097, 5974, 599, 473, 11420, 22440, 239, 556, 20899, 237, 2029, 3039, 18725, 211, 1237, 30416, 208, 11503, 198, 15936, 1679, 22262, 13, 141, 16201, 1478, 10577, 211, 399, 3505, 198, 2905, 382, 323, 1625, 2555, 211, 700, 1004, 224, 228, 13570, 188, 21801, 211, 322, 13, 10479, 321, 281, 198, 224, 228, 12923, 295, 211, 7785, 211, 760, 9744, 224, 3266, 402, 13, 141, 141, 141, 463, 12830, 141, 141, 141, 1796, 396, 1948, 363, 40176, 211, 2616, 141, 1081, 4943, 20688, 198, 10595, 11, 141, 758, 2431, 429, 11]]\n",
            "EX: 1000 EX: 2000 EX: 3000 EX: 4000 EX: 5000 EX: 6000 EX: 7000 EX: 8000 EX: 9000 EX: 10000 EX: 11000 EX: 12000 EX: 13000 EX: 14000 EX: 15000 EX: 16000 EX: 17000 EX: 18000 EX: 19000 EX: 20000 EX: 21000 EX: 22000 EX: 23000 EX: 24000 EX: 25000 EX: 26000 EX: 27000 EX: 28000 EX: 29000 EX: 30000 EX: 31000 EX: 32000 EX: 33000 EX: 34000 EX: 35000 EX: 36000 EX: 37000 EX: 38000 EX: 39000 EX: 40000 EX: 41000 EX: 42000 EX: 43000 EX: 44000 EX: 45000 EX: 46000 EX: 47000 EX: 48000 EX: 49000 EX: 50000 EX: 51000 EX: 52000 EX: 53000 EX: 54000 EX: 55000 EX: 56000 EX: 57000 EX: 58000 EX: 59000 EX: 60000 EX: 61000 EX: 62000 EX: 63000 EX: 64000 EX: 65000 EX: 66000 EX: 67000 EX: 68000 EX: 69000 EX: 70000 EX: 71000 71139\n",
            "71139\n",
            "71139\n",
            "1138224\n",
            "11382240\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "deVV6COKfJL5",
        "outputId": "08e1bf6a-3302-42fc-e3bf-447b0fac0c67"
      },
      "source": [
        "print(24264*150)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3639600\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 244
        },
        "id": "8P7mk_whYnkf",
        "outputId": "865e55a8-c037-458f-e859-26e43df15384"
      },
      "source": [
        "#DEBUG\n",
        "#for i,l in zip(inputs[-4000:], labels[-4000:]):\n",
        "#    print(len(i), len(l))\n",
        "ls = []\n",
        "for i,l in zip(inputs, labels):\n",
        "    ls.append(str(len(i)) + \":\" + str(len(l))+\"\\r\\n\")\n",
        "    for i1,l1 in zip(i,l):\n",
        "      ls.append(\"T: \" + str(type(i1)) + \"#\" + str(type(l1)) + \"\\r\\n\")\n",
        "j = ''.join(ls)\n",
        "f = open(\"guz.txt\", \"wt\")\n",
        "f.write(j)\n",
        "f.close()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-14-00e06f806527>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0mls\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\":\"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;34m\"\\r\\n\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m     \u001b[0;32mfor\u001b[0m \u001b[0mi1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0ml1\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0ml\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m       \u001b[0mls\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"T: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\"#\"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m\"\\r\\n\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0mj\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mls\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"guz.txt\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"wt\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WDULTHVE0BOj",
        "outputId": "a22789ba-c2f2-45e6-b433-db14a195c18a"
      },
      "source": [
        "!zip guz.zip guz.txt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  adding: guz.txt (deflated 100%)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r_dHrv8Mn2NW"
      },
      "source": [
        "#**DATASET** **SKIP NONE!**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lbhNiaM-TTT6",
        "outputId": "caa33782-a6b8-405e-8814-96a32ddd99a3"
      },
      "source": [
        "# SKIP NONEs ...\n",
        "\n",
        "import numpy as np\n",
        "print(len(inputs), len(labels))\n",
        "L = len(inputs)-1\n",
        "correct = L #1000 #L#-100\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((inputs[-correct:], labels[-correct:]))\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((inputs[:correct], labels[:correct]))\n",
        "#no dataset2 = tf.data.Dataset.from_tensor_slices((np.asarray(inputs), np.asarray(labels)))\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((tf.convert_to_tensor(inputs), tf.convert_to_tensor(labels))) # error on the last ?\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((inputs, labels))\n",
        "inputs2 = []\n",
        "labels2 = []\n",
        "k=0\n",
        "n=0\n",
        "bSkip = False\n",
        "for i,l in zip(inputs,labels):\n",
        "   if (i==None): print(\"inputs are None!\")\n",
        "   if (l==None): print(\"labels are None!\")   \n",
        "   for j1, j2 in zip(i,l):     \n",
        "     if j1 == None: print(\"None in inputs?(SKIP): \",k); bSkip=True; break;\n",
        "     if j2 == None: print(\"None in labels?(SKIP): \",k); bSkip=True; break;\n",
        "     bSkip = False\n",
        "   \n",
        "   if not bSkip: inputs2.append(i); labels2.append(l) #each second\n",
        "   #if k%2==0: inputs2.append(i); labels2.append(l) #each second\n",
        "   #if k%2==0 or k%3==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3 #fails\n",
        "   #if k%2==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%3==0 or k%5==0 or k%7==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%3==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%3==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   k+=1\n",
        "\n",
        "print(len(inputs2), len(labels2))\n",
        "############dataset2 = tf.data.Dataset.from_tensor_slices((inputs[:15000], labels[:15000]))\n",
        "\n",
        "###############dataset2 = dataset2.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\n",
        "dataset2 = tf.data.Dataset.from_tensor_slices((inputs2, labels2))\n",
        "dataset2 = dataset2.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=False)\n",
        "print(len(string_tokenized))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "71139 71139\n",
            "None in labels?(SKIP):  151\n",
            "None in labels?(SKIP):  11185\n",
            "None in labels?(SKIP):  11197\n",
            "None in labels?(SKIP):  11658\n",
            "None in labels?(SKIP):  11664\n",
            "None in labels?(SKIP):  11677\n",
            "None in labels?(SKIP):  11678\n",
            "None in labels?(SKIP):  11679\n",
            "None in labels?(SKIP):  13598\n",
            "None in labels?(SKIP):  13610\n",
            "None in labels?(SKIP):  13636\n",
            "None in labels?(SKIP):  18556\n",
            "None in labels?(SKIP):  51880\n",
            "None in labels?(SKIP):  51892\n",
            "None in labels?(SKIP):  51917\n",
            "71124 71124\n",
            "11382385\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 849
        },
        "id": "RU7KEtrlJTrI",
        "outputId": "e391b517-c771-4c25-bec7-24ecfcc1579f"
      },
      "source": [
        "#DIVISION OF DATASET\n",
        "\n",
        "import numpy as np\n",
        "print(len(inputs), len(labels))\n",
        "L = len(inputs)-1\n",
        "correct = L #1000 #L#-100\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((inputs[-correct:], labels[-correct:]))\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((inputs[:correct], labels[:correct]))\n",
        "#no dataset2 = tf.data.Dataset.from_tensor_slices((np.asarray(inputs), np.asarray(labels)))\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((tf.convert_to_tensor(inputs), tf.convert_to_tensor(labels))) # error on the last ?\n",
        "#dataset2 = tf.data.Dataset.from_tensor_slices((inputs, labels))\n",
        "inputs2 = []\n",
        "labels2 = []\n",
        "k=0\n",
        "for i,l in zip(inputs,labels):\n",
        "   #if True: inputs2.append(i); labels2.append(l) #each second\n",
        "   if k%2==0: inputs2.append(i); labels2.append(l) #each second\n",
        "   #if k%2==0 or k%3==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3 #fails\n",
        "   #if k%2==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%3==0 or k%5==0 or k%7==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%3==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%3==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   #if k%2==0 or k%5==0: inputs2.append(i); labels2.append(l) #each 2,3\n",
        "   k+=1\n",
        "\n",
        "print(len(inputs2), len(labels2))\n",
        "############dataset2 = tf.data.Dataset.from_tensor_slices((inputs[:15000], labels[:15000]))\n",
        "\n",
        "###############dataset2 = dataset2.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\n",
        "dataset2 = tf.data.Dataset.from_tensor_slices((inputs2, labels2))\n",
        "dataset2 = dataset2.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=False)\n",
        "print(len(string_tokenized))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "39935 39935\n",
            "19968 19968\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "ValueError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/util/structure.py\u001b[0m in \u001b[0;36mnormalize_element\u001b[0;34m(element, element_signature)\u001b[0m\n\u001b[1;32m    105\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mspec\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 106\u001b[0;31m           \u001b[0mspec\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtype_spec_from_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muse_fallback\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    107\u001b[0m       \u001b[0;32mexcept\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/util/structure.py\u001b[0m in \u001b[0;36mtype_spec_from_value\u001b[0;34m(element, use_fallback)\u001b[0m\n\u001b[1;32m    479\u001b[0m   raise TypeError(\"Could not build a TypeSpec for %r with type %s\" %\n\u001b[0;32m--> 480\u001b[0;31m                   (element, type(element).__name__))\n\u001b[0m\u001b[1;32m    481\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: Could not build a TypeSpec for [[16632, 140, 24181, 15970, 141, 140, 593, 192, 3158, 20609, 16349, 141, 141, 141, 141, 140, 58, 41072, 13117, 211, 32969, 399, 5064, 571, 892, 211, 20609, 16349, 382, 60, 141, 141, 140, 478, 11, 25489, 1673, 11, 3830, 226, 1739, 0, 141, 140, 7168, 771, 14227, 11, 771, 645, 14226, 19753, 30, 141, 140, 816, 321, 515, 198, 430, 11, 2883, 9682, 19729, 11, 141, 140, 3856, 5983, 881, 3169, 771, 19305, 5545, 14531, 30, 141, 141, 140, 478, 11, 2626, 11, 2626, 11, 430, 19753, 11, 11649, 11, 141, 140, 34855, 11, 273, 430, 258, 5830, 16069, 26, 141, 140, 34855, 11, 273, 7557, 16336, 1164, 11, 11649, 11, 141, 140, 179, 1164, 476, 1775, 11, 1164, 202, 218, 10628, 0, 141, 141, 140, 25428, 364, 0, 2644, 3096, 343, 815, 6478, 141, 140, 329, 369, 523, 41116, 6356, 8727, 1211, 13, 141, 140, 457, 7557, 507, 862, 11, 2263, 23316, 180, 11, 141, 140, 329, 305], [37739, 141, 141, 140, 3417, 497, 13, 141, 141, 140, 16, 13, 347, 4825, 1381, 16288, 2616, 198, 1194, 13, 141, 140, 17, 13, 318, 1194, 424, 476, 2125, 222, 198, 29452, 26, 14492, 228, 42516, 580, 18734, 11, 198, 7989, 9501, 228, 4781, 580, 2661, 13, 141, 140, 18, 13, 42046, 1381, 25, 224, 625, 2821, 13, 338, 7223, 2821, 13, 141, 140, 19, 13, 13028, 1381, 11, 273, 4711, 239, 2208, 638, 11, 198, 8290, 1381, 4711, 233, 7485, 13, 141, 140, 20, 13, 29784, 1381, 3986, 764, 11, 270, 7485, 255, 1050, 13, 3644, 348, 1653, 11, 7223, 8503, 255, 764, 507, 13, 141, 140, 21, 13, 338, 967, 1381, 25,...",
            "\nDuring handling of the above exception, another exception occurred:\n",
            "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-17-289444b0efe2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     28\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     29\u001b[0m \u001b[0;31m###############dataset2 = dataset2.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 30\u001b[0;31m \u001b[0mdataset2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDataset\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_tensor_slices\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minputs2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabels2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     31\u001b[0m \u001b[0mdataset2\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdataset2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshuffle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mBUFFER_SIZE\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbatch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mBATCH_SIZE\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdrop_remainder\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     32\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstring_tokenized\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py\u001b[0m in \u001b[0;36mfrom_tensor_slices\u001b[0;34m(tensors)\u001b[0m\n\u001b[1;32m    758\u001b[0m       \u001b[0mDataset\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mA\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m`\u001b[0m\u001b[0mDataset\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    759\u001b[0m     \"\"\"\n\u001b[0;32m--> 760\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mTensorSliceDataset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtensors\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    761\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    762\u001b[0m   \u001b[0;32mclass\u001b[0m \u001b[0m_GeneratorState\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, element)\u001b[0m\n\u001b[1;32m   3318\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0melement\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3319\u001b[0m     \u001b[0;34m\"\"\"See `Dataset.from_tensor_slices()` for details.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3320\u001b[0;31m     \u001b[0melement\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstructure\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnormalize_element\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melement\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   3321\u001b[0m     \u001b[0mbatched_spec\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstructure\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtype_spec_from_value\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0melement\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3322\u001b[0m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tensors\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstructure\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_batched_tensor_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbatched_spec\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0melement\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/util/structure.py\u001b[0m in \u001b[0;36mnormalize_element\u001b[0;34m(element, element_signature)\u001b[0m\n\u001b[1;32m    109\u001b[0m         \u001b[0;31m# the value. As a fallback try converting the value to a tensor.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    110\u001b[0m         normalized_components.append(\n\u001b[0;32m--> 111\u001b[0;31m             ops.convert_to_tensor(t, name=\"component_%d\" % i))\n\u001b[0m\u001b[1;32m    112\u001b[0m       \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    113\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mspec\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msparse_tensor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSparseTensorSpec\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/profiler/trace.py\u001b[0m in \u001b[0;36mwrapped\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    161\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mTrace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtrace_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mtrace_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    162\u001b[0m           \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 163\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    164\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    165\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mwrapped\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mconvert_to_tensor\u001b[0;34m(value, dtype, name, as_ref, preferred_dtype, dtype_hint, ctx, accepted_result_types)\u001b[0m\n\u001b[1;32m   1564\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1565\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mret\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1566\u001b[0;31m       \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconversion_func\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mas_ref\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mas_ref\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1567\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1568\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mret\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mNotImplemented\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py\u001b[0m in \u001b[0;36m_constant_tensor_conversion_function\u001b[0;34m(v, dtype, name, as_ref)\u001b[0m\n\u001b[1;32m    337\u001b[0m                                          as_ref=False):\n\u001b[1;32m    338\u001b[0m   \u001b[0m_\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mas_ref\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 339\u001b[0;31m   \u001b[0;32mreturn\u001b[0m \u001b[0mconstant\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mv\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    340\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    341\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py\u001b[0m in \u001b[0;36mconstant\u001b[0;34m(value, dtype, shape, name)\u001b[0m\n\u001b[1;32m    263\u001b[0m   \"\"\"\n\u001b[1;32m    264\u001b[0m   return _constant_impl(value, dtype, shape, name, verify_shape=False,\n\u001b[0;32m--> 265\u001b[0;31m                         allow_broadcast=True)\n\u001b[0m\u001b[1;32m    266\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    267\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py\u001b[0m in \u001b[0;36m_constant_impl\u001b[0;34m(value, dtype, shape, name, verify_shape, allow_broadcast)\u001b[0m\n\u001b[1;32m    274\u001b[0m       \u001b[0;32mwith\u001b[0m \u001b[0mtrace\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTrace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"tf.constant\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    275\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0m_constant_eager_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mctx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mverify_shape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 276\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0m_constant_eager_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mctx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mverify_shape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    277\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    278\u001b[0m   \u001b[0mg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_default_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py\u001b[0m in \u001b[0;36m_constant_eager_impl\u001b[0;34m(ctx, value, dtype, shape, verify_shape)\u001b[0m\n\u001b[1;32m    299\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0m_constant_eager_impl\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mctx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mverify_shape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    300\u001b[0m   \u001b[0;34m\"\"\"Implementation of eager constant.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 301\u001b[0;31m   \u001b[0mt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconvert_to_eager_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mctx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    302\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0mshape\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    303\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mt\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/constant_op.py\u001b[0m in \u001b[0;36mconvert_to_eager_tensor\u001b[0;34m(value, ctx, dtype)\u001b[0m\n\u001b[1;32m     96\u001b[0m       \u001b[0mdtype\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdtypes\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_dtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_datatype_enum\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     97\u001b[0m   \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 98\u001b[0;31m   \u001b[0;32mreturn\u001b[0m \u001b[0mops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mEagerTensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdevice_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     99\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    100\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mValueError\u001b[0m: Can't convert Python sequence with mixed types to Tensor."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B9hak5hsHmr2"
      },
      "source": [
        "del dataset2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GLA0ohSYordy"
      },
      "source": [
        "#!cp /content/c/*  /content/ab/UTF8/\n",
        "#!cp /content/a/K*  /content/ab/\n",
        "#!cp /content/a/L*  /content/ab/\n",
        "#!cp /content/a/M*  /content/ab/\n",
        "#!cp /content/a/N*  /content/ab/\n",
        "#!cp /content/a/O*  /content/ab/\n",
        "#!cp /content/a/P*  /content/ab/\n",
        "#!cp /content/a/R*  /content/ab/\n",
        "!cp /content/a/S*  /content/ab/\n",
        "!cp /content/a/T*  /content/ab/\n",
        "!cp /content/a/U*  /content/ab/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VFU0N96Qssxt"
      },
      "source": [
        "!cp /content/a/V*  /content/ab/\n",
        "!cp /content/a/W*  /content/ab/\n",
        "!cp /content/a/Z*  /content/ab/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xPITnFBHssA1"
      },
      "source": [],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ck7ko9vVX2xE"
      },
      "source": [
        "s = []\n",
        "for i in inputs: s.append(type(i))\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6dsZWWMpYGck"
      },
      "source": [
        "for i in s: print(i)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T85cP-FVJhRS"
      },
      "source": [
        "print(len(inputs), len(labels))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GEwJhlBr5Cxj"
      },
      "source": [
        "#**COMPILE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RXMA8Ey5KQ4Y",
        "outputId": "ca9cb923-79b1-4539-fd40-2d268b16ee85"
      },
      "source": [
        "from datetime import datetime\n",
        "# defining our optimizer\n",
        "optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)\n",
        "#optimizer = tf.keras.optimizers.Adam(learning_rate=5e-5, epsilon=1e-08, clipnorm=1.0) #29.6. try bigger learning rate: 5e-5\n",
        "#optimizer = tf.keras.optimizers.Adam(learning_rate=4e-5, epsilon=1e-08, clipnorm=1.0) #29.6. no say 4e-5 revert: then again to 3;  try bigger learning rate: 5e-5\n",
        "# definining our loss function\n",
        "loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "# defining our metric which we want to observe\n",
        "metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')\n",
        "# compiling the model\n",
        "model2.compile(optimizer=optimizer, loss=[loss, *[None] * model2.config.n_layer], metrics=[metric])\n",
        "print(\"MODEL2 COMPILED: \" + str(datetime.now())) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "MODEL2 COMPILED: 2021-07-28 22:31:03.310378\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JNi36bQqPPEs"
      },
      "source": [
        "print(\"MODEL2 COMPILED: \" + str(datetime.now())) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9HNiuMhRQBa-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "723b05b8-70dc-4a72-d310-73338cbbe707"
      },
      "source": [
        "%tensorflow_version"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Colab only includes TensorFlow 2.x; %tensorflow_version has no effect.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S9O7ffdPI3N3",
        "outputId": "7e9cc56e-8642-4338-cf68-b78f5cd3d312"
      },
      "source": [
        "#RUN THE OTHER ONE FOR SAVING ON INTERRUPT!!!\n",
        "#but first save LOCALLY in order to check\n",
        "num_epoch = 1 #1\n",
        "historyBIG = model2.fit(dataset2, epochs=num_epoch)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).\n",
            "WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7fd14d732de0>> and will run it as-is.\n",
            "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
            "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING: AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7fd14d732de0>> and will run it as-is.\n",
            "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
            "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING:tensorflow:AutoGraph could not transform <function wrap at 0x7fd168fdddd0> and will run it as-is.\n",
            "Cause: while/else statement not yet supported\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING: AutoGraph could not transform <function wrap at 0x7fd168fdddd0> and will run it as-is.\n",
            "Cause: while/else statement not yet supported\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING:tensorflow:The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.\n",
            "WARNING:tensorflow:The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).\n",
            "WARNING:tensorflow:The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.\n",
            "2553/2553 [==============================] - 5194s 2s/step - loss: 0.9001 - logits_loss: 0.9001 - logits_accuracy: 0.8152 - past_key_values_1_accuracy: 0.0019 - past_key_values_2_accuracy: 0.0023 - past_key_values_3_accuracy: 0.0015 - past_key_values_4_accuracy: 0.0019 - past_key_values_5_accuracy: 0.0018 - past_key_values_6_accuracy: 0.0017 - past_key_values_7_accuracy: 0.0022 - past_key_values_8_accuracy: 0.0019 - past_key_values_9_accuracy: 0.0017 - past_key_values_10_accuracy: 0.0018 - past_key_values_11_accuracy: 0.0018 - past_key_values_12_accuracy: 0.0020 - past_key_values_13_accuracy: 0.0021 - past_key_values_14_accuracy: 0.0019 - past_key_values_15_accuracy: 0.0020 - past_key_values_16_accuracy: 0.0018 - past_key_values_17_accuracy: 0.0018 - past_key_values_18_accuracy: 0.0020 - past_key_values_19_accuracy: 0.0019 - past_key_values_20_accuracy: 0.0019 - past_key_values_21_accuracy: 0.0019 - past_key_values_22_accuracy: 0.0019 - past_key_values_23_accuracy: 0.0019 - past_key_values_24_accuracy: 0.0019\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        },
        "id": "hXmFer2GLdss",
        "outputId": "ba7d078a-d381-4f40-a0f8-dce199e4085f"
      },
      "source": [
        "def save_model(dir, model, tokenizer):\n",
        "  from transformers import WEIGHTS_NAME, CONFIG_NAME\n",
        "  import os\n",
        "  output_dir = dir #'./model_bn_custom/'\n",
        "  # creating directory if it is not present\n",
        "  if not os.path.exists(output_dir):\n",
        "    os.mkdir(output_dir)\n",
        "  model_to_save = model.module if hasattr(model, 'module') else model\n",
        "  output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\n",
        "  output_config_file = os.path.join(output_dir, CONFIG_NAME)\n",
        "  # save model and model configs\n",
        "  model.save_pretrained(output_dir)\n",
        "  model_to_save.config.to_json_file(output_config_file)\n",
        "  # save tokenizer\n",
        "  tokenizer.save_pretrained(output_dir)\n",
        "\"\"\"\n",
        "from transformers import WEIGHTS_NAME, CONFIG_NAME import os\n",
        "def save_to_drive(dir, model2, tokenizer2):  \n",
        "  #output_dir = '/content/1024/'\n",
        "  #output_dir = '/content/gpt2/' #FOR tosh.bg ...\n",
        "  #output_dir = '/content/drive/MyDrive/gpt2-medium/'\n",
        "  output_dir = '/content/drive/MyDrive/gpt2/' #-medium/' TWENKID\n",
        "  #output_dir = '/content/drive/My Drive/gpt2-medium/'\n",
        "  #or drive\n",
        "  #output_dir = '/content/drive/My Drive/gpt-bg/'  #for emil\n",
        "\n",
        "  # creating directory if it is not present\n",
        "  if not os.path.exists(output_dir):\n",
        "    os.mkdir(output_dir)\n",
        "  model_to_save = model2.module if hasattr(model2, 'module') else model2\n",
        "  output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\n",
        "  output_config_file = os.path.join(output_dir, CONFIG_NAME)\n",
        "  # save model and model configs\n",
        "  model2.save_pretrained(output_dir)\n",
        "  model_to_save.config.to_json_file(output_config_file)\n",
        "  # save tokenizer\n",
        "  tokenizer2.save_pretrained(output_dir)\n",
        "\"\"\""
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"\\nfrom transformers import WEIGHTS_NAME, CONFIG_NAME import os\\ndef save_to_drive(dir, model2, tokenizer2):  \\n  #output_dir = '/content/1024/'\\n  #output_dir = '/content/gpt2/' #FOR tosh.bg ...\\n  #output_dir = '/content/drive/MyDrive/gpt2-medium/'\\n  output_dir = '/content/drive/MyDrive/gpt2/' #-medium/' TWENKID\\n  #output_dir = '/content/drive/My Drive/gpt2-medium/'\\n  #or drive\\n  #output_dir = '/content/drive/My Drive/gpt-bg/'  #for emil\\n\\n  # creating directory if it is not present\\n  if not os.path.exists(output_dir):\\n    os.mkdir(output_dir)\\n  model_to_save = model2.module if hasattr(model2, 'module') else model2\\n  output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\\n  output_config_file = os.path.join(output_dir, CONFIG_NAME)\\n  # save model and model configs\\n  model2.save_pretrained(output_dir)\\n  model_to_save.config.to_json_file(output_config_file)\\n  # save tokenizer\\n  tokenizer2.save_pretrained(output_dir)\\n\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 171
        },
        "id": "qYQXB2W2gY3a",
        "outputId": "65cb938f-b0b5-46b1-bb7f-0397d67b9aa0"
      },
      "source": [
        "half = model2.half()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-28-2e5dfe7c2bcb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mhalf\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhalf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m: 'TFGPT2LMHeadModel' object has no attribute 'half'"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JBFTdVnhFHXL",
        "outputId": "724a1788-f15b-4cc2-fb65-9549a8f617bd"
      },
      "source": [
        "L = len(inputs2)# - 20 #-1 #skip last\n",
        "save_dir = '/content/gpt/'\n",
        "num_epoch = 1 #1\n",
        "try:\n",
        "   historyBIG = model2.fit(dataset2, epochs=num_epoch)\n",
        "   #historyBIG = model2.fit(dataset2.take(L//2), epochs=num_epoch)\n",
        "   #historyBIG = model2.fit(dataset2.take(L), epochs=num_epoch)\n",
        "except Exception as e:\n",
        "       print(e)\n",
        "       print('Saving model to... ', save_dir)\n",
        "       save_model(save_dir, model2, tokenizer2)\n",
        "       print('Model Saved to: ', save_dir)\n",
        "       print('Generating...')\n",
        "       #save_to_drive(save_drive_dir, model2, tokenizer2)\n",
        "       from transformers import WEIGHTS_NAME, CONFIG_NAME\n",
        "       import os\n",
        "       output_dir = '/content/1024/'\n",
        "       output_dir = '/content/gpt2/' #FOR tosh.bg ...\n",
        "       output_dir = '/content/drive/MyDrive/gpt2-medium/'\n",
        "       output_dir = '/content/drive/MyDrive/gpt2/' #-medium/' TWENKID\n",
        "       #output_dir = '/content/drive/My Drive/gpt2-medium/'\n",
        "       #or drive\n",
        "       #output_dir = '/content/drive/My Drive/gpt-bg/'  #for emil\n",
        "       \n",
        "       # creating directory if it is not present\n",
        "       if not os.path.exists(output_dir):\n",
        "         os.mkdir(output_dir)\n",
        "       model_to_save = model2.module if hasattr(model2, 'module') else model2\n",
        "       output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\n",
        "       output_config_file = os.path.join(output_dir, CONFIG_NAME)\n",
        "       # save model and model configs\n",
        "       model2.save_pretrained(output_dir)\n",
        "       model_to_save.config.to_json_file(output_config_file)\n",
        "       # save tokenizer\n",
        "       tokenizer2.save_pretrained(output_dir)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).\n",
            "WARNING:tensorflow:AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7f87408eed70>> and will run it as-is.\n",
            "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
            "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING: AutoGraph could not transform <bound method Socket.send of <zmq.sugar.socket.Socket object at 0x7f87408eed70>> and will run it as-is.\n",
            "Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.\n",
            "Cause: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING:tensorflow:AutoGraph could not transform <function wrap at 0x7f875baa3170> and will run it as-is.\n",
            "Cause: while/else statement not yet supported\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING: AutoGraph could not transform <function wrap at 0x7f875baa3170> and will run it as-is.\n",
            "Cause: while/else statement not yet supported\n",
            "To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert\n",
            "WARNING:tensorflow:The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.\n",
            "WARNING:tensorflow:The parameters `output_attentions`, `output_hidden_states` and `use_cache` cannot be updated when calling a model.They have to be set to True/False in the config object (i.e.: `config=XConfig.from_pretrained('name', output_attentions=True)`).\n",
            "WARNING:tensorflow:The parameter `return_dict` cannot be set in graph mode and will always be set to `True`.\n",
            "4445/4446 [============================>.] - ETA: 1s - loss: 1.0497 - logits_loss: 1.0497 - logits_accuracy: 0.7922 - past_key_values_1_accuracy: 0.0019 - past_key_values_2_accuracy: 0.0024 - past_key_values_3_accuracy: 0.0015 - past_key_values_4_accuracy: 0.0019 - past_key_values_5_accuracy: 0.0018 - past_key_values_6_accuracy: 0.0016 - past_key_values_7_accuracy: 0.0022 - past_key_values_8_accuracy: 0.0019 - past_key_values_9_accuracy: 0.0017 - past_key_values_10_accuracy: 0.0019 - past_key_values_11_accuracy: 0.0018 - past_key_values_12_accuracy: 0.0020 - past_key_values_13_accuracy: 0.0021 - past_key_values_14_accuracy: 0.0019 - past_key_values_15_accuracy: 0.0019 - past_key_values_16_accuracy: 0.0018 - past_key_values_17_accuracy: 0.0018 - past_key_values_18_accuracy: 0.0020 - past_key_values_19_accuracy: 0.0019 - past_key_values_20_accuracy: 0.0019 - past_key_values_21_accuracy: 0.0019 - past_key_values_22_accuracy: 0.0019 - past_key_values_23_accuracy: 0.0020 - past_key_values_24_accuracy: 0.0019 required broadcastable shapes at loc(unknown)\n",
            "\t [[node Equal_1 (defined at <ipython-input-10-c61b1dbaeb54>:5) ]] [Op:__inference_train_function_55935]\n",
            "\n",
            "Function call stack:\n",
            "train_function\n",
            "\n",
            "Saving model to...  /content/gpt/\n",
            "Model Saved to:  /content/gpt/\n",
            "Generating...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sUA5ez_wxay6"
      },
      "source": [],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PCxw55-pjLSZ"
      },
      "source": [
        "del optimizer"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mDlWlcNslR7U",
        "outputId": "edcd25c4-c285-4f49-b6a4-7222ca49655a"
      },
      "source": [
        "print(str(historyBIG))\n",
        "dir(historyBIG)\n",
        "print(historyBIG.history)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<tensorflow.python.keras.callbacks.History object at 0x7f4930552e90>\n",
            "{'loss': [4.420501232147217], 'logits_loss': [4.420501232147217], 'logits_accuracy': [0.2932157516479492], 'past_key_values_1_accuracy': [0.002017057267948985], 'past_key_values_2_accuracy': [0.0020584198646247387], 'past_key_values_3_accuracy': [0.0015886330511420965], 'past_key_values_4_accuracy': [0.0018523227190598845], 'past_key_values_5_accuracy': [0.0018676117761060596], 'past_key_values_6_accuracy': [0.0017533914651721716], 'past_key_values_7_accuracy': [0.002183150500059128], 'past_key_values_8_accuracy': [0.0018833773210644722], 'past_key_values_9_accuracy': [0.0017308334354311228], 'past_key_values_10_accuracy': [0.0018118303269147873], 'past_key_values_11_accuracy': [0.0017842792440205812], 'past_key_values_12_accuracy': [0.0020730772521346807], 'past_key_values_13_accuracy': [0.0021226024255156517], 'past_key_values_14_accuracy': [0.001943532144650817], 'past_key_values_15_accuracy': [0.0018918976420536637], 'past_key_values_16_accuracy': [0.0017440726514905691], 'past_key_values_17_accuracy': [0.0019193415064364672], 'past_key_values_18_accuracy': [0.001767059788107872], 'past_key_values_19_accuracy': [0.0019479769980534911], 'past_key_values_20_accuracy': [0.001957653323188424], 'past_key_values_21_accuracy': [0.001919544069096446], 'past_key_values_22_accuracy': [0.0019071032293140888], 'past_key_values_23_accuracy': [0.0016108812997117639], 'past_key_values_24_accuracy': [0.0019688429310917854]}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sxtXLaHER-aU"
      },
      "source": [
        "!cp /content/ab/3/* /content/ab/2/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nTT3l13KlnFi",
        "outputId": "39ab14ce-48f3-4517-d981-4e630a13cfee"
      },
      "source": [
        " !unzip -j /content/z/sel.zip /content/ab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  /content/z/sel.zip\n",
            "caution: filename not matched:  /content/ab\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TY_SOtkSqwnX",
        "outputId": "b9cd2a93-4b6a-42a1-84bf-c273d6f4e4e2"
      },
      "source": [
        "!ls /content/ab\n",
        "!cp /content/z/* /content/ab/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " 135-godini.txt\n",
            " Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            " Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt\n",
            " Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt'\n",
            " Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            " alyosha1.txt\n",
            " alyosha2.txt\n",
            " AMD-Intel.txt\n",
            " Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            " Balzak-chast.txt\n",
            " belezhki-istinata.txt\n",
            " Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt\n",
            " Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt\n",
            " Brian-Sibley-John-Howe_-_Karta_na_Tolkinovata_Sredna_zemja_-_6310-b.txt\n",
            " Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt\n",
            " chlen.txt\n",
            " comment-nerds-2-nerds-128-clean1.txt\n",
            " corpus\n",
            " democratius_prilozhenie_drugi.txt\n",
            " democratius_stamboliyski.txt\n",
            " doktoratut.txt\n",
            " DZBE_ponyatiya_7-2020_eim-33.txt\n",
            " EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            " Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            " Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            " Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            " Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt\n",
            " ezikovi-raboti.txt\n",
            " Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            " Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_2_-_6752-b.txt\n",
            " goden.txt\n",
            " hristo-bozhinov.txt\n",
            " Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            " Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            " ivo_hristov_svruh-obobshteniya_6-2021.txt\n",
            "'Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt'\n",
            " Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            " Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            " john-1.txt\n",
            " john-2.txt\n",
            " John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt\n",
            " John-Holt_-_Kak_detsata_uchat_-_5232-b.txt\n",
            " John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            " Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            " kakvomu_contents.txt\n",
            " kakvomu_notes_e.txt\n",
            " kakvomu_notes.txt\n",
            " kakvomu.txt\n",
            " Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt\n",
            " kopiraneto-EIM-1-bez-imena.txt\n",
            " kreativen.txt\n",
            " Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt\n",
            " Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            " Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            " Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            " mainstream_media.txt\n",
            " Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            " Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            " Mitut.txt\n",
            " Musketari-chast.txt\n",
            " necessary_illusions.txt\n",
            " Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            " Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            " podaruk.txt\n",
            " pred3.txt\n",
            " pred4.txt\n",
            " prehvur-vidyo.txt\n",
            " Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt\n",
            " razumir-br-4.txt\n",
            " RISC-CISC.txt\n",
            " ritum.txt\n",
            " seks-glavna.txt\n",
            " Srednovekovni_kitajski_noveli_-_8142-b.txt\n",
            " sreshtu-chuzhdico.txt\n",
            " telefonni-suveti.txt\n",
            " terminator-i-herkules.txt\n",
            " toj-vyara.txt\n",
            " tupanasov.txt\n",
            "'Untitled Folder'\n",
            " upaduk_dzbe.txt\n",
            " voyna.txt\n",
            " vyarata_i_svobodata.txt\n",
            " yunaka-i-taz.txt\n",
            " zaklyuchena-poetry.txt\n",
            " Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt\n",
            " zmee.txt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RdtDqfpprDZ2",
        "outputId": "c8682a64-bbb6-4bc0-b26a-d23782d13f74"
      },
      "source": [
        "!du -sh /content/ab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "134M\t/content/ab\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "id": "nFBS4WJ_kYSv",
        "outputId": "b5c35ad4-04f9-451b-de48-e808fd70ca1c"
      },
      "source": [
        "#SAVE BIG   \n",
        "   \n",
        "from transformers import WEIGHTS_NAME, CONFIG_NAME\n",
        "import os\n",
        "output_dir = '/content/1024/'\n",
        "output_dir = '/content/gpt2/' #FOR tosh.bg ...\n",
        "output_dir = '/content/drive/MyDrive/gpt2-medium/'\n",
        "output_dir = '/content/drive/MyDrive/gpt2/' #-medium/' TWENKID\n",
        "#output_dir = '/content/drive/My Drive/gpt2-medium/'\n",
        "#or drive\n",
        "#output_dir = '/content/drive/My Drive/gpt-bg/'  #for emil\n",
        "\n",
        "# creating directory if it is not present\n",
        "if not os.path.exists(output_dir):\n",
        "  os.mkdir(output_dir)\n",
        "model_to_save = model2.module if hasattr(model2, 'module') else model2\n",
        "output_model_file = os.path.join(output_dir, WEIGHTS_NAME)\n",
        "output_config_file = os.path.join(output_dir, CONFIG_NAME)\n",
        "# save model and model configs\n",
        "model2.save_pretrained(output_dir)\n",
        "model_to_save.config.to_json_file(output_config_file)\n",
        "# save tokenizer\n",
        "tokenizer2.save_pretrained(output_dir)\n",
        "\n",
        "# 1.31 GB model for\n",
        "'''\n",
        "~ MEDIUM \n",
        " n_embd= 1024,\n",
        " n_head = 16\n",
        " n_layer = 24 \n",
        "'''\n",
        "'''\n",
        "The default small model: 474 MB\n",
        "'''"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'\\nThe default small model: 474 MB\\n'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_csXLtPWWuLa"
      },
      "source": [
        "half = model2.half()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AtFvVhw7cjNt",
        "outputId": "7a543c21-de62-49e8-a909-5327e39aac4f"
      },
      "source": [
        "!zip /content/ab.zip /content/ab/*"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  adding: content/ab/_-  - 2. Biblija - Nov zavet - 2256.txt (deflated 75%)\n",
            "  adding: content/ab/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt (deflated 74%)\n",
            "  adding: content/ab/Agatha-Christie_-_Deset_malki_negyrcheta_-_1839-b.txt (deflated 73%)\n",
            "  adding: content/ab/Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt (deflated 72%)\n",
            "  adding: content/ab/Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt (deflated 73%)\n",
            "  adding: content/ab/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt (deflated 72%)\n",
            "  adding: content/ab/Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt (deflated 71%)\n",
            "  adding: content/ab/Aleko-Schweizariya.txt (deflated 70%)\n",
            "  adding: content/ab/Alexandre-Jardin_-_Zebrata_-_6966-b.txt (deflated 72%)\n",
            "  adding: content/ab/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt (deflated 73%)\n",
            "  adding: content/ab/Andre-Maurois_-_Bajron_-_3615-b.txt (deflated 72%)\n",
            "  adding: content/ab/Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt (deflated 74%)\n",
            "  adding: content/ab/Anglijski_narodni_prikazki_-_3637-b.txt (deflated 76%)\n",
            "  adding: content/ab/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt (deflated 72%)\n",
            "  adding: content/ab/Arthur-Hailey_-_Letishte_-_32-b.txt (deflated 74%)\n",
            "  adding: content/ab/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt (deflated 73%)\n",
            "  adding: content/ab/baladi.txt (deflated 63%)\n",
            "  adding: content/ab/Balzak-chast.txt (deflated 73%)\n",
            "  adding: content/ab/Benjamin-Spock_-_Grizhi_za_bebeto_i_deteto_-_1475-b.txt (deflated 76%)\n",
            "  adding: content/ab/Biblija_-_1435-b.txt (deflated 76%)\n",
            "  adding: content/ab/bivalica.txt (deflated 68%)\n",
            "  adding: content/ab/Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt (deflated 80%)\n",
            "  adding: content/ab/Bruder-Grimm_-_Prikazki_-_648-b.txt (deflated 75%)\n",
            "  adding: content/ab/Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt (deflated 75%)\n",
            "  adding: content/ab/Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt (deflated 73%)\n",
            "  adding: content/ab/darenie.txt (deflated 62%)\n",
            "  adding: content/ab/dimov-tyutyun.txt (deflated 73%)\n",
            "  adding: content/ab/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt (deflated 74%)\n",
            "  adding: content/ab/dve-narechiya.txt (deflated 57%)\n",
            "  adding: content/ab/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt (deflated 68%)\n",
            "  adding: content/ab/Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt (deflated 75%)\n",
            "  adding: content/ab/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt (deflated 79%)\n",
            "  adding: content/ab/Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt (deflated 73%)\n",
            "  adding: content/ab/Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt (deflated 72%)\n",
            "  adding: content/ab/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt (deflated 74%)\n",
            "  adding: content/ab/Eshil_-_Tragedii_-_334-b.txt (deflated 72%)\n",
            "  adding: content/ab/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt (deflated 74%)\n",
            "  adding: content/ab/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt (deflated 74%)\n",
            "  adding: content/ab/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt (deflated 71%)\n",
            "  adding: content/ab/George-Orwell_-_1984_-_53-b.txt (deflated 74%)\n",
            "  adding: content/ab/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt (deflated 74%)\n",
            "  adding: content/ab/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt (deflated 74%)\n",
            "  adding: content/ab/gnyav.txt (deflated 61%)\n",
            "  adding: content/ab/Gore-Vidal_-_Julian_-_7517-b.txt (deflated 73%)\n",
            "  adding: content/ab/Gore-Vidal_-_Sytvorenieto_-_987-b.txt (deflated 73%)\n",
            "  adding: content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt (deflated 74%)\n",
            "  adding: content/ab/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt (deflated 74%)\n",
            "  adding: content/ab/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt (deflated 75%)\n",
            "  adding: content/ab/Hristo Botev -  - . Borba - 3236.txt (deflated 60%)\n",
            "  adding: content/ab/Hristo Botev -  - . Delba - 3230.txt (deflated 56%)\n",
            "  adding: content/ab/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt (deflated 56%)\n",
            "  adding: content/ab/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt (deflated 61%)\n",
            "  adding: content/ab/Hristo Botev -  - . Elegija - 3229.txt (deflated 56%)\n",
            "  adding: content/ab/Hristo Botev -  - . Gergxovden - 3238.txt (deflated 57%)\n",
            "  adding: content/ab/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt (deflated 59%)\n",
            "  adding: content/ab/Hristo Botev -  - . Hajduti - 3232.txt (deflated 66%)\n",
            "  adding: content/ab/Hristo Botev -  - . Kym brata si - 3228.txt (deflated 55%)\n",
            "  adding: content/ab/Hristo Botev -  - . Majtse si - 3227.txt (deflated 58%)\n",
            "  adding: content/ab/Hristo Botev -  - . Mojata molitva - 3244.txt (deflated 60%)\n",
            "  adding: content/ab/Hristo Botev -  - . Na proshtavane - 3235.txt (deflated 65%)\n",
            "  adding: content/ab/Hristo Botev -  - . Nej - 3234.txt (deflated 56%)\n",
            "  adding: content/ab/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt (deflated 55%)\n",
            "  adding: content/ab/Hristo Botev -  - . Patriot - 3239.txt (deflated 65%)\n",
            "  adding: content/ab/Hristo Botev -  - . Poslanie - 3240.txt (deflated 49%)\n",
            "  adding: content/ab/Hristo Botev -  - . Pristanala - 3233.txt (deflated 62%)\n",
            "  adding: content/ab/Hristo Botev -  - . Strannik - 3237.txt (deflated 57%)\n",
            "  adding: content/ab/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt (deflated 71%)\n",
            "  adding: content/ab/Hristo Botev - V - . Dolapyt - 3670.txt (deflated 63%)\n",
            "  adding: content/ab/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt (deflated 69%)\n",
            "  adding: content/ab/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt (deflated 70%)\n",
            "  adding: content/ab/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt (deflated 71%)\n",
            "  adding: content/ab/Hristo Botev - V - . Epitafii - 3679.txt (deflated 61%)\n",
            "  adding: content/ab/Hristo Botev - V - . Fabrika za popove - 3673.txt (deflated 62%)\n",
            "  adding: content/ab/Hristo Botev - V - . Gatanki - 3685.txt (deflated 61%)\n",
            "  adding: content/ab/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt (deflated 65%)\n",
            "  adding: content/ab/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt (deflated 69%)\n",
            "  adding: content/ab/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt (deflated 70%)\n",
            "  adding: content/ab/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt (deflated 71%)\n",
            "  adding: content/ab/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt (deflated 58%)\n",
            "  adding: content/ab/Hristo Botev - V - . Kazano - 3677.txt (deflated 48%)\n",
            "  adding: content/ab/Hristo Botev -  - . V mehanata - 3243.txt (deflated 57%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt (deflated 64%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . Namesto programa - 3635.txt (deflated 67%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt (deflated 68%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . Petrushan - 3638.txt (deflated 67%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt (deflated 69%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt (deflated 68%)\n",
            "  adding: content/ab/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt (deflated 66%)\n",
            "  adding: content/ab/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt (deflated 63%)\n",
            "  adding: content/ab/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt (deflated 64%)\n",
            "  adding: content/ab/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt (deflated 60%)\n",
            "  adding: content/ab/Hristo Botev - V - . Podslushano - 3675.txt (deflated 53%)\n",
            "  adding: content/ab/Hristo Botev - V - . Politicheska zima - 3666.txt (deflated 67%)\n",
            "  adding: content/ab/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt (deflated 69%)\n",
            "  adding: content/ab/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt (deflated 70%)\n",
            "  adding: content/ab/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt (deflated 72%)\n",
            "  adding: content/ab/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt (deflated 68%)\n",
            "  adding: content/ab/Hristo Botev - V - . Retsept - 3678.txt (deflated 46%)\n",
            "  adding: content/ab/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt (deflated 68%)\n",
            "  adding: content/ab/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt (deflated 59%)\n",
            "  adding: content/ab/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt (deflated 70%)\n",
            "  adding: content/ab/Hristo Botev - V - . Shte se nameri li_ - 3686.txt (deflated 53%)\n",
            "  adding: content/ab/Hristo Botev - V - . Shte vidite - 3687.txt (deflated 46%)\n",
            "  adding: content/ab/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt (deflated 71%)\n",
            "  adding: content/ab/Hristo Botev - V - . Synovnik - 3680.txt (deflated 66%)\n",
            "  adding: content/ab/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt (deflated 70%)\n",
            "  adding: content/ab/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt (deflated 68%)\n",
            "  adding: content/ab/Hristo Botev - V - . Tova vi chaka! - 3668.txt (deflated 67%)\n",
            "  adding: content/ab/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt (deflated 69%)\n",
            "  adding: content/ab/Hristo Botev - V - . Vazhno neshto - 3681.txt (deflated 57%)\n",
            "  adding: content/ab/Hristo Botev - V - . Videno - 3676.txt (deflated 48%)\n",
            "  adding: content/ab/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt (deflated 56%)\n",
            "  adding: content/ab/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt (deflated 69%)\n",
            "  adding: content/ab/Hristo Botev -  - . Zadade se oblak temen - 3245.txt (deflated 60%)\n",
            "  adding: content/ab/Hristo Botev -  - . Zashto ne sym_... - 3241.txt (deflated 59%)\n",
            "  adding: content/ab/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt (deflated 71%)\n",
            "  adding: content/ab/igrachi-yunaci.txt (deflated 61%)\n",
            "  adding: content/ab/igrach-mekichar.txt (deflated 62%)\n",
            "  adding: content/ab/igrach-na.txt (deflated 60%)\n",
            "  adding: content/ab/igrachut-i-lyubovta.txt (deflated 64%)\n",
            "  adding: content/ab/igrachut.txt (deflated 58%)\n",
            "  adding: content/ab/igri-za.txt (deflated 64%)\n",
            "  adding: content/ab/Irvine-Welsh_-_Porno_-_2467-b.txt (deflated 71%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt (deflated 73%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt (deflated 74%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Fondatsijata_-_1021-b.txt (deflated 74%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt (deflated 74%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt (deflated 75%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Samite_bogove_-_1197-b.txt (deflated 75%)\n",
            "  adding: content/ab/Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt (deflated 74%)\n",
            "  adding: content/ab/IT_Kaldata-razdel-bez-html.txt (deflated 71%)\n",
            "  adding: content/ab/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt (deflated 78%)\n",
            "  adding: content/ab/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt (deflated 73%)\n",
            "  adding: content/ab/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt (deflated 76%)\n",
            "  adding: content/ab/Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt (deflated 72%)\n",
            "  adding: content/ab/Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt (deflated 71%)\n",
            "  adding: content/ab/Ivan Vazov - Tc - 3. Nova zemja - 3754.txt (deflated 72%)\n",
            "  adding: content/ab/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt (deflated 72%)\n",
            "  adding: content/ab/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt (deflated 73%)\n",
            "  adding: content/ab/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt (deflated 73%)\n",
            "  adding: content/ab/Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt (deflated 73%)\n",
            "  adding: content/ab/John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt (deflated 72%)\n",
            "  adding: content/ab/John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt (deflated 73%)\n",
            "  adding: content/ab/John-C-Maxwell_-_21_nezamenimi_kachestva_na_lidera_-_Da_stanesh_chovekyt_kogoto_drugite_shte_poiskat_da_sledvat_-_4739-b.txt (deflated 74%)\n",
            "  adding: content/ab/John-Holt_-_Kak_detsata_uchat_-_5232-b.txt (deflated 74%)\n",
            "  adding: content/ab/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt (deflated 72%)\n",
            "  adding: content/ab/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt (deflated 73%)\n",
            "  adding: content/ab/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt (deflated 72%)\n",
            "  adding: content/ab/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt (deflated 74%)\n",
            "  adding: content/ab/Karl-May_-_Vinetu_I_-_1325-b.txt (deflated 74%)\n",
            "  adding: content/ab/Karl-May_-_Vinetu_II_-_1326-b.txt (deflated 74%)\n",
            "  adding: content/ab/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt (deflated 71%)\n",
            "  adding: content/ab/Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt (deflated 75%)\n",
            "  adding: content/ab/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt (deflated 77%)\n",
            "  adding: content/ab/Lev_Tolstoj_Vojna_i_Mir-KRAYA_NARODITE.txt (deflated 78%)\n",
            "  adding: content/ab/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt (deflated 73%)\n",
            "  adding: content/ab/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt (deflated 76%)\n",
            "  adding: content/ab/Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt (deflated 73%)\n",
            "  adding: content/ab/Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt (deflated 73%)\n",
            "  adding: content/ab/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt (deflated 74%)\n",
            "  adding: content/ab/Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt (deflated 75%)\n",
            "  adding: content/ab/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt (deflated 74%)\n",
            "  adding: content/ab/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt (deflated 74%)\n",
            "  adding: content/ab/Musketari-chast.txt (deflated 74%)\n",
            "  adding: content/ab/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt (deflated 73%)\n",
            "  adding: content/ab/Nikolaj_Hajtov_-_Divi_razkazi_-_558-b.txt (deflated 72%)\n",
            "  adding: content/ab/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt (deflated 75%)\n",
            "  adding: content/ab/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt (deflated 73%)\n",
            "  adding: content/ab/Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt (deflated 74%)\n",
            "  adding: content/ab/Omir_-_Odiseja_-_6123-b.txt (deflated 72%)\n",
            "  adding: content/ab/Pavel_Vezhinov-Duh-na-bademi.txt (deflated 73%)\n",
            "  adding: content/ab/Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt (deflated 74%)\n",
            "  adding: content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt (deflated 73%)\n",
            "  adding: content/ab/Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt (deflated 73%)\n",
            "  adding: content/ab/Pavel_Vezhinov_-_Vezni_-_10103-b.txt (deflated 73%)\n",
            "  adding: content/ab/Pesen_za_nibelungite_-_6576-b.txt (deflated 73%)\n",
            "  adding: content/ab/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt (deflated 69%)\n",
            "  adding: content/ab/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt (deflated 75%)\n",
            "  adding: content/ab/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt (deflated 73%)\n",
            "  adding: content/ab/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt (deflated 72%)\n",
            "  adding: content/ab/Platon_-_Dyrzhavata_-_9825-b.txt (deflated 76%)\n",
            "  adding: content/ab/podvizi-akro-mezo-stih.txt (deflated 59%)\n",
            "  adding: content/ab/Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt (deflated 74%)\n",
            "  adding: content/ab/radost.txt (deflated 62%)\n",
            "  adding: content/ab/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt (deflated 73%)\n",
            "  adding: content/ab/rechnik.txt (deflated 72%)\n",
            "  adding: content/ab/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt (deflated 74%)\n",
            "  adding: content/ab/seksut.txt (deflated 65%)\n",
            "  adding: content/ab/Skandinavska_mitologija_-_8141-b.txt (deflated 74%)\n",
            "  adding: content/ab/Srednovekovni_kitajski_noveli_-_8142-b.txt (deflated 72%)\n",
            "  adding: content/ab/StanisYaw-Lem_-_Edem_-_1368-b.txt (deflated 74%)\n",
            "  adding: content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt (deflated 73%)\n",
            "  adding: content/ab/StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt (deflated 73%)\n",
            "  adding: content/ab/StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt (deflated 73%)\n",
            "  adding: content/ab/stari.txt (deflated 62%)\n",
            "  adding: content/ab/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt (deflated 74%)\n",
            "  adding: content/ab/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt (deflated 72%)\n",
            "  adding: content/ab/stranichni.txt (deflated 56%)\n",
            "  adding: content/ab/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt (deflated 75%)\n",
            "  adding: content/ab/Talev-svetilnik-2-chast.txt (deflated 73%)\n",
            "  adding: content/ab/Talev-svetilnik-nachalo.txt (deflated 73%)\n",
            "  adding: content/ab/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt (deflated 73%)\n",
            "  adding: content/ab/Tosh_Chovekut-i-Misleshtata_Mashina.txt (deflated 73%)\n",
            "  adding: content/ab/trus.txt (deflated 57%)\n",
            "  adding: content/ab/Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt (deflated 73%)\n",
            "  adding: content/ab/Vladimir_Nabokov_-_Lolita_-_275-b.txt (deflated 70%)\n",
            "  adding: content/ab/Voyna_Kum_Vechnata_Voyna.txt (deflated 70%)\n",
            "  adding: content/ab/William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt (deflated 72%)\n",
            "  adding: content/ab/Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt (deflated 73%)\n",
            "  adding: content/ab/Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt (deflated 73%)\n",
            "  adding: content/ab/Zapadnoevropejski_poeti_romantitsi_-_Antologija_-_262-b.txt (deflated 71%)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "80iK3NsQhpZH",
        "outputId": "9dda3e2c-3c09-4bc5-a76a-55de55021b54"
      },
      "source": [
        "!ls '/content/drive/MyDrive/gpt2/'"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "added_tokens.json  merges.txt\t\t    tf_model.h5\t\t   vocab.json\n",
            "config.json\t   special_tokens_map.json  tokenizer_config.json\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NuMfB43hjx8_",
        "outputId": "f695790c-d56c-4867-f8b6-3903abfbbc4a"
      },
      "source": [
        "print(len(paths))\n",
        "print(len(string_tokenized))\n",
        "print(len(single_string))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "198\n",
            "5113696\n",
            "21122380\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f4noxuCyu18X"
      },
      "source": [
        "from transformers import pipeline, set_seed\n",
        "import random\n",
        "set_seed(random.randint(1,99999))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# GENERATE"
      ],
      "metadata": {
        "id": "0sBDQe42oT8x"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 504
        },
        "id": "KvrlPbcwplvS",
        "outputId": "a19d19ae-0b9d-4c20-ff4f-b03193909e70"
      },
      "source": [
        "#GENERATE\n",
        "\n",
        "text = \"Компютрите са бързи, но хората обичат да летят покрай космически кораби.\"\n",
        "text = \"Членът отишъл в космическия кораб и излетял.\"\n",
        "text = \"What is Todor?\"\n",
        "#text = \"Ада отвори вратата и Емил я последва. На земята лежаха златни бижута.\"\n",
        "text = \"Ада отвори вратата и Емил я последва. Коридорът светеше до бижутера, когато Тичко се уплаши. Чернокосата дръпна метлата.\"\n",
        "text = \"Чернокосата ритна Тичко в топките и той излетя с метлата. Ана го погледна и се разсмя, а Емил падна с лазера.\"\n",
        "text = \"Обичам книги и Ана. Асансьорът потегли и срещнаха Широкоплещестият обаче метлата излетя от коридора и падна при картините в асансьора.\"\n",
        "#text = \"Ана, ти си единственото заради което момче като мен има сили да живее.\"\n",
        "#text = \"Компютрите са бързи, но хората обичат да летят покрай космически кораби.\"\n",
        "#Ада отвори вратата и Емил я последва в Англия. Коридорът светеше до бижутера, когато той се страхуваше за свободата си. Какво му трябва на човек\n",
        "text = \"Най-бързият процесор е 1 ГХц и има 500 милиона транзистора, но не е по-добър от Жигули, най-хубавата кола според песните на Наско Ментата. Какво му трябва на човек, освен едно ферари?\"\n",
        "#text = \"\"\"Да бъдеш или да не бъдеш - попита проститутката? Това е въпросът, задник! - отговори Терминаторът - застреля БМВ-то с пушката си и колата с Джон Конър се преобърна. Тогава момичето засмука хия му, докато той не свърши в устата и тя рече: О, толкова ти е голям!\"\"\"\n",
        "text = \"\"\"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът - застреля президента с пушката си и колата с Емил се преобърна. Тогава момичето го целуна и те умряха.\"\"\"\n",
        "text = \"\"\"Програмирането и изкуствения интелект са тъпотии, с които не се занимават феминистки. Демокрацията е мит.\"\"\"\n",
        "text = \"\"\"Универсалният изкуствен разум и какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Демокрацията е мит, всичко е тоталитарна свобода на словото и любовта. Емил се усмихна на процесора.\"\"\"\n",
        "text = \"\"\"Какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Рап. Демокрацията е мит. Любовта и свещеният сметач са най-добрите - рече Тош и се обади на държавата.\"\"\"\n",
        "text = \"\"\"Какво му трябва на човек? Парите, колите, жените и песните!\"\"\"\n",
        "text = \"\"\"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът - застреля президента с пушката си и колата с Емил се преобърна. Тогава момичето го целуна и те умряха.\"\"\"\n",
        "text = \"Членът на ЦК на БКП отишъл в космическия кораб и излетял при лордовете господари от Англия, когато демокрацията дошла при Сталин, който се борил с германеца Шваб.\"\n",
        "text = \"Най-бързият процесор е 1 ГХц и има 500 милиона транзистора, но не е по-добър от Жигули, най-хубавата кола според песните на Наско Ментата. Какво му трябва на човек, освен едно ферари?\"\n",
        "#text = \"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът Емил.\"\"\"\n",
        "#text = \"\"\"Какво му трябва на човек? Парите, колите, жените и песните!\"\"\"\n",
        "#text = \"Тръмп се срещнал със Сталин в Съветския съюз и набили Сорос, за да защити капитализма от Шваб и западния елит.\" #, за да обсъдят бъдещето на компютърната ера в Кремъл.\"\"\"\n",
        "#text = \"Членът на ЦК на БКП отишъл в космическия кораб и излетял при Сталин и Тръмп след Брекзит в СССР, по времето на Горбачов и отишъл в бъдещето.\"\n",
        "#text = \"Извънземното стреля по каубоя, който се обади на капиталистическата система и тя рече: Сорос е лесбийка!\"\n",
        "#text = \"Intel участват в корупционна оргия с Dell и се бият с AMD, обаче компютрите им са стари като Пентиум.\"\n",
        "#text = \"\"\"Какво му трябва на човек? Парите, колите, жените и песните!\"\"\"\n",
        "#text = \"Ада отвори вратата и Емил я последва. Коридорът светеше до бижутера, когато Тичко се уплаши. Чернокосата дръпна метлата.\"\n",
        "#text = \"Мислещите машини са по-бързи от AMD, но NVIDIA произведоха нов изкуствен интелект, който ще работи под Windows и обича да пие ракия с бай Ганьо.\"\n",
        "#text = \"Тръмп се срещнал със Сталин в Съветския съюз и набили Сорос, за да защитят капитализма от Шваб и западния елит с Intel и AMD процесори и Microsoft операционна система.\" #, за да обсъдят бъдещето на компютърната ера в Кремъл.\"\"\"\n",
        "#text = \"\"\"Какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Рап. Демокрацията е мит. Intel и AMD се мразят.\"\"\"\n",
        "## Любовта и свещеният сметач са най-добрите - рече Тош и се обади на Microsoft.\"\"\"\n",
        "#text = \"\"\"Intel и AMD процесори.\"\"\"\n",
        "#text = \"\"\"Да бъдеш или да не бъдеш - попита политикът. Това е въпросът, задник! - отговори роботът - застреля президента с пушката си и колата с Емил се преобърна. Тогава момичето го целуна и те умряха.\"\"\"\n",
        "#text = \"Членът отишъл в космическия кораб, изял хляба и излетял в космоса.\"\n",
        "#text = \"Членът отишъл в космическия кораб, изял хляба и излетял в космоса.\"\n",
        "text = \"Intel участват в корупционна оргия с Nvidia и се бият с AMD, обаче компютрите им са бавни като Geforce 750 Ti и играят само Quake.\"\n",
        "#text = \"Да бъдеш или да не бъдеш - попита Сара. Терминаторът я погледна студено и застреля форда с пушката си и колата с Джон Конър се преобърна. Тогава любимата му го целуна, засмя се и го погледна в очите.\"\n",
        "#text = \"Да бъдеш или да не бъдеш - попита Сара. Терминаторът я погледна студено и гръмна с пушката си по Джон Конър, а любимата му го целуна по металния череп. Той обича да играе тенис.\"\n",
        "#text = \"Тенисът е страхотна игра - Агаси срещна Сампрас на финала в Ню Йорк.\"\n",
        "#text = \"Тенисът е страхотна игра - Агаси срещна Сампрас на финала в Ню Йорк.\"\n",
        "#text = \"Терминаторът стреля с пушката, после Агаси срещна Сампрас на финала в Ню Йорк.\"\n",
        "#text = \"Тръмп се срещнал със Сталин в Съветския съюз и набили Сорос, за да защити капитализма от Шваб и западния елит.\" \n",
        "#text = \"Най-бързият процесор е 1 ГХц и има 500 милиона транзистора, но не е по-добър от Жигули, най-хубавата кола според песните на Наско Ментата.\"\n",
        "#text = \"\"\"Какво му трябва на човек? Парите, колите, жените и песните! - Т1000 се превърна в течен метал.\"\"\"\n",
        "#text = \"\"\"Какво му трябва на човек? Парите, колите, жените обичаме, но не и феминистки. Рап. Демокрацията е мит. Любовта и свещеният сметач са най-добрите - рече Тош и се обади на държавата.\"\"\"\n",
        "#text = \"\"\"Какво му трябва на човек? Парите, колите, жените и песните!\"\"\" #, но не и феминистки. Демокрацията е мит, любовта е способност на роботите. - рече Терминаторът\"\"\"\n",
        "#text = \"Intel участват в корупционна оргия с Dell и се бият с AMD, обаче компютрите им са стари - каза Терминаторът и прегърна Сара.\"\n",
        "#text = \"Ада отвори вратата и Емил я последва. Коридорът светеше до бижутера, когато Тичко се уплаши. Чернокосата дръпна метлата.\"\n",
        "#text = \"Intel участват в корупционна оргия с AMD - двамата обичат процесори и видеокарти.\"\n",
        "#text = \"\"\"Програмирането и изкуствения интелект са тъпотии, с които не се занимават феминистки. Демокрацията е мит.\"\"\"\n",
        "#text = \"Бойко Борисов се срещна със симпатизанти в България. ГЕРБ паднаха от власт.\"\n",
        "#text = \"Мислещите машини са по-бързи от AMD, но NVIDIA произведоха нов изкуствен интелект, който ще работи под Windows и обича да пие ракия.\"\n",
        "text = \"Обичам те, скъпи! - тя хвърли страстен поглед към него и го целуна.\"\n",
        "text = \"Обичам шопската салата, ракия ледена да пия! Вълните морски да ме галят...\"\n",
        "number = 1\n",
        "# encoding the input text\n",
        "input_ids = tokenizer2.encode(text, return_tensors='tf')\n",
        "# getting out output\n",
        "beam_output = model2.generate(\n",
        "  input_ids,\n",
        "  max_length = 200, #120, #200, #300, #200 #320, #160, #150, #200, #250, #250, #120, #,200, # 400, #80\n",
        "  num_beams = 16, #12, 6 # 5,\n",
        "  temperature = 1.0, #1.2, #1.5, #1.25, #1.0, #1,\n",
        "  top_k = 40, #40, #, 40, #40\n",
        "  #top_p = 0.95,\n",
        "  no_repeat_ngram_size=2, #4, #2\n",
        "  num_return_sequences=number\n",
        ")\n",
        "for i in range(0,number):\n",
        "  print(tokenizer2.decode(beam_output[i]))"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "InvalidArgumentError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-11-c0092c71cb99>\u001b[0m in \u001b[0;36m<cell line: 62>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     60\u001b[0m \u001b[0minput_ids\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtokenizer2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreturn_tensors\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'tf'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     61\u001b[0m \u001b[0;31m# getting out output\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 62\u001b[0;31m beam_output = model2.generate(\n\u001b[0m\u001b[1;32m     63\u001b[0m   \u001b[0minput_ids\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     64\u001b[0m   \u001b[0mmax_length\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m200\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;31m#120, #200, #300, #200 #320, #160, #150, #200, #250, #250, #120, #,200, # 400, #80\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/generation/tf_utils.py\u001b[0m in \u001b[0;36mgenerate\u001b[0;34m(self, inputs, generation_config, logits_processor, seed, **kwargs)\u001b[0m\n\u001b[1;32m    996\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    997\u001b[0m             \u001b[0;31m# 12. run beam search\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 998\u001b[0;31m             return self.beam_search(\n\u001b[0m\u001b[1;32m    999\u001b[0m                 \u001b[0minput_ids\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1000\u001b[0m                 \u001b[0mmax_length\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mgeneration_config\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmax_length\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/generation/tf_utils.py\u001b[0m in \u001b[0;36mbeam_search\u001b[0;34m(self, input_ids, do_sample, max_length, pad_token_id, eos_token_id, length_penalty, early_stopping, logits_processor, logits_warper, num_return_sequences, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, **model_kwargs)\u001b[0m\n\u001b[1;32m   2550\u001b[0m             \u001b[0mis_sent_finished\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2551\u001b[0m             \u001b[0mmodel_kwargs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2552\u001b[0;31m         \u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mbeam_search_body_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2553\u001b[0m             \u001b[0mcur_len\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2554\u001b[0m             \u001b[0mrunning_sequences\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/generation/tf_utils.py\u001b[0m in \u001b[0;36mbeam_search_body_fn\u001b[0;34m(cur_len, running_sequences, running_scores, running_beam_indices, sequences, scores, beam_indices, is_sent_finished, model_kwargs)\u001b[0m\n\u001b[1;32m   2361\u001b[0m             \u001b[0mlog_probs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0munflatten_beam_dim\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlog_probs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_beams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2362\u001b[0m             \u001b[0mlog_probs_processed\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlog_probs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2363\u001b[0;31m             \u001b[0mlog_probs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlog_probs\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexpand_dims\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrunning_scores\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2364\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mdo_sample\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2365\u001b[0m                 \u001b[0;31m# Note: logits warpers are intentionally applied after adding running beam scores. On some logits\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/tensorflow/python/util/traceback_utils.py\u001b[0m in \u001b[0;36merror_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    151\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    152\u001b[0m       \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_process_traceback_frames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__traceback__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 153\u001b[0;31m       \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwith_traceback\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfiltered_tb\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    154\u001b[0m     \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    155\u001b[0m       \u001b[0;32mdel\u001b[0m \u001b[0mfiltered_tb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mraise_from_not_ok_status\u001b[0;34m(e, name)\u001b[0m\n\u001b[1;32m   7260\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mraise_from_not_ok_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   7261\u001b[0m   \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" name: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 7262\u001b[0;31m   \u001b[0;32mraise\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_status_to_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   7263\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   7264\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mInvalidArgumentError\u001b[0m: cannot compute AddV2 as input #1(zero-based) was expected to be a half tensor but is a float tensor [Op:AddV2]"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pjiWkRFOlNjp",
        "outputId": "0c606fdb-1bb7-4aab-f270-1820f18c9cf4"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GPT2Tokenizer(name_or_path='/content/drive/MyDrive/gpt2/', vocab_size=50250, model_max_length=1000000000000000019884624838656, is_fast=False, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '<unk>', 'pad_token': '<pad>', 'mask_token': '<mask>'}, clean_up_tokenization_spaces=True)"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "num=3\n",
        "text = \"Обичам шопската салата и електронни часовници с Терминатор 2 и видеокарти.\"\n",
        "input_ids =  tokenizer2.encode(text, return_tensors='tf')\n",
        "out = model2.generate(input_ids, max_length=240, num_beams=12, temperature=0.8, top_k=40, no_repeat_ngram_size=2, num_return_sequences=num, pad_token_id=tokenizer2.eos_token_id)"
      ],
      "metadata": {
        "id": "4EV0QgEDlSO0"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Z7ovvBw6rzOr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(tokenizer2.decode(out[0])) "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zFnPh71mo2wt",
        "outputId": "bbde95a7-db47-4f6a-a6e0-4280154c7efe"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Обичам шопската салата.\n",
            "— Ти да не си готвач? — започна да проумява Лаура, тъй като беше слушал тази реч много пъти. — Ще сготвиш нещо страхотно и аз ще се направя пред Томазо, че съм го сготвил сам?\n",
            "Бруно се разсмя и перна приятеля си по рамото. Не че не беше достатъчно смущаващо, ами да, известно бе как да действа. Всъщност дори не му се искаше да си играе с него — да му поднесе одеколончето, съд\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "for i in range(0,num):\n",
        "  print(tokenizer2.decode(out[i])) "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PJ0ENwFUq5K6",
        "outputId": "b622553b-5fac-46c1-f927-570797d691ba"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Обичам шопската салата и електронни часовници с Терминатор 2 и видеокарти.\n",
            "— Значи това е колата на телескопа? — запитвам, като се опитвам да събера мислите си, без да се замислям, защото знам, че времето няма да им е вече за мястото и че всички трябва да чуят думите ми. Най-добре ще бъде да го гледам, след като видя колко бързо се връща, за да види дали всички малки деца достигат до това, което се опитват да влязат. Това, разбира се, не е никак лесно. Поставя си главата, върху която още не се е появила, иначе ще го бъде. Както казва Жаклинов, всеки може да си влезе и да ви каже: „Измъкна я — казва — колкото е възможно по-дълго, дори ще ида в посолството да видя как изглежда. Да кажа, да извади пари за него, ако си дам малко чай?“ Да ми даде. Аз ги държа сега, повлякох му някоя бакшинена и му ги напълних. Не иска да дава пари, ама не мога — джанда, жената не иска сама да пусне. Ами да тури, гледа! Тая му дойде му даде сърце за жена му:\n",
            "\n",
            "Обичам шопската салата и електронни часовници с Терминатор 2 и видеокарти.\n",
            "— Значи това е колата на телескопа? — запитвам, като се опитвам да събера мислите си, без да се замислям, защото знам, че времето няма да им е вече за мястото и че всички трябва да чуят думите ми. Най-добре ще бъде да го гледам, след като видя колко бързо се връща, за да види дали всички малки деца достигат до това, което се опитват да влязат. Това, разбира се, не е никак лесно. Поставя си главата, върху която още не се е появила, иначе ще го бъде. Както казва Жаклинов, всеки може да си влезе и да ви каже: „Измъкна я — казва — колкото е възможно по-дълго, дори ще ида в посолството да видя как изглежда. Да кажа, да извади пари за него, ако си дам малко чай?“ Да ми даде. Аз ги държа сега, повлякох му някоя бакшинена и му ги напълних. Не иска да дава пари, ама не мога — джанда, жената не иска сама да пусне. Ами да тури — каквото има, гледа, дойде ми се. Разглежда ме по\n",
            "Обичам шопската салата и електронни часовници с Терминатор 2 и видеокарти.\n",
            "— Значи това е колата на телескопа? — запитвам, като се опитвам да събера мислите си, без да се замислям, защото знам, че времето няма да им е вече за мястото и че всички трябва да чуят думите ми. Най-добре ще бъде да го гледам, след като видя колко бързо се връща, за да види дали всички малки деца достигат до това, което се опитват да влязат. Това, разбира се, не е никак лесно. Поставя си главата, върху която още не се е появила, иначе ще го бъде. Както казва Жаклинов, всеки може да си влезе и да ви каже: „Измъкна я — казва — колкото е възможно по-дълго, дори ще ида в посолството да видя как изглежда. Да кажа, да извади пари за него, ако си дам малко чай?“ Да ми даде. Аз ги държа сега, повлякох му някоя бакшинена и му ги напълних. Не иска да дава пари, ама не мога — джанда, жената не иска сама да пусне. Ами да тури — каквото има, гледа, дойде ми се. Виж за такова нещо\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(out)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e231aJ3SqAFU",
        "outputId": "c0ff55ad-bd44-4b1d-e04b-c45092998d3d"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(\n",
            "[[46718 39212   677 28590    13   141   230  1042   224   250   258 16685\n",
            "     30   255  1723   224 16258   672  8506    11   771   352   424 13197\n",
            "    738  3548   503  1357    13   255  1047 17379 27462   638 16212   198\n",
            "    523   323   228  3017   353  7905    11   273   546   307 17379  3944\n",
            "    394    30   141 19041   228  9147   198  2824   222  9114   258   254\n",
            "   5439    13   549   273   250   424  2116 18909   315    11  5453   224\n",
            "     11  2386   494   370   224  5337    13  4567   834   250   325   228\n",
            "   1981   224   258  5420   188   498   255   224   325 11642  4738   331\n",
            "  28154  1209    11   934]], shape=(1, 100), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "\n",
        "#-*- coding: utf-8 -*-\n",
        "import os\n",
        "#from tokenise import BPE_token\n",
        "from tokenizers.models import BPE\n",
        "from tokenizers import Tokenizer\n",
        "from tokenizers.decoders import ByteLevel as ByteLevelDecoder\n",
        "from tokenizers.normalizers import NFKC, Sequence\n",
        "from tokenizers.pre_tokenizers import ByteLevel\n",
        "from tokenizers.trainers import BpeTrainer\n",
        "\n",
        "import tensorflow as tf\n",
        "from transformers import GPT2Config, TFGPT2LMHeadModel, GPT2Tokenizer\n",
        "\n",
        "from transformers import pipeline, set_seed \n",
        "from datetime import datetime\n",
        "import random\n",
        "\n",
        "\"\"\"\n",
        "Author: Todor Arnaudov, \"Sacred Computer\":\n",
        "http://artificial-mind.blogspot.com\n",
        "http://github.com/Twenkid - Looking for partners for my AGI Research Institute and other projects. Check it out!\n",
        "https://youtu.be/V1eO2OpsXBE\n",
        "https://github.com/Twenkid/GPT2-Bulgarian-Training-Tips-and-Tools\n",
        "\n",
        "A method for:\n",
        "\n",
        "Chained Unlimited Directed GPT2 Generation by overlapping prompts-injection and Removing the Injected Beginning of the Following Generated Sequence\n",
        "\n",
        "Created in June-July 2021 while training GPT2-Medium on Colab.\n",
        "\n",
        "Published in 1.2023\n",
        "\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "os.environ['CUDA_VISIBLE_DEVICES'] = '-1' #Can't fit MEDIUM in GPU - can't train even small?\n",
        "\n",
        "#LOAD\n",
        "model_dir = r\"Z:\\gpt2\"\n",
        "\n",
        "tokenizer = None; model = None\n",
        "\n",
        "#tokenizer = GPT2Tokenizer.from_pretrained(model_dir)\n",
        "#model = TFGPT2LMHeadModel.from_pretrained(model_dir)\n",
        "\n",
        "\n",
        "def generate(tokenizer, text=None):\n",
        "  if text == None:\n",
        "    text = \"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя.\"\n",
        "    text = \"Какво му трябва на човек? Парите, колите, жените и песните!\"\n",
        "    text = \"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя: Иване, купи ми кола или ще си намеря друг!\"\n",
        "  # encoding the input text\n",
        "  input_ids = tokenizer.encode(text, return_tensors='tf')\n",
        "  # getting out output\n",
        "  beam_output = model.generate(\n",
        "    input_ids,\n",
        "    max_length = 100,\n",
        "    num_beams = 16,\n",
        "    temperature = 1.0,\n",
        "    no_repeat_ngram_size=2,\n",
        "    num_return_sequences=1,\n",
        "    pad_token_id=tokenizer2.eos_token_id #6-5-2023\n",
        "  )   \n",
        "  for i in range(0,1):\n",
        "    print(tokenizer.decode(beam_output[i]))    \n",
        "  now = datetime.now()\n",
        "  date_time = now.strftime(\"%d-%m-%Y_%H-%M-%S\")\n",
        "  #f = open(\"gen.txt\", \"wt\", encoding='utf-8') #date etc.\n",
        "  fname = \"gen\" +  date_time + \".txt\"\n",
        "  f = open(fname, \"wt\", encoding='utf-8') #date etc.\n",
        "  f.write(tokenizer.decode(beam_output[i]))\n",
        "  f.close()\n",
        "\n",
        "\n",
        "def gen_return(tokenizer, length=100, beams=16, t=1.0, ngram=2, seq=1, top_k=40, text=None):\n",
        "  if text == None:\n",
        "    text = \"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя.\"\n",
        "    text = \"Какво му трябва на човек? Парите, колите, жените и песните!\"\n",
        "    #text = \"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя.\"\n",
        "  # encoding the input text\n",
        "  input_ids = tokenizer.encode(text, return_tensors='tf')\n",
        "  # getting out output\n",
        "  beam_output = model.generate(\n",
        "    input_ids,\n",
        "    max_length = length, #150,\n",
        "    num_beams = beams, #16,\n",
        "    temperature = t, #1.0,\n",
        "    no_repeat_ngram_size=ngram, #2,\n",
        "    num_return_sequences=seq, #1\n",
        "    top_k =  top_k)\n",
        "  ret = []  \n",
        "  for i in range(0,seq):\n",
        "    print(tokenizer.decode(beam_output[i]))    \n",
        "    ret.append(beam_output[i])\n",
        "  print(len(ret))\n",
        "  \n",
        "  now = datetime.now()\n",
        "  date_time = now.strftime(\"%d-%m-%Y_%H-%M-%S\")\n",
        "  #f = open(\"gen.txt\", \"wt\", encoding='utf-8') #date etc.\n",
        "  fname = \"gen-\" +  date_time + \".txt\"\n",
        "  f = open(fname, \"wt\", encoding='utf-8') #date etc.\n",
        "  f.write(tokenizer.decode(beam_output[i]))\n",
        "  f.close()\n",
        "  return(ret)\n",
        "\n",
        "def basic():  \n",
        "  print('Generating...')\n",
        "  generate(tokenizer)    \n",
        "\n",
        "def advanced(tokenizer, text=None):\n",
        "  now = datetime.now()\n",
        "  date_time = now.strftime(\"%d-%m-%Y_%H-%M-%S\")\n",
        "  #f = open(\"gen.txt\", \"wt\", encoding='utf-8') #date etc.\n",
        "  fname = \"gen-\" +  date_time + \".txt\"\n",
        "  f = open(fname, \"wt\", encoding='utf-8') #date etc.\n",
        "  f.write(\"gn = gen_return(tokenizer, length=200, beams=16, t=1.0, ngram=2, seq=1, text=None)\")\n",
        "  print(\"Generating...\" )  \n",
        "  beams = 16  \n",
        "  for b in [16, 8, 4, 2, 1]:\n",
        "    f.write(\"BEAMS=\"+str(b)+\"\\r\\n\")\n",
        "    gn = gen_return(tokenizer, length=200, beams=b, t=1.2, ngram=2, seq=1, text=text)\n",
        "    for nm, item in enumerate(gn):\n",
        "      output = \"\\r\\n\" + str(nm) + \"\\r\\n\" + tokenizer.decode(item)      \n",
        "    f.write(output)\n",
        "    f.write(\"\\r\\n\")\n",
        "  f.close()  \n",
        "  \n",
        "\n",
        "def advancedMany(tokenizer, text=None):\n",
        "  length = 200 #280 # 200#125 #250 #250 #400 #150 #200\n",
        "  now = datetime.now()\n",
        "  date_time = now.strftime(\"%d-%m-%Y_%H-%M-%S\")\n",
        "  #f = open(\"gen.txt\", \"wt\", encoding='utf-8') #date etc.\n",
        "  fname = \"gen-\" +  date_time + \".txt\"\n",
        "  f = open(fname, \"wt\", encoding='utf-8') #date etc.  \n",
        "  beams = 16 #16 #8 #16 # 8 #16  \n",
        "  t = 1.1 #3 #1.2 #not 1.5 #.2 #until 24.6. incl. was 1.2 then a run ~ 20:00 25.6 =1; now 1.5\n",
        "  n = 0\n",
        "  ngram = 2 #2 #3 - not good # was 2 until 25.6 20:hh\n",
        "  f.write(\"gn = gen_return(tokenizer, \" + \"length=\"+str(length)+\" , beams= \" + str(beams) + \", t=\" +str(t) + \", ngram=\" + str(ngram))\n",
        "  print(\"Generating...\")  \n",
        "  #\n",
        "  texts = [\"Барам си хуя и му късам главата! Свършвам във банята, свършвам на земята! Барам си хуя и му късам главата!\", \"Банките спират депозитите на граждани. Вече ще се плаща само в натура, като цените варират според сексапила на клиентите. Грозните ще трябва повече да се потрудят.\", \"Карам си колата с 300 километра, нещо се изпречи като самолет! Но какво да видя самолетът изостава като костенурка!\", \"Върви народе възродени, към Англия, САЩ и Европа върви! И в гъза се еби!\", \"Когато бях овчарче и овците пасях, бях много благодарен, макар и сиромах! Амин!\", \"Императорът се обади на любовницата си от Париж: скъпа, чакам те в Рим. Времето е чудесно!\", \"Да бъдеш или да не бъдеш? - попита Терминаторът и застреля Хамлет, който полетя към стената.\"]\n",
        "  texts = [\"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя: Иване, купи ми кола или ще си намеря друг ебач!\", \"Интел участват в корупционна оргия с Dell, в която са замесени висши търговски представители. За да злепоставят конкурентите си от AMD, те са изключили 10% от ядрата на графичния ускорител GeForce 3.\"]\n",
        "  \n",
        "  texts = [\"Intel участват в корупционна оргия с Dell и се бият с AMD, обаче компютрите им са стари като Пентиум.\"]\n",
        "  texts = [\"Предсказването на бъдещето е основна способност на мозъка - казал Конфуций и изпил една чаша ракия.\"]\n",
        "  texts = [\"Мезалианс, принцесата се отказва от привилегиите си. Или майка ѝ, която е кралица, по всички правила на родословията, те обявява тържествено за рицар, след определен брой подвизи в името на верността към короната.\", \"Intel участват в корупционна оргия с Dell и се бият с AMD, обаче компютрите им са стари като Пентиум.\", \"Предсказването на бъдещето е основна способност на мозъка - казал Конфуций и изпил една чаша ракия.\", \"Барам си хуя и му късам главата! Свършавам във банята, свършвам на земята! Барам си хуя и му късам главата!\", \"Банките спират депозитите на граждани. Вече ще се плаща само в натура, като цените варират според сексапила на клиентите. Грозните ще трябва повече да се потрудят.\", \"Карам си колата с 300 километра, нещо се изпречи като самолет! Но какво да видя самолетът изостава като костенурка!\", \"Върви народе възродени, към Англия, САЩ и Европа върви! И в гъза се еби!\", \"Когато бях овчарче и овците пасях, бях много благодарен, макар и сиромах! Амин!\", \"Императорът се обади на любовницата си от Париж: скъпа, чакам те в Рим. Времето е чудесно!\", \"Да бъдеш или да не бъдеш? - попита Терминаторът и застреля Хамлет, който полетя към стената.\", \"Тодор Живков се срещна с Владимир Путин на тристранна среща с президента на САЩ Ким Кардашиян, след което се развихри дива оргия, при която има пострадали животни.\"]\n",
        "  texts = [\"Microsoft се обадили на Apple за да се разберат за процесора Пентиум.\", \"- Не! - раздра простора Бай Гочо. - Народът е омаян от змейщината. Змейовете са разляли лудо биле по всички извори на словото.\", \"Къде се е чуло и видяло, куче да лети и тухла да чете вестник?\", \"- Не знам, но печката е включена и телевизорът работи. Знаеш ли колко ток гори?!\", \"Херкулес извадил пистолета и гръмнал злия робот в дупарата.\"]\n",
        "  \n",
        "  texts = [\"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя: Ивaне, купи ми кола или ще си намеря друг ебач!\", \"Интел участват в корупционна оргия с Dell, в която са замесени висши търговски представители. За да злепоставят конкурентите си от AMD, те са изключили 10% от ядрата на графичния ускорител GeForce 3.\", \"Microsoft се обадили на Apple за да се разберат за процесора на Intel.\", \"- Не! - раздра простора Бай Гочо. - Народът е омаян от змейщината!\", \"Къде се е чуло и видяло, куче да лети и тухла да чете вестник?\", \"- Не знам, но печката е включена и телевизорът работи. Знаеш ли колко ток гори?!\", \"Херкулес извадил пистолета и гръмнал злия робот в дупарата. Терминаторът се разсмял и отвърнал.\", \"Американският президентът Байдън се срещнал с Ким Кардашиян, която била любовница на Бойко Борисов в резиденцията му. Те отпразнували демокрацията с чаша вино.\", \"Боговете сигурно са полудели, но компютрите са бързи! - рекъл Тош и звъннал на Калоян да се видят с чичо Ачо на лостовете.\", \"Предсказването на бъдещето е основна способност на мозъка - казал Конфуций и изпил една чаша ракия с бай Ганьо.\", \"Барам си хуя и му късам главата! Свършавам във банята, свършвам на земята! Барам си хуя и му късам главата!\", \"Банките спират депозитите на граждани. Вече ще се плаща само в натура, като цените варират според сексапила на клиентите.\", \"Мезалианс, принцесата се отказва от привилегиите си. Или майка ѝ, която е кралица, по всички правила на родословията, те обявява тържествено за рицар, след определен брой подвизи в името на верността към короната.\", \"Когато бях овчарче и овците пасях, бях много благодарен, макар и сиромах!\", \"Алелуя, хвани ме за хуя!\", \"Телевизионните новини и СМИ - системите за масова дезинформация тровят ума на скотонаселението. Изхвърлете си телевизора и ще прогледнете! - казал доктор Пламен Пасков.\", \"Карам си колата с 300 километра, нещо се изпречи като самолет! Но какво да видя самолетът изостава като костенурка!\", \"Върви народе възродени, към Англия, САЩ и Европа върви! И в гъза се еби!\", \"Тодор Живков се срещна с Владимир Путин на тристранна среща с президента на САЩ Ким Кардашиян, след което се развихри дива оргия, при която има пострадали животни.\", \"Императорът се обади на любовницата си от Париж: скъпа, чакам те в Рим. Времето е чудесно!\", \"Да бъдеш или да не бъдеш? - попита Терминаторът и застреля Хамлет, който полетя към стената.\"]\n",
        "  #texts = [\"Тодор Живков се срещна с Владимир Путин на тристранна среща с президента на САЩ Ким Кардашиян, след което се развихри дива оргия, при която има пострадали животни.\"]\n",
        "  #for b in [16, 8, 4, 2, 1]:\n",
        "  #for b in [16, 16, 16, 16, 16]:\n",
        "  #for b in [8, 8, 8, 16, 16]:\n",
        "  #set_seed(random.randint(1,9999999))\n",
        "  #texts = [\"Windows 11 е най-якият процесор на AMD!\",\"Intel пускат нова версия на операционната система AMD 11\", \"ЕИМ СВЯТ и юнаците срещу GPU-тата\"]\n",
        "  texts.append(\"Windows 11 е най-якият процесор на AMD!\")\n",
        "  texts.append(\"Intel пускат нова версия на операционната система AMD 11\")\n",
        "  texts.append(\"ЕИМ СВЯТ и юнаците срещу GPU-тата\")\n",
        "  #3-7-2021 prostotiya\n",
        "  #texts = [\"Да бъдеш или да не бъдеш - попита проститутката? Това е въпросът, задник! - отговори Терминаторът - застреля БМВ-то с пушката си и колата с Джон Конър се преобърна. Тогава момичето засмука хуя му, докато той не свърши в устата и тя рече: О, толкова ти е голям!\"]\n",
        "  \n",
        "  b = beams\n",
        "  top_k = 40\n",
        "  sd = 0\n",
        "  for text in texts:\n",
        "    f.write(str(n) + \":\" + \", BEAMS = \"+str(b)+\" seed: \" + str(sd) + \"t: \" + str(t) + \", top_k: \" + str(top_k) + \"\\r\\n\")\n",
        "    #set_seed(random.randint(1,9999999))\n",
        "    sd = random.randint(1,9999999)\n",
        "    set_seed(sd)        \n",
        "    gn = gen_return(tokenizer, length=length, beams=b, t=t, ngram=2, seq=1, top_k=top_k, text=text) #texts[n])\n",
        "    for nm, item in enumerate(gn):\n",
        "      output = \"\\r\\n\" + str(nm) + \"\\r\\n\" + tokenizer.decode(item)      \n",
        "      nm+=1\n",
        "    f.write(output)\n",
        "    f.write(\"\\r\\n\") \n",
        "    n+=1    \n",
        "  f.close()  \n",
        "  \n",
        "#params = [0 = text, 1 = length, 2 = beams, 3 = temperature, 4 = ngram, 5 = top_k]\n",
        "#or text - separated?\n",
        "#def advancedSingleRet(tokenizer, text, params) --> str : \n",
        "def advancedSingleRet(tokenizer, model, params) -> str : \n",
        "  text, sd, length, beams, t, ngram, top_k = params\n",
        "  seq = 1 #one only for now - possible future many and select from them etc. but too slow\n",
        "  now = datetime.now() \n",
        "  #length = 200#125 #250 #250 #400 #150 #200  \n",
        "  date_time = now.strftime(\"%d-%m-%Y_%H-%M-%S\")\n",
        "  #f = open(\"gen.txt\", \"wt\", encoding='utf-8') #date etc.\n",
        "  fname = \"gen-\" +  date_time + \".txt\"\n",
        "  f = open(fname, \"wt\", encoding='utf-8') #date etc.  \n",
        "  #beams = 8 #16 #8 #16 # 8 #16  \n",
        "  #t = 1 #3 #1.2 #not 1.5 #.2 #until 24.6. incl. was 1.2 then a run ~ 20:00 25.6 =1; now 1.5\n",
        "  #n = 0\n",
        "  #ngram = 2 #2 #3 - not good # was 2 until 25.6 20:hh\n",
        "  f.write(\"gn = gen_return(tokenizer, \" + \"length=\"+str(length)+\" , beams= \" + str(beams) + \", t=\" +str(t) + \", ngram=\" + str(ngram))\n",
        "  print(\"Generating...\")  \n",
        "  input_ids = tokenizer.encode(text, return_tensors='tf')\n",
        "  # getting out output\n",
        "  beam_output = model.generate(\n",
        "    input_ids,\n",
        "    max_length = length, #150,\n",
        "    num_beams = beams, #16,\n",
        "    temperature = t, #1.0,\n",
        "    no_repeat_ngram_size=ngram, #2,\n",
        "    num_return_sequences=seq, #1\n",
        "    top_k = top_k\n",
        "  ) \n",
        "  #ret = []  \n",
        "  #for i in range(0,seq):\n",
        "  #  print(tokenizer.decode(beam_output[i]))    \n",
        "  #  ret.append(beam_output[i])\n",
        "  #return beam_output[0] #this is encoded! \n",
        "  dec = tokenizer.decode(beam_output[0]) #beam_output[0] #this is encoded!\n",
        "  f.write(dec); #includes the input -- sometimes would be excluded in sequential generation\n",
        "  f.close()\n",
        "  return dec #tokenizer.decode(beam_output[0]) #beam_output[0] #this is encoded!\n",
        "  \n",
        "\n",
        "def advancedCycle(tokenizer, feed=None):\n",
        "  pass\n",
        "  \n",
        "def genMany():\n",
        "  text = \"Той каза на жена си, че много я обича и винаги след нея ще тича, а тя се разсмя: Иване, купи ми кола или ще си намеря друг!\"\n",
        "  #advanced(text)\n",
        "  set_seed(345)\n",
        "  advancedMany(tokenizer, None)  \n",
        "  \n",
        "\n",
        "simpleEnd = ' '\n",
        "def endSentence():\n",
        "  if simpleEnd != None: return simpleEnd\n",
        "  ends = ['.','!',';','?','...','?!']\n",
        "  r = random.randint(1,len(ends)-1)\n",
        "  return ends[r]\n",
        "def endSentenceGeneration(text):\n",
        "  return ' ' #more complex, ensure finishing previous burst properly\n",
        "  #return '.' #more complex, ensure finishing previous burst properly\n",
        "  \n",
        "def genRecurse(): #28.6.2021\n",
        "    '''\n",
        "    0 -- characters, names\n",
        "    1 -- locations\n",
        "    2 -- topics\n",
        "    3 -- specifics ...\n",
        "    '''\n",
        "    feed = []\n",
        "    \n",
        "    feed.append([[\"Емил\"], [\"Ада\"], [\"Иван\"], [\"Петър\"], [\"Ангел\"], [\"Мария\"]])\n",
        "    feed.append([[\"София\"], [\"България\"], [\"Париж\"], [\"Франция\"]])\n",
        "    feed.append([[\"отивам\"], [\"пристигам\"], [\"заминавам\"], [\"кацам\"], [\"срещна\"], [\"видя\"], [\"целуна\"], [\"прегърна\"]])\n",
        "    feed.append([[\"телевизора\"], [\"колата\"], [\"ракия\"], [\"москвич\"], [\"летище\"], [\"самолет\"], [\"хляб\"], [\"нива\"]])    \n",
        "    \n",
        "    ln = 80\n",
        "    ln_step = 50 #80\n",
        "    total = ''\n",
        "    ftotal = open(r\"e:\\gpt\\total.txt\", \"wt\", encoding='utf-8')\n",
        "    print(type(feed[0][0]),type(feed[1][0]),type(feed[2][0]),type(feed[3][0]))\n",
        "    print(type(feed[0][0][0]),type(feed[1][0]),type(feed[2][0]),type(feed[3][0]))\n",
        "    mix = feed[0][0][0] + \" \" + feed[1][0][0] + \" \" + feed[2][0][0] + \" \" + feed[3][0][0] + endSentence()\n",
        "    sd = set_seed(random.randint(1,9999))\n",
        "    par = (mix, sd, ln, 16, 1.2, 2, 40)\n",
        "    lenmix = len(mix)\n",
        "        \n",
        "    #LOAD\n",
        "    #'''\n",
        "    model_dir = r\"Z:\\gpt2\"\n",
        "    tokenizer = GPT2Tokenizer.from_pretrained(model_dir)\n",
        "    model = TFGPT2LMHeadModel.from_pretrained(model_dir)\n",
        "    #'''\n",
        "\n",
        "    ret = advancedSingleRet(tokenizer, model, par)\n",
        "    print(ret)\n",
        "    fn = ret.find(mix) #initial\n",
        "    print(\"FN=\",fn)\n",
        "    if fn > -1:\n",
        "               trim = ret[fn+len(mix):]\n",
        "               print(fn+len(mix))\n",
        "               ftotal.write(\"\\r\\n=======\\r\\n\"+trim+\"\\r\\n\")\n",
        "    else: trim = ret               \n",
        "    total = trim + endSentenceGeneration(trim) #ret[len(mix):]  not+\n",
        "    ftotal.write(\"\\r\\n\"+ret[fn:fn+len(mix)] + \"\\r\\n\" + \"\\r\\n\" + mix + \"\\r\\n=====\\r\\n\")\n",
        "    ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "    \n",
        "    #next = ret[len(mix):] #exclude the beginning already given\n",
        "    next = total\n",
        "    start = len(total)\n",
        "    #to_add =\". \" + feed[0][1] + \" на \" + feed[1][1] + \" тогава \" + feed[2][1] + \"когато \" + feed[3][1]\n",
        "    \n",
        "    #ADD PRE-\n",
        "    to_add = \". \" + feed[0][1][0] + \" за да \" + feed[1][1][0] + \" понеже \" + feed[2][1][0] + \" когато \" + feed[3][1][0] + endSentence()\n",
        "    #next = next + to_add\n",
        "    next = to_add + next  #in the beginning in order to smooth the connection\n",
        "    #new_start = len(to_add) + start #trim from here\n",
        "    new_start = len(to_add) # + start #trim from here\n",
        "\n",
        "    ln+=ln_step #should tokenize, check len etc...\n",
        "    len_to_add = len(to_add)    \n",
        "    sd = set_seed(random.randint(1,9999))\n",
        "    \n",
        "    input_ids = tokenizer.encode(next, return_tensors='tf')\n",
        "    if len(input_ids) > ln: ln = len(input_ids)+lnstep\n",
        "    \n",
        "    par = (next, sd, ln, 16, 1.2, 2, 40)\n",
        "    ret = advancedSingleRet(tokenizer,model, par)\n",
        "    \n",
        "    fn = ret.find(to_add) #, len(mix)) #this is not exactly - not precise; there are no repetitions anyway\n",
        "    print(\"FN=\",fn)    \n",
        "    if fn > -1:\n",
        "               trim = ret[fn+len(mix):]\n",
        "               ftotal.write(\"\\r\\n=======\\r\\n\"+trim+\"\\r\\n\")               \n",
        "    else: trim = ret               \n",
        "    #total += trim\n",
        "    total +=ret[new_start:] + endSentenceGeneration(trim)\n",
        "    print(fn)    \n",
        "    \n",
        "    #total += trim #ret[len(next):] \n",
        "    print(ret)\n",
        "    print(total)\n",
        "    ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "\n",
        "    #THIRD    \n",
        "    next = total #ret[len(mix):] #exclude the beginning already given\n",
        "    pred_len = len(next)\n",
        "    #to_add =\". \" + feed[0][1] + \" на \" + feed[1][1] + \" тогава \" + feed[2][1] + \"когато \" + feed[3][1]\n",
        "    to_add = \". \" + feed[0][2][0] + \" към \" + feed[1][2][0] + \" защото \" + feed[2][2][0] + \" голям \" + feed[3][2][0] + endSentence()       \n",
        "    #next = next + to_add\n",
        "    next = to_add + next\n",
        "    #new_start = len(next) #to_add)\n",
        "    new_start = len(to_add)\n",
        "    ln+=ln_step\n",
        "    len_to_add = len(to_add) \n",
        "    sd = set_seed(random.randint(1,9999))    \n",
        "    \n",
        "    input_ids = tokenizer.encode(next, return_tensors='tf')\n",
        "    if len(input_ids) > ln: ln = len(input_ids)+lnstep\n",
        "    par = (next, sd, ln, 16, 1.2, 2, 40)\n",
        "    \n",
        "    ret = advancedSingleRet(tokenizer,model, par)\n",
        "    \n",
        "    fn = ret.find(to_add, pred_len)\n",
        "    print(\"FN=\",fn)\n",
        "    if fn > -1:\n",
        "               trim = ret[fn+len(to_add):]\n",
        "               ftotal.write(\"\\r\\n=======\\r\\n\"+trim+\"\\r\\n\")\n",
        "    else: trim = ret               \n",
        "    #total += trim\n",
        "    print(fn)\n",
        "    \n",
        "    total +=ret[new_start:] + endSentenceGeneration(trim)\n",
        "    #total += trim #ret[len(next):] \n",
        "    #total += ret[len(next):] \n",
        "    print(ret)\n",
        "    print(total)\n",
        "    ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "\n",
        "    ftotal.write(total)\n",
        "    ftotal.close()\n",
        "\n",
        "    #Generate in a batch etc. ... \n",
        "    #Interactive ... Keep model loaded etc. - takes a lot of time\n",
        "\n",
        "    '''\n",
        "      f.write(\"\\r\\nbeams = 8 etc. down to 1\\r\\n\")\n",
        "      gn = gen_return(tokenizer, length=150, beams=8, t=1.0, ngram=2, seq=1, text=None)\n",
        "      f.write(''.join(tokenizer.decode(gn)))      \n",
        "      f.write(\"\\r\\n\")\n",
        "      gn = gen_return(tokenizer, length=150, beams=4, t=1.0, ngram=2, seq=1, text=None)\n",
        "      f.write(''.join(tokenizer.decode(gn)))      \n",
        "      f.write(\"\\r\\n\")\n",
        "      gn = gen_return(tokenizer, length=150, beams=2, t=1.0, ngram=2, seq=1, text=None)\n",
        "      f.write(''.join(tokenizer.decode(gn)))      \n",
        "      f.write(\"\\r\\n\")\n",
        "      gn = gen_return(tokenizer, length=150, beams=1, t=1.0, ngram=2, seq=1, text=None)\n",
        "      f.write(''.join(tokenizer.decode(gn)))    \n",
        "      f.write(\"\\r\\n\")\n",
        "      f.close()\n",
        "    '''\n",
        "    \n",
        "\n",
        "\n",
        "def genRecurseTwo():\n",
        "    '''\n",
        "    0 -- characters, names\n",
        "    1 -- locations\n",
        "    2 -- topics\n",
        "    3 -- specifics ...\n",
        "    '''\n",
        "    feed = []\n",
        "    '''\n",
        "    feed.append([[\"Емил\"], [\"Ада\"], [\"Иван\"], [\"Петър\"], [\"Ангел\"], [\"Мария\"]])\n",
        "    feed.append([[\"София\"], [\"България\"], [\"Париж\"], [\"Франция\"]])\n",
        "    feed.append([[\"отивам\"], [\"пристигам\"], [\"заминавам\"], [\"кацам\"], [\"срещна\"], [\"видя\"], [\"целуна\"], [\"прегърна\"]])\n",
        "    feed.append([[\"телевизора\"], [\"колата\"], [\"ракия\"], [\"москвич\"], [\"летище\"], [\"самолет\"], [\"хляб\"], [\"нива\"]])    \n",
        "  \n",
        "    feed.append([[\"Емил\"], [\"Петя\"], [\"Иван\"], [\"Петър\"], [\"Ангел\"], [\"Мария\"]])\n",
        "    feed.append([[\"Лондон\"], [\"България\"], [\"Париж\"], [\"Франция\"]])\n",
        "    feed.append([[\"идвам\"], [\"карам\"],[\"пристигам\"], [\"заминавам\"], [\"кацам\"], [\"срещна\"], [\"видя\"], [\"целуна\"], [\"прегърна\"]])\n",
        "    feed.append([[\"професор\"],[\"университет\"], [\"колата\"], [\"ракия\"], [\"москвич\"], [\"летище\"], [\"самолет\"], [\"хляб\"], [\"нива\"]])\n",
        "    '''\n",
        "    '''\n",
        "    feed.append([[\"Георги\"], [\"говори\"], [\"Иван\"], [\"Петър\"], [\"Ангел\"], [\"Мария\"]])\n",
        "    feed.append([[\"Морето\"], [\"България\"], [\"Париж\"], [\"Франция\"]])\n",
        "    feed.append([[\"целувам\"], [\"красива\"],[\"обичам\"], [\"заминавам\"], [\"кацам\"], [\"срещна\"], [\"видя\"], [\"целуна\"], [\"прегърна\"]])\n",
        "    feed.append([[\"леглото\"],[\"града\"], [\"колата\"], [\"ракия\"], [\"москвич\"], [\"летище\"], [\"самолет\"], [\"хляб\"], [\"нива\"]])\n",
        "    '''\n",
        "   \n",
        "    feed.append([[\"жена\"], [\"процесор\"], [\"Иван\"], [\"Петър\"], [\"Ангел\"], [\"Мария\"]])\n",
        "    #feed.append([[\"процесор\"], [\"България\"], [\"Париж\"], [\"Франция\"], [\"Италия\"], [\"Гърция\"]])\n",
        "    feed.append([[\"обичам\"], [\"България\"], [\"Париж\"], [\"Microsoft\"], [\"Италия\"], [\"Гърция\"]])\n",
        "    feed.append([[\"целувам\"], [\"красива\"],[\"обичам\"], [\"заминавам\"], [\"кацам\"], [\"срещна\"], [\"видя\"], [\"целуна\"], [\"прегърна\"]])\n",
        "    feed.append([[\"леглото\"],[\"града\"], [\"колата\"], [\"вода\"], [\"терминатор\"], [\"летище\"], [\"самолет\"], [\"хляб\"], [\"нива\"]])\n",
        "    \n",
        "    #del feed\n",
        "    #feed = []\n",
        "    \n",
        "    feed.append([[\"AMD\"], [\"процесор\"], [\"Intel\"], [\"секс\"], [\"ускорител\"], [\"Мария\"]])\n",
        "    #feed.append([[\"процесор\"], [\"България\"], [\"Париж\"], [\"Франция\"], [\"Италия\"], [\"Гърция\"]])\n",
        "    feed.append([[\"обичам\"], [\"България\"], [\"Париж\"], [\"Франция\"], [\"Италия\"], [\"1080\"]])\n",
        "    feed.append([[\"тест\"], [\"издание\"],[\"обичам\"], [\"Microsoft\"], [\"съюз\"], [\"система\"], [\"видя\"], [\"целуна\"], [\"прегърна\"]])\n",
        "    feed.append([[\"летя\"],[\"цена\"], [\"3080\"], [\"вода\"], [\"нов\"], [\"летище\"], [\"самолет\"], [\"хляб\"], [\"нива\"]])\n",
        "    \n",
        "    feed = []\n",
        "    '''\n",
        "    feed.append([[\"жена\"], [\"процесор\"], [\"Иван\"], [\"Петър\"], [\"Ангел\"], [\"Мария\"], [\"хляб\"], [\"маса\"]])\n",
        "    #feed.append([[\"процесор\"], [\"България\"], [\"Париж\"], [\"Франция\"], [\"Италия\"], [\"Гърция\"]])\n",
        "    feed.append([[\"обичам\"], [\"България\"], [\"Париж\"], [\"Microsoft\"], [\"Италия\"], [\"Гърция\"], [\"часовник\"], [\"кацам\"]])\n",
        "    feed.append([[\"целувам\"], [\"красива\"],[\"обичам\"], [\"заминавам\"], [\"кацам\"], [\"срещна\"], [\"видя\"], [\"целуна\"], [\"прегърна\"], [\"падам\"]])\n",
        "    feed.append([[\"леглото\"],[\"града\"], [\"колата\"], [\"вода\"], [\"терминатор\"], [\"летище\"], [\"самолет\"], [\"общество\"], [\"нива\"]])\n",
        "    '''\n",
        "    feed.append([[\"жена\"], [\"стая\"], [\"Тодор\"], [\"Терминатор\"], [\"Ангел\"], [\"целувам\"], [\"месо\"], [\"летя\"]])\n",
        "    #feed.append([[\"процесор\"], [\"България\"], [\"Париж\"], [\"Франция\"], [\"Италия\"], [\"Гърция\"]])\n",
        "    feed.append([[\"обичам\"], [\"лягам\"], [\"София\"], [\"Иван\"], [\"Италия\"], [\"гърди\"], [\"часовник\"], [\"кацам\"]])\n",
        "    feed.append([[\"целувам\"], [\"красива\"],[\"обичам\"], [\"победа\"], [\"летище\"], [\"крака\"], [\"мъж\"], [\"целуна\"], [\"прегърна\"], [\"киборг\"]])\n",
        "    feed.append([[\"леглото\"],[\"град\"], [\"колата\"], [\"тенис\"], [\"терминатор\"], [\"оргазъм\"], [\"самолет\"], [\"общество\"], [\"любов\"]])\n",
        "\n",
        "\n",
        "    #москвич\n",
        "    print(len(feed[0]))\n",
        "    #return\n",
        "    #print(feed[0][0], feed[1][0], feed[2][0])\n",
        "    # must be three redirections\n",
        "    #return\n",
        "    #1)\n",
        "    ln = 120\n",
        "    ln_step = 100 #100 #100 #80\n",
        "    maxln = 200#220 #250 #180 less, was 180, to 130 - 3.7.2021\n",
        "    bms = 8 #16\n",
        "    total = ''\n",
        "    now = datetime.now()\n",
        "    date_time = now.strftime(\"%d-%m-%Y_%H-%M-%S\")\n",
        "    fname = \"gen-recurse-total-\" +  date_time + \".txt\"\n",
        "    dir = ''    \n",
        "    #ftotal = open(r\"e:\\gpt\\totalTwo.txt\", \"wt\", encoding='utf-8')\n",
        "    ftotal = open(dir + fname, \"wt\", encoding='utf-8') #+ \"total-\" + \".txt\"\n",
        "    print(type(feed[0][0]),type(feed[1][0]),type(feed[2][0]),type(feed[3][0]))\n",
        "    print(type(feed[0][0][0]),type(feed[1][0]),type(feed[2][0]),type(feed[3][0]))\n",
        "    mix = feed[0][0][0] + \" \" + feed[1][0][0] + \" \" + feed[2][0][0] + \" \" + feed[3][0][0] + endSentence()\n",
        "    sd = set_seed(random.randint(1,9999))\n",
        "    par = (mix, sd, ln, bms, 1.2, 2, 40)\n",
        "    lenmix = len(mix)\n",
        "    last_context = 70 #80 #symbols from the previous stretch  #intermediate?\n",
        "    #LOAD\n",
        "    #'''\n",
        "    #model_dir = r\"Z:\\gpt2\"\n",
        "    #model_dir = r\".\"\n",
        "    ##tokenizer = GPT2Tokenizer.from_pretrained(model_dir)\n",
        "    ##model = TFGPT2LMHeadModel.from_pretrained(model_dir)\n",
        "    ### USE the loaded model, #6-5-2023\n",
        "    tokenizer = tokenizer2\n",
        "    model = model2 \n",
        "    #'''\n",
        "\n",
        "    ret = advancedSingleRet(tokenizer, model, par)\n",
        "    print(ret)\n",
        "    fn = ret.find(mix) #initial\n",
        "    print(\"FN=\",fn)\n",
        "    if fn > -1:\n",
        "               trim = ret[fn+len(mix):]\n",
        "               print(fn+len(mix))\n",
        "               ftotal.write(\"\\r\\n=======================\\r\\n\"+trim+\"\\r\\n\")\n",
        "    else: trim = ret               \n",
        "    total = trim + endSentenceGeneration(trim) #ret[len(mix):]  not+\n",
        "    ftotal.write(\"\\r\\n\"+ret[fn:fn+len(mix)] + \"\\r\\n\" + \"\\r\\n\" + mix + \"\\r\\n=====\\r\\n\")\n",
        "    ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "    \n",
        "    #next = ret[len(mix):] #exclude the beginning already given\n",
        "    next = total\n",
        "    start = len(total)\n",
        "    #to_add =\". \" + feed[0][1] + \" на \" + feed[1][1] + \" тогава \" + feed[2][1] + \"когато \" + feed[3][1]\n",
        "    \n",
        "    #last context + ... should split words etc.\n",
        "    #ADD PRE-\n",
        "    to_add = total[-last_context:] + \" \" + feed[0][1][0] + \" за да \" + feed[1][1][0] + \" понеже \" + feed[2][1][0] + \" когато \" + feed[3][1][0] + endSentence()\n",
        "    #or POST?\n",
        "    #to_add = feed[0][1][0] + \" за да \" + feed[1][1][0] + \" понеже \" + feed[2][1][0] + \" когато \" + feed[3][1][0] + endSentence() + total[-last_context:] + \" \"\n",
        "    #next = next + to_add\n",
        "    next = to_add # only the new words ... + next  #in the beginning in order to smooth the connection\n",
        "    #new_start = len(to_add) + start #trim from here\n",
        "    new_start = len(to_add) # + start #trim from here\n",
        "\n",
        "    ln+=ln_step #should tokenize, check len etc...\n",
        "    len_to_add = len(to_add)    \n",
        "    sd = set_seed(random.randint(1,9999))\n",
        "    temp = 2.0 #1.2\n",
        "    par = (next, sd, ln, bms, temp, 2, 40)\n",
        "    ret = advancedSingleRet(tokenizer,model, par)\n",
        "    \n",
        "    fn = ret.find(to_add) #, len(mix)) #this is not exactly - not precise; there are no repetitions anyway\n",
        "    print(\"FN=\",fn)    \n",
        "    if fn > -1:\n",
        "               trim = ret[fn+len(mix):]\n",
        "               ftotal.write(\"\\r\\n=======\\r\\n\"+trim+\"\\r\\n\")               \n",
        "    else: trim = ret               \n",
        "    #total += trim\n",
        "    total +=ret[new_start:] + endSentenceGeneration(trim)\n",
        "    print(fn)    \n",
        "    \n",
        "    #total += trim #ret[len(next):] \n",
        "    print(ret)\n",
        "    print(total)\n",
        "    ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "\n",
        "    #THIRD    \n",
        "    next = ret[-last_context:]  #ret[len(mix):] #exclude the beginning already given\n",
        "    pred_len = len(next)\n",
        "    #to_add =\". \" + feed[0][1] + \" на \" + feed[1][1] + \" тогава \" + feed[2][1] + \"когато \" + feed[3][1]\n",
        "    to_add = ret[-last_context:] +  \"  \" +feed[0][2][0] + \" към \" + feed[1][2][0] + \" защото \" + feed[2][2][0] + \" голям \" + feed[3][2][0] + endSentence()\n",
        "    #to_add = feed[0][2][0] + \" към \" + feed[1][2][0] + \" защото \" + feed[2][2][0] + \" голям \" + feed[3][2][0] + endSentence() + ret[-last_context:] + \" \"  #PRE new keywords\n",
        "    \n",
        "    #next = next + to_add\n",
        "    next = to_add #+ next\n",
        "    #new_start = len(next) #to_add)\n",
        "    new_start = len(to_add)\n",
        "    #ln+=ln_step\n",
        "    len_to_add = len(to_add) \n",
        "    sd = set_seed(random.randint(1,9999))\n",
        "    par = (next, sd, ln, bms, 1.0, 2, 40) #1.2, 2, 40)\n",
        "    \n",
        "    input_ids = tokenizer.encode(next, return_tensors='tf')\n",
        "    if len(input_ids) > ln: ln = len(input_ids)+lnstep\n",
        "    \n",
        "    ret = advancedSingleRet(tokenizer,model, par)\n",
        "    \n",
        "    fn = ret.find(to_add, pred_len)\n",
        "    print(\"FN=\",fn)\n",
        "    if fn > -1:\n",
        "               trim = ret[fn+len(to_add):]\n",
        "               ftotal.write(\"\\r\\n=======\\r\\n\"+trim+\"\\r\\n\")\n",
        "    else: trim = ret               \n",
        "    #total += trim\n",
        "    print(fn)\n",
        "    \n",
        "    total +=ret[new_start:] + endSentenceGeneration(trim)\n",
        "    #total += trim #ret[len(next):] \n",
        "    #total += ret[len(next):] \n",
        "    print(ret)\n",
        "    print(total)\n",
        "    ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "\n",
        "    choose = \"към,защото,голям,малък,хубав,и,но,ако,с,без,да,където,когато,никога,ха-ха,не мога,мога,искам,не искам,къде,най-голям,стига,нея,аз,ти,той,тя,ние,вие,те\"\n",
        "    spl = choose.split(',')\n",
        "    print(spl)\n",
        "    space = \" \"\n",
        "    for k in range(3,8): #5): #, len(feed[0])):     \n",
        "      print(\"K=\",k)    \n",
        "      #random.choice(seq)¶\n",
        "      #to_add = ret[-last_context:] + \". \" + feed[0][k][0] + \" към \" + feed[1][k][0] + \" защото \" + feed[2][k][0] + \" голям \" + feed[3][k][0] + endSentence()      \n",
        "      rch1 = space + random.choice(spl) + space  \n",
        "      rch2 = space + random.choice(spl) + space\n",
        "      rch3 = space + random.choice(spl) + space\n",
        "      #rch4 = random.choice(seq)\n",
        "      \n",
        "      #step = ret[-last_context:] should split on whole words etc. ... #11-7-2021\n",
        "      #to_add = ret[-last_context:] + \". \" + feed[0][k][0] + \" \" + rch1 + \" \" + feed[1][k][0] + rch2 + feed[2][k][0] + rch3 + feed[3][k][0] + endSentence() \n",
        "      to_add = feed[0][k][0] + \" \" + rch1 + \" \" + feed[1][k][0] + rch2 + feed[2][k][0] + rch3 + feed[3][k][0] + endSentence() + ret[-last_context:] + \" \" # за да не прекъсва изреченията неестествено?\n",
        "      \n",
        "      ftotal.write(\"\\r\\n:TO_ADD=[\"+to_add+\"]\\r\\n\")\n",
        "      next = to_add #+ next\n",
        "      #new_start = len(next) #to_add)\n",
        "      new_start = len(to_add)\n",
        "      #ln+=ln_step\n",
        "      len_to_add = len(to_add) \n",
        "      sd = set_seed(random.randint(1,9999))\n",
        "      \n",
        "      input_ids = tokenizer.encode(next, return_tensors='tf')\n",
        "      if len(input_ids) > ln: ln = len(input_ids)+lnstep   \n",
        "      else: ln = min(ln, maxln)\n",
        "      print(\"LN=\",ln)      \n",
        "      par = (next, sd, ln, bms, 1.2, 2, 40)    \n",
        "      ret = advancedSingleRet(tokenizer,model, par)\n",
        "   \n",
        "      fn = ret.find(to_add, pred_len)\n",
        "      print(\"FN=\",fn)\n",
        "      if fn > -1:\n",
        "                 trim = ret[fn+len(to_add):]\n",
        "                 ftotal.write(\"\\r\\n=======\\r\\n\"+trim+\"\\r\\n\")\n",
        "      else: trim = ret               \n",
        "      #total += trim\n",
        "      print(fn)\n",
        "    \n",
        "      total +=ret[new_start:] + endSentenceGeneration(trim)\n",
        "      #total += trim #ret[len(next):] \n",
        "      #total += ret[len(next):] \n",
        "      print(ret)\n",
        "      print(total)\n",
        "      ftotal.write(\"\\r\\n\"+total+\"\\r\\n\")\n",
        "      ftotal.write(total)      \n",
        " \n",
        "\n",
        "    #Generate in a batch etc. ... \n",
        "    #Interactive ... Keep model loaded etc. - takes a lot of time  \n",
        "    \n",
        "#genRecurse()\n",
        "#genRecurseTwo()\n",
        "####if tokenizer == None: tokenizer = GPT2Tokenizer.from_pretrained(model_dir)\n",
        "####if model == None: model = TFGPT2LMHeadModel.from_pretrained(model_dir)\n",
        "\n",
        "#genRecurseTwo()\n",
        "#advancedMany(tokenizer)\n",
        "#genRecurseTwo()\n",
        "#advancedMany(tokenizer2)\n",
        "model = model2\n",
        "#generate(tokenizer2) #OK! 6-5-2023\n",
        "#advancedMany(tokenizer2)\n",
        "genRecurseTwo()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t-rJzsKGsWWj",
        "outputId": "be128975-da31-404f-96cc-9044310f242d"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "8\n",
            "<class 'list'> <class 'list'> <class 'list'> <class 'list'>\n",
            "<class 'str'> <class 'list'> <class 'list'> <class 'list'>\n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "жена обичам целувам леглото  — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея;\n",
            "FN= 0\n",
            "28\n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FN= 0\n",
            "0\n",
            "деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея;  стая за да лягам понеже красива когато град рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците \n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FN= -1\n",
            "-1\n",
            "е би могло да има на всяка крачка дори и за пет минути поети. Облаците  Тодор към София защото обичам голям колата ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "['към', 'защото', 'голям', 'малък', 'хубав', 'и', 'но', 'ако', 'с', 'без', 'да', 'където', 'когато', 'никога', 'ха-ха', 'не мога', 'мога', 'искам', 'не искам', 'къде', 'най-голям', 'стига', 'нея', 'аз', 'ти', 'той', 'тя', 'ние', 'вие', 'те']\n",
            "K= 3\n",
            "LN= 200\n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FN= -1\n",
            "-1\n",
            "Терминатор  той  Иван те победа най-голям тенис видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "Белите.\n",
            "Тя изглежда е забравила, че съм забравила и някой друг мъж. От нея не става въпрос за любов. Баща ми ме хваща от главата ми и ме насочва към колата на господин Фонг. Поднасям се, печеля втория сет с 6–3. В началото на трети сет Таранго си изкълчва глезена. Започвам да удрям къси топки, карам го да се разхожда по корта с болния си крак. Но той само се преструва. Глезенът му си е наред. Започва да подскача, забива всичките ми къси удари и печели всяка точка. Заедно с тениса двамата с баща ми се справяме с ракетата си и отиваме до кръчмата за тренировки. Затваряме се на тавана и след мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва,\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "Белите.\n",
            "Тя изглежда е забравила, че съм забравила и някой друг мъж. От нея не става въпрос за любов. Баща ми ме хваща от главата ми и ме насочва към колата на господин Фонг. Поднасям се, печеля втория сет с 6–3. В началото на трети сет Таранго си изкълчва глезена. Започвам да удрям къси топки, карам го да се разхожда по корта с болния си крак. Но той само се преструва. Глезенът му си е наред. Започва да подскача, забива всичките ми къси удари и печели всяка точка. Заедно с тениса двамата с баща ми се справяме с ракетата си и отиваме до кръчмата за тренировки. Затваряме се на тавана и след мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва, \n",
            "K= 4\n",
            "LN= 200\n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FN= -1\n",
            "-1\n",
            "Ангел  аз  Италия вие летище тя терминатор лед мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва, \n",
            "Пиколо тук, не е алчен…\n",
            "Ще минем навън, ще видиш!\n",
            "\n",
            "Спусна се по стръмната рътлина на шосето. Пред нея течеше грамада вършини. Наоколо не се виждаше жива душа, но тя бе умряла и той не можеше да й се нарадва.\n",
            "— Хайде, стига! — заповяда й той, а в същото време я прегърна и се загледа към покритата с гъста гора, на която се белееше равна и равна пътека. Тук-там се виждаха само бледосивите тънки силуети на „чашите“. Никой не разбра кога пак са се измъкнали. Другите ги настигнаха. Химикът хвърли на земята скатания брезент на палатката и потегли заедно с Янка, който летеше. Изведнъж той забеляза, че е в гората, дето се намираше нещо гигантско, и\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "Белите.\n",
            "Тя изглежда е забравила, че съм забравила и някой друг мъж. От нея не става въпрос за любов. Баща ми ме хваща от главата ми и ме насочва към колата на господин Фонг. Поднасям се, печеля втория сет с 6–3. В началото на трети сет Таранго си изкълчва глезена. Започвам да удрям къси топки, карам го да се разхожда по корта с болния си крак. Но той само се преструва. Глезенът му си е наред. Започва да подскача, забива всичките ми къси удари и печели всяка точка. Заедно с тениса двамата с баща ми се справяме с ракетата си и отиваме до кръчмата за тренировки. Затваряме се на тавана и след мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва, \n",
            "Пиколо тук, не е алчен…\n",
            "Ще минем навън, ще видиш!\n",
            "\n",
            "Спусна се по стръмната рътлина на шосето. Пред нея течеше грамада вършини. Наоколо не се виждаше жива душа, но тя бе умряла и той не можеше да й се нарадва.\n",
            "— Хайде, стига! — заповяда й той, а в същото време я прегърна и се загледа към покритата с гъста гора, на която се белееше равна и равна пътека. Тук-там се виждаха само бледосивите тънки силуети на „чашите“. Никой не разбра кога пак са се измъкнали. Другите ги настигнаха. Химикът хвърли на земята скатания брезент на палатката и потегли заедно с Янка, който летеше. Изведнъж той забеляза, че е в гората, дето се намираше нещо гигантско, и \n",
            "K= 5\n",
            "LN= 200\n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FN= -1\n",
            "-1\n",
            "целувам  хубав  гърди къде крака малък оргазъм веднъж той забеляза, че е в гората, дето се намираше нещо гигантско, и  по-нататък не съм го виждал. Или може би съм и аз като него.\n",
            "И изведнъж се досетих. Видях в далечината кафявите скали на планината, бялата камбанария на селската църква и на самотната планина. Близо до нея се издигаха високи планини, все още покрити със сняг. Погледнах назад и видях трите коли, които пълзяха, разделени от облаци прах. Минахме покрай дълга върволица натоварени мулета. Водачите им, с червени фесове, вървяха отстрана край животните. Това бяха берсалиери. Един въоръжен с пушка в ръка им извика:\n",
            "— Гери, ела при мене!\n",
            "Северин хвана здравата револвера, но не можа да се стърпи и да види кой е пръв, като вече приближаваше земята. Силите му ръце бяха вързани и краката, защото краката не бяха мокри от ръжени, а краката му\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "Белите.\n",
            "Тя изглежда е забравила, че съм забравила и някой друг мъж. От нея не става въпрос за любов. Баща ми ме хваща от главата ми и ме насочва към колата на господин Фонг. Поднасям се, печеля втория сет с 6–3. В началото на трети сет Таранго си изкълчва глезена. Започвам да удрям къси топки, карам го да се разхожда по корта с болния си крак. Но той само се преструва. Глезенът му си е наред. Започва да подскача, забива всичките ми къси удари и печели всяка точка. Заедно с тениса двамата с баща ми се справяме с ракетата си и отиваме до кръчмата за тренировки. Затваряме се на тавана и след мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва, \n",
            "Пиколо тук, не е алчен…\n",
            "Ще минем навън, ще видиш!\n",
            "\n",
            "Спусна се по стръмната рътлина на шосето. Пред нея течеше грамада вършини. Наоколо не се виждаше жива душа, но тя бе умряла и той не можеше да й се нарадва.\n",
            "— Хайде, стига! — заповяда й той, а в същото време я прегърна и се загледа към покритата с гъста гора, на която се белееше равна и равна пътека. Тук-там се виждаха само бледосивите тънки силуети на „чашите“. Никой не разбра кога пак са се измъкнали. Другите ги настигнаха. Химикът хвърли на земята скатания брезент на палатката и потегли заедно с Янка, който летеше. Изведнъж той забеляза, че е в гората, дето се намираше нещо гигантско, и  по-нататък не съм го виждал. Или може би съм и аз като него.\n",
            "И изведнъж се досетих. Видях в далечината кафявите скали на планината, бялата камбанария на селската църква и на самотната планина. Близо до нея се издигаха високи планини, все още покрити със сняг. Погледнах назад и видях трите коли, които пълзяха, разделени от облаци прах. Минахме покрай дълга върволица натоварени мулета. Водачите им, с червени фесове, вървяха отстрана край животните. Това бяха берсалиери. Един въоръжен с пушка в ръка им извика:\n",
            "— Гери, ела при мене!\n",
            "Северин хвана здравата револвера, но не можа да се стърпи и да види кой е пръв, като вече приближаваше земята. Силите му ръце бяха вързани и краката, защото краката не бяха мокри от ръжени, а краката му \n",
            "K= 6\n",
            "LN= 200\n",
            "Generating...\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50250 for open-end generation.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FN= -1\n",
            "-1\n",
            "месо  той  часовник защото мъж те самолет ързани и краката, защото краката не бяха мокри от ръжени, а краката му  се бяха вкопчили в един ъгъл на друг в празничен ден. Пък и отговорът не беше толкова прост, колкото ми се стори през оня ден в кабинета на доктор Топалов. И аз изобщо нямаше да спомена тоя разговор, ако не се бяхме случили по-късно толкова важни мисли, които за пръв път са преценили. Във всеки случай не на себе си, но все пак имаше нещо подозрително.\n",
            "— А къде се намираме? — запита Яна, навярно прикована отново в някаква кръчма, която намери на пода до самия под и едва тогава си направи труда да помоли турчина. — Добре, ще ви кажа. Но затуй пък…\n",
            "Витан Велев като че ли дори се смали в ръцете му. Той се поусми за миг, после някак се спря, като откри, че не може да се обърне, и промълви:\n",
            "\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "Белите.\n",
            "Тя изглежда е забравила, че съм забравила и някой друг мъж. От нея не става въпрос за любов. Баща ми ме хваща от главата ми и ме насочва към колата на господин Фонг. Поднасям се, печеля втория сет с 6–3. В началото на трети сет Таранго си изкълчва глезена. Започвам да удрям къси топки, карам го да се разхожда по корта с болния си крак. Но той само се преструва. Глезенът му си е наред. Започва да подскача, забива всичките ми къси удари и печели всяка точка. Заедно с тениса двамата с баща ми се справяме с ракетата си и отиваме до кръчмата за тренировки. Затваряме се на тавана и след мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва, \n",
            "Пиколо тук, не е алчен…\n",
            "Ще минем навън, ще видиш!\n",
            "\n",
            "Спусна се по стръмната рътлина на шосето. Пред нея течеше грамада вършини. Наоколо не се виждаше жива душа, но тя бе умряла и той не можеше да й се нарадва.\n",
            "— Хайде, стига! — заповяда й той, а в същото време я прегърна и се загледа към покритата с гъста гора, на която се белееше равна и равна пътека. Тук-там се виждаха само бледосивите тънки силуети на „чашите“. Никой не разбра кога пак са се измъкнали. Другите ги настигнаха. Химикът хвърли на земята скатания брезент на палатката и потегли заедно с Янка, който летеше. Изведнъж той забеляза, че е в гората, дето се намираше нещо гигантско, и  по-нататък не съм го виждал. Или може би съм и аз като него.\n",
            "И изведнъж се досетих. Видях в далечината кафявите скали на планината, бялата камбанария на селската църква и на самотната планина. Близо до нея се издигаха високи планини, все още покрити със сняг. Погледнах назад и видях трите коли, които пълзяха, разделени от облаци прах. Минахме покрай дълга върволица натоварени мулета. Водачите им, с червени фесове, вървяха отстрана край животните. Това бяха берсалиери. Един въоръжен с пушка в ръка им извика:\n",
            "— Гери, ела при мене!\n",
            "Северин хвана здравата револвера, но не можа да се стърпи и да види кой е пръв, като вече приближаваше земята. Силите му ръце бяха вързани и краката, защото краката не бяха мокри от ръжени, а краката му  се бяха вкопчили в един ъгъл на друг в празничен ден. Пък и отговорът не беше толкова прост, колкото ми се стори през оня ден в кабинета на доктор Топалов. И аз изобщо нямаше да спомена тоя разговор, ако не се бяхме случили по-късно толкова важни мисли, които за пръв път са преценили. Във всеки случай не на себе си, но все пак имаше нещо подозрително.\n",
            "— А къде се намираме? — запита Яна, навярно прикована отново в някаква кръчма, която намери на пода до самия под и едва тогава си направи труда да помоли турчина. — Добре, ще ви кажа. Но затуй пък…\n",
            "Витан Велев като че ли дори се смали в ръцете му. Той се поусми за миг, после някак се спря, като откри, че не може да се обърне, и промълви:\n",
            " \n",
            "K= 7\n",
            "LN= 200\n",
            "Generating...\n",
            "FN= -1\n",
            "-1\n",
            "летя  хубав  кацам мога целуна но общество после някак се спря, като откри, че не може да се обърне, и промълви:\n",
            " \n",
            "— Ти ще спиш тук…\n",
            "\n",
            "Сара не отговори веднага. Джинджър я грабна и се опита да я поправи, но Мат залитна и с всичка сила я стовари върху лицето на Мат, улови бузата й и я целуна. Двамата се целунаха, един след друг, в заслепяващ поглед. Тя се извъртя и направи няколко крачки към нея. Пъгсли стоеше в пластмасовия си терариум в замръзналата поза на препариран кожен динозавър. Немигащите му очи се блещеха насреща с хладнокръвното достойнство на влечуго, включено на първа скорост. Мат се втренчи в лъскавия. Не бе намерил онова, което търсеше Сара, измък го и го изтръгна от ледения си край. Блъсна на земята и надникна точно преди да влезе в без да падне на пода.\n",
            "\n",
            " — казва баща ми и майка ми започва да плаче.\n",
            "Прибирам се вкъщи след работа, докато мама помага на баща си. Майка ми разказва, че когато съм бил бебе, влязла в кухнята и заварила баба ми да ме кърми. Оттогава отношенията между двете жени съвсем не вървят. Точно сега не ми се удаде да живея в къща, близо до дома на баба си, която гледаше към къщата. Не знам защо, но този път ми раздрънка. Цялата къща бе пълна с деца, децата също като нея. Аз я харесвах и още ми беше мъчно за нея; рониса. В нея се намираше нейната къща и аз трябваше да я чакам, така че тя никога няма да си стои вкъщи.\n",
            "Крайната река се простираше ливада, оградена с венец от готически дървета. Тук не растяха нито цветя, нито морска трева — само голо пясъчно дъно се разстилаше до най-високата ледена урва. Все по-често се качвах на терасата — така наричахме гладкия покрив на нашия небостъргач. Май че я използувах само аз и няколко домашни котки, които гонеха там гълъбите. От въздушната бездна я отделяше невисока преграда. Избягвах да се приближавам до нея, протягах се на някой от платнените шезлонги, с наслада дишах свежия въздух. Между мен и планината имаше само поле, пресечено като с черен тебешир от елен от шосето, който висеше на скалите. Може би могло да има на всяка крачка дори и за пет минути поети. Облаците ѝ, а тя ми се струва най-хубавото нещо на света – извън града. В момента тя е друга, за която може да се каже, но в нея няма нищо, което да е макар и наполовина интересно в сравнение с човешкия разум.\n",
            "И тъй Бог прогонил Адам и Ева от райската градина!\n",
            "Това бяха последните му думи, които ме напътствуваха от дълги години и оттогава не можах да разбера нито дума. Именно тези мои разсъждения ме огорчаваха още повече, че по време на дългото събрание, когато по-голямата част от нашето семейство съм се родил, аз разпитвах по какъв начин бих могъл да имам щастието да видя отново човек, който по своята природа е съвсем естествен и хармоничен. Запознах се с думите й така, сякаш искаше сам да каже: „Извинете, госпожо, тогава да си вървя само с вас и да ви видя лицето“.\n",
            "— Много ви моля — отговорих аз. След тия думи и с голямо \n",
            "Белите.\n",
            "Тя изглежда е забравила, че съм забравила и някой друг мъж. От нея не става въпрос за любов. Баща ми ме хваща от главата ми и ме насочва към колата на господин Фонг. Поднасям се, печеля втория сет с 6–3. В началото на трети сет Таранго си изкълчва глезена. Започвам да удрям къси топки, карам го да се разхожда по корта с болния си крак. Но той само се преструва. Глезенът му си е наред. Започва да подскача, забива всичките ми къси удари и печели всяка точка. Заедно с тениса двамата с баща ми се справяме с ракетата си и отиваме до кръчмата за тренировки. Затваряме се на тавана и след мен. Лягам на една от лобито вече не продумва. Не е ядосан, казва, \n",
            "Пиколо тук, не е алчен…\n",
            "Ще минем навън, ще видиш!\n",
            "\n",
            "Спусна се по стръмната рътлина на шосето. Пред нея течеше грамада вършини. Наоколо не се виждаше жива душа, но тя бе умряла и той не можеше да й се нарадва.\n",
            "— Хайде, стига! — заповяда й той, а в същото време я прегърна и се загледа към покритата с гъста гора, на която се белееше равна и равна пътека. Тук-там се виждаха само бледосивите тънки силуети на „чашите“. Никой не разбра кога пак са се измъкнали. Другите ги настигнаха. Химикът хвърли на земята скатания брезент на палатката и потегли заедно с Янка, който летеше. Изведнъж той забеляза, че е в гората, дето се намираше нещо гигантско, и  по-нататък не съм го виждал. Или може би съм и аз като него.\n",
            "И изведнъж се досетих. Видях в далечината кафявите скали на планината, бялата камбанария на селската църква и на самотната планина. Близо до нея се издигаха високи планини, все още покрити със сняг. Погледнах назад и видях трите коли, които пълзяха, разделени от облаци прах. Минахме покрай дълга върволица натоварени мулета. Водачите им, с червени фесове, вървяха отстрана край животните. Това бяха берсалиери. Един въоръжен с пушка в ръка им извика:\n",
            "— Гери, ела при мене!\n",
            "Северин хвана здравата револвера, но не можа да се стърпи и да види кой е пръв, като вече приближаваше земята. Силите му ръце бяха вързани и краката, защото краката не бяха мокри от ръжени, а краката му  се бяха вкопчили в един ъгъл на друг в празничен ден. Пък и отговорът не беше толкова прост, колкото ми се стори през оня ден в кабинета на доктор Топалов. И аз изобщо нямаше да спомена тоя разговор, ако не се бяхме случили по-късно толкова важни мисли, които за пръв път са преценили. Във всеки случай не на себе си, но все пак имаше нещо подозрително.\n",
            "— А къде се намираме? — запита Яна, навярно прикована отново в някаква кръчма, която намери на пода до самия под и едва тогава си направи труда да помоли турчина. — Добре, ще ви кажа. Но затуй пък…\n",
            "Витан Велев като че ли дори се смали в ръцете му. Той се поусми за миг, после някак се спря, като откри, че не може да се обърне, и промълви:\n",
            " \n",
            "— Ти ще спиш тук…\n",
            "\n",
            "Сара не отговори веднага. Джинджър я грабна и се опита да я поправи, но Мат залитна и с всичка сила я стовари върху лицето на Мат, улови бузата й и я целуна. Двамата се целунаха, един след друг, в заслепяващ поглед. Тя се извъртя и направи няколко крачки към нея. Пъгсли стоеше в пластмасовия си терариум в замръзналата поза на препариран кожен динозавър. Немигащите му очи се блещеха насреща с хладнокръвното достойнство на влечуго, включено на първа скорост. Мат се втренчи в лъскавия. Не бе намерил онова, което търсеше Сара, измък го и го изтръгна от ледения си край. Блъсна на земята и надникна точно преди да влезе в без да падне на пода.\n",
            " \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "genRecurseTwo()"
      ],
      "metadata": {
        "id": "aAsNnkDn8KBH",
        "outputId": "6a9ddb12-4f14-49f4-f562-8c3c7e2953f1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "8\n",
            "<class 'list'> <class 'list'> <class 'list'> <class 'list'>\n",
            "<class 'str'> <class 'list'> <class 'list'> <class 'list'>\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "HFValidationError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mHFValidationError\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-28-382f4e7bdf73>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mgenRecurseTwo\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m<ipython-input-26-8f01d2e16070>\u001b[0m in \u001b[0;36mgenRecurseTwo\u001b[0;34m()\u001b[0m\n\u001b[1;32m    465\u001b[0m     \u001b[0;31m#'''\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    466\u001b[0m     \u001b[0mmodel_dir\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34mr\"Z:\\gpt2\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 467\u001b[0;31m     \u001b[0mtokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mGPT2Tokenizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_pretrained\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel_dir\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    468\u001b[0m     \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTFGPT2LMHeadModel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_pretrained\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel_dir\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    469\u001b[0m     \u001b[0;31m#'''\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py\u001b[0m in \u001b[0;36mfrom_pretrained\u001b[0;34m(cls, pretrained_model_name_or_path, *init_inputs, **kwargs)\u001b[0m\n\u001b[1;32m   1768\u001b[0m                     \u001b[0mresolved_vocab_files\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfile_id\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdownload_url\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfile_path\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mproxies\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mproxies\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1769\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1770\u001b[0;31m                 resolved_vocab_files[file_id] = cached_file(\n\u001b[0m\u001b[1;32m   1771\u001b[0m                     \u001b[0mpretrained_model_name_or_path\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1772\u001b[0m                     \u001b[0mfile_path\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py\u001b[0m in \u001b[0;36mcached_file\u001b[0;34m(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, use_auth_token, revision, local_files_only, subfolder, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash)\u001b[0m\n\u001b[1;32m    407\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    408\u001b[0m         \u001b[0;31m# Load from URL or cache if already cached\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 409\u001b[0;31m         resolved_file = hf_hub_download(\n\u001b[0m\u001b[1;32m    410\u001b[0m             \u001b[0mpath_or_repo_id\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    411\u001b[0m             \u001b[0mfilename\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py\u001b[0m in \u001b[0;36m_inner_fn\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    110\u001b[0m         ):\n\u001b[1;32m    111\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0marg_name\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m\"repo_id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"from_id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"to_id\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 112\u001b[0;31m                 \u001b[0mvalidate_repo_id\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg_value\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    113\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    114\u001b[0m             \u001b[0;32melif\u001b[0m \u001b[0marg_name\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"token\"\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0marg_value\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py\u001b[0m in \u001b[0;36mvalidate_repo_id\u001b[0;34m(repo_id)\u001b[0m\n\u001b[1;32m    164\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    165\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mREPO_ID_REGEX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrepo_id\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 166\u001b[0;31m         raise HFValidationError(\n\u001b[0m\u001b[1;32m    167\u001b[0m             \u001b[0;34m\"Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    168\u001b[0m             \u001b[0;34m\" forbidden, '-' and '.' cannot start or end the name, max length is 96:\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mHFValidationError\u001b[0m: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'Z:\\gpt2'."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(bytes(tokenizer2.decode(out[0])))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 345
        },
        "id": "_nY5hjRIpmRs",
        "outputId": "8202c400-0e77-45b6-e3f9-19c19dd1e86b"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-13-06923dedba15>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbytes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtokenizer2\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mout\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(self, token_ids, skip_special_tokens, clean_up_tokenization_spaces, **kwargs)\u001b[0m\n\u001b[1;32m   3484\u001b[0m         \u001b[0mtoken_ids\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_py_obj\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtoken_ids\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3485\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3486\u001b[0;31m         return self._decode(\n\u001b[0m\u001b[1;32m   3487\u001b[0m             \u001b[0mtoken_ids\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtoken_ids\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3488\u001b[0m             \u001b[0mskip_special_tokens\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mskip_special_tokens\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py\u001b[0m in \u001b[0;36m_decode\u001b[0;34m(self, token_ids, skip_special_tokens, clean_up_tokenization_spaces, spaces_between_special_tokens, **kwargs)\u001b[0m\n\u001b[1;32m    929\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_decode_use_source_tokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"use_source_tokenizer\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    930\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 931\u001b[0;31m         \u001b[0mfiltered_tokens\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mconvert_ids_to_tokens\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtoken_ids\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mskip_special_tokens\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mskip_special_tokens\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    932\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    933\u001b[0m         \u001b[0;31m# To avoid mixing byte-level and unicode for byte-level BPT\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils.py\u001b[0m in \u001b[0;36mconvert_ids_to_tokens\u001b[0;34m(self, ids, skip_special_tokens)\u001b[0m\n\u001b[1;32m    904\u001b[0m         \u001b[0mtokens\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    905\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mindex\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mids\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 906\u001b[0;31m             \u001b[0mindex\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    907\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mskip_special_tokens\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mindex\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mall_special_ids\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    908\u001b[0m                 \u001b[0;32mcontinue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: int() argument must be a string, a bytes-like object or a real number, not 'list'"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "help (model2.generate)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v4yY33A8l5Dd",
        "outputId": "86f72d3a-d3c9-44a2-8cbb-3e9e87c8a0f5"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Help on method generate in module transformers.generation.tf_utils:\n",
            "\n",
            "generate(inputs: Optional[tensorflow.python.framework.ops.Tensor] = None, generation_config: Optional[transformers.generation.configuration_utils.GenerationConfig] = None, logits_processor: Optional[transformers.generation.tf_logits_process.TFLogitsProcessorList] = None, seed=None, **kwargs) -> Union[transformers.generation.tf_utils.TFGreedySearchEncoderDecoderOutput, transformers.generation.tf_utils.TFGreedySearchDecoderOnlyOutput, transformers.generation.tf_utils.TFSampleEncoderDecoderOutput, transformers.generation.tf_utils.TFSampleDecoderOnlyOutput, transformers.generation.tf_utils.TFBeamSearchEncoderDecoderOutput, transformers.generation.tf_utils.TFBeamSearchDecoderOnlyOutput, transformers.generation.tf_utils.TFBeamSampleEncoderDecoderOutput, transformers.generation.tf_utils.TFBeamSampleDecoderOnlyOutput, transformers.generation.tf_utils.TFContrastiveSearchEncoderDecoderOutput, transformers.generation.tf_utils.TFContrastiveSearchDecoderOnlyOutput, tensorflow.python.framework.ops.Tensor] method of transformers.models.gpt2.modeling_tf_gpt2.TFGPT2LMHeadModel instance\n",
            "    Generates sequences of token ids for models with a language modeling head.\n",
            "    \n",
            "    <Tip warning={true}>\n",
            "    \n",
            "    Most generation-controlling parameters are set in `generation_config` which, if not passed, will be set to the\n",
            "    model's default generation configuration. You can override any `generation_config` by passing the corresponding\n",
            "    parameters to generate, e.g. `.generate(inputs, num_beams=4, do_sample=True)`.\n",
            "    \n",
            "    For an overview of generation strategies and code examples, check out the [following\n",
            "    guide](../generation_strategies).\n",
            "    \n",
            "    </Tip>\n",
            "    \n",
            "    Parameters:\n",
            "        inputs (`tf.Tensor` of varying shape depending on the modality, *optional*):\n",
            "            The sequence used as a prompt for the generation or as model inputs to the encoder. If `None` the\n",
            "            method initializes it with `bos_token_id` and a batch size of 1. For decoder-only models `inputs`\n",
            "            should of in the format of `input_ids`. For encoder-decoder models *inputs* can represent any of\n",
            "            `input_ids`, `input_values`, `input_features`, or `pixel_values`.\n",
            "        generation_config (`~generation.GenerationConfig`, *optional*):\n",
            "            The generation configuration to be used as base parametrization for the generation call. `**kwargs`\n",
            "            passed to generate matching the attributes of `generation_config` will override them. If\n",
            "            `generation_config` is not provided, the default will be used, which had the following loading\n",
            "            priority: 1) from the `generation_config.json` model file, if it exists; 2) from the model\n",
            "            configuration. Please note that unspecified parameters will inherit [`~generation.GenerationConfig`]'s\n",
            "            default values, whose documentation should be checked to parameterize generation.\n",
            "        logits_processor (`LogitsProcessorList`, *optional*):\n",
            "            Custom logits processors that complement the default logits processors built from arguments and\n",
            "            generation config. If a logit processor is passed that is already created with the arguments or a\n",
            "            generation config an error is thrown. This feature is intended for advanced users.\n",
            "        seed (`List[int]`, *optional*):\n",
            "            Random seed to control sampling, containing two integers, used when `do_sample` is `True`. See the\n",
            "            `seed` argument from stateless functions in `tf.random`.\n",
            "        kwargs:\n",
            "            Ad hoc parametrization of `generate_config` and/or additional model-specific kwargs that will be\n",
            "            forwarded to the `forward` function of the model. If the model is an encoder-decoder model, encoder\n",
            "            specific kwargs should not be prefixed and decoder specific kwargs should be prefixed with *decoder_*.\n",
            "    \n",
            "    Return:\n",
            "        [`~utils.ModelOutput`] or `tf.Tensor`: A [`~utils.ModelOutput`] (if `return_dict_in_generate=True` or when\n",
            "        `config.return_dict_in_generate=True`) or a `tf.Tensor`.\n",
            "    \n",
            "            If the model is *not* an encoder-decoder model (`model.config.is_encoder_decoder=False`), the possible\n",
            "            [`~utils.ModelOutput`] types are:\n",
            "    \n",
            "                - [`~generation.TFGreedySearchDecoderOnlyOutput`],\n",
            "                - [`~generation.TFSampleDecoderOnlyOutput`],\n",
            "                - [`~generation.TFBeamSearchDecoderOnlyOutput`],\n",
            "                - [`~generation.TFBeamSampleDecoderOnlyOutput`]\n",
            "    \n",
            "            If the model is an encoder-decoder model (`model.config.is_encoder_decoder=True`), the possible\n",
            "            [`~utils.ModelOutput`] types are:\n",
            "    \n",
            "                - [`~generation.TFGreedySearchEncoderDecoderOutput`],\n",
            "                - [`~generation.TFSampleEncoderDecoderOutput`],\n",
            "                - [`~generation.TFBeamSearchEncoderDecoderOutput`],\n",
            "                - [`~generation.TFBeamSampleEncoderDecoderOutput`]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E_z58GEvGCYw"
      },
      "source": [
        "# Нова секция"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hdUHnSOJBI5r",
        "outputId": "3b0c0fca-68a2-49d5-f9e8-437418217286"
      },
      "source": [
        "for i in paths: print(i)\n",
        "!du -sh /content/ab/2\n",
        "print(len(paths))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/ab/2/Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            "/content/ab/2/ivo_hristov_svruh-obobshteniya_6-2021.txt\n",
            "/content/ab/2/Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            "/content/ab/2/alyosha2.txt\n",
            "/content/ab/2/Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt\n",
            "/content/ab/2/Arthur-Hailey_-_Letishte_-_32-b.txt\n",
            "/content/ab/2/yunaka-i-taz.txt\n",
            "/content/ab/2/George-Orwell_-_1984_-_53-b.txt\n",
            "/content/ab/2/tupanasov.txt\n",
            "/content/ab/2/Hristo Botev - V - . Tova vi chaka! - 3668.txt\n",
            "/content/ab/2/Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            "/content/ab/2/Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            "/content/ab/2/EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            "/content/ab/2/terminator-i-herkules.txt\n",
            "/content/ab/2/Hristo Botev - V - . Epitafii - 3679.txt\n",
            "/content/ab/2/Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt\n",
            "/content/ab/2/Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            "/content/ab/2/kakvomu_notes.txt\n",
            "/content/ab/2/Hristo Botev -  - . Kym brata si - 3228.txt\n",
            "/content/ab/2/Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            "/content/ab/2/talev-svetilnik-nachalo.txt\n",
            "/content/ab/2/upaduk_dzbe.txt\n",
            "/content/ab/2/Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt\n",
            "/content/ab/2/Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt\n",
            "/content/ab/2/Arthur-Hailey_-_Vechernite_novini_-_2402-b.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . Petrushan - 3638.txt\n",
            "/content/ab/2/zmee.txt\n",
            "/content/ab/2/Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt\n",
            "/content/ab/2/Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt\n",
            "/content/ab/2/voyna.txt\n",
            "/content/ab/2/Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            "/content/ab/2/kakvomu.txt\n",
            "/content/ab/2/ritum.txt\n",
            "/content/ab/2/Hristo Botev -  - . Strannik - 3237.txt\n",
            "/content/ab/2/AMD-Intel.txt\n",
            "/content/ab/2/Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . Na proshtavane - 3235.txt\n",
            "/content/ab/2/democratius_stamboliyski.txt\n",
            "/content/ab/2/Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt\n",
            "/content/ab/2/Hristo Botev - V - . Obrezanie na zhenite - 3683.txt\n",
            "/content/ab/2/Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt\n",
            "/content/ab/2/Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Synovnik - 3680.txt\n",
            "/content/ab/2/Hristo Botev -  - . Mojata molitva - 3244.txt\n",
            "/content/ab/2/Hristo Botev -  - . Zashto ne sym_... - 3241.txt\n",
            "/content/ab/2/Anglijski_narodni_prikazki_-_3637-b.txt\n",
            "/content/ab/2/Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Shte se nameri li_ - 3686.txt\n",
            "/content/ab/2/Jurij_Dold-Mihajlik_-_I_sam_voinyt_e_voin_-_7213-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt\n",
            "/content/ab/2/Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "/content/ab/2/hristo-bozhinov.txt\n",
            "/content/ab/2/belezhki-istinata.txt\n",
            "/content/ab/2/Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt\n",
            "/content/ab/2/Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            "/content/ab/2/Petja_Dubarova_-_Az_i_moreto_-_524-b.txt\n",
            "/content/ab/2/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            "/content/ab/2/toj-vyara.txt\n",
            "/content/ab/2/Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            "/content/ab/2/RISC-CISC.txt\n",
            "/content/ab/2/Clean-Kaldata-razdel-bez-html.txt\n",
            "/content/ab/2/Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt\n",
            "/content/ab/2/Hristo Botev -  - . Delba - 3230.txt\n",
            "/content/ab/2/Mitut.txt\n",
            "/content/ab/2/Hristo Botev - V - . Taja nosht synuvah... - 3672.txt\n",
            "/content/ab/2/Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . V mehanata - 3243.txt\n",
            "/content/ab/2/Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt\n",
            "/content/ab/2/Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt\n",
            "/content/ab/2/Hristo Botev -  - . Nej - 3234.txt\n",
            "/content/ab/2/Hristo Botev -  - . Djakon Vasil Levski - 3246.txt\n",
            "/content/ab/2/Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . Gergxovden - 3238.txt\n",
            "/content/ab/2/Hristo Botev - V - . Vazhno neshto - 3681.txt\n",
            "/content/ab/2/Hristo Botev -  - . Hajduti - 3232.txt\n",
            "/content/ab/2/Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt\n",
            "/content/ab/2/john-2.txt\n",
            "/content/ab/2/Hristo Botev -  - . Patriot - 3239.txt\n",
            "/content/ab/2/Boris_Kitanov_-_Razpoznavane_i_sybirane_na_bilki_-_1453-b.txt\n",
            "/content/ab/2/sreshtu-chuzhdico.txt\n",
            "/content/ab/2/Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            "/content/ab/2/kopiraneto-EIM-1-bez-imena.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt\n",
            "/content/ab/2/Hristo Botev - V - . Fabrika za popove - 3673.txt\n",
            "/content/ab/2/Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt\n",
            "/content/ab/2/Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt\n",
            "/content/ab/2/dimov-tyutyun.txt\n",
            "/content/ab/2/kreativen.txt\n",
            "/content/ab/2/Hristo Botev -  - . Zadade se oblak temen - 3245.txt\n",
            "/content/ab/2/Eshil_-_Tragedii_-_334-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . Borba - 3236.txt\n",
            "/content/ab/2/Hristo Botev -  - . Elegija - 3229.txt\n",
            "/content/ab/2/kakvomu_contents.txt\n",
            "/content/ab/2/Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . Smeshen plach - 3637.txt\n",
            "/content/ab/2/Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt\n",
            "/content/ab/2/135-godini.txt\n",
            "/content/ab/2/Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            "/content/ab/2/democratius_prilozhenie_drugi.txt\n",
            "/content/ab/2/seks-glavna.txt\n",
            "/content/ab/2/Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            "/content/ab/2/comment-nerds-2-nerds-128-clean1.txt\n",
            "/content/ab/2/Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            "/content/ab/2/kakvomu_notes_e.txt\n",
            "/content/ab/2/Aleko_Konstantinov_-_Baj_Ganxo_-_1763-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . Hadzhi Dimityr - 3242.txt\n",
            "/content/ab/2/necessary_illusions.txt\n",
            "/content/ab/2/Hristo Botev -  - . Majtse si - 3227.txt\n",
            "/content/ab/2/Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt\n",
            "/content/ab/2/Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Kalyp za uchiteli - 3674.txt\n",
            "/content/ab/2/Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            "/content/ab/2/prehvur-vidyo.txt\n",
            "/content/ab/2/Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            "/content/ab/2/Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt\n",
            "/content/ab/2/Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            "/content/ab/2/Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            "/content/ab/2/Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            "/content/ab/2/Alex-Bradley_-_24_momicheta_za_7_dni_-_6383-b.txt\n",
            "/content/ab/2/Afrikanski_prikazki_-_Prerazkazani_ot_Renata_Pakarie_-_651-b.txt\n",
            "/content/ab/2/pred4.txt\n",
            "/content/ab/2/Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt\n",
            "/content/ab/2/John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Gatanki - 3685.txt\n",
            "/content/ab/2/pred3.txt\n",
            "/content/ab/2/Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt\n",
            "/content/ab/2/Omir_-_Odiseja_-_6123-b.txt\n",
            "/content/ab/2/ezikovi-raboti.txt\n",
            "/content/ab/2/vyarata_i_svobodata.txt\n",
            "/content/ab/2/mainstream_media.txt\n",
            "/content/ab/2/Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            "/content/ab/2/goden.txt\n",
            "/content/ab/2/William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt\n",
            "/content/ab/2/alyosha1.txt\n",
            "/content/ab/2/Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt\n",
            "/content/ab/2/podaruk.txt\n",
            "/content/ab/2/John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            "/content/ab/2/doktoratut.txt\n",
            "/content/ab/2/Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            "/content/ab/2/Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Kazano - 3677.txt\n",
            "/content/ab/2/Anthony-Capella_-_Ako_iskash_da_si_legnesh_s_men..._sgotvi_mi_-_5658-b.txt\n",
            "/content/ab/2/zaklyuchena-poetry.txt\n",
            "/content/ab/2/talev-svetilnik-2-chast.txt\n",
            "/content/ab/2/Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            "/content/ab/2/telefonni-suveti.txt\n",
            "/content/ab/2/Hristo Botev - V - . Politicheska zima - 3666.txt\n",
            "/content/ab/2/Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            "/content/ab/2/Pesen_za_nibelungite_-_6576-b.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt\n",
            "/content/ab/2/Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            "/content/ab/2/Alexandre-Jardin_-_Zebrata_-_6966-b.txt\n",
            "/content/ab/2/Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            "/content/ab/2/Platon_-_Dyrzhavata_-_9825-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . Pristanala - 3233.txt\n",
            "/content/ab/2/Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt\n",
            "/content/ab/2/Hristo Botev - Vnbe - . Namesto programa - 3635.txt\n",
            "/content/ab/2/Hristo Botev - V - . O, tempora! O, mores! - 3669.txt\n",
            "/content/ab/2/Biblija_-_1435-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Videno - 3676.txt\n",
            "/content/ab/2/Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt\n",
            "/content/ab/2/_-  - 2. Biblija - Nov zavet - 2256.txt\n",
            "/content/ab/2/Hristo Botev - V - . Retsept - 3678.txt\n",
            "/content/ab/2/Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt\n",
            "/content/ab/2/Izkustven_RAZUM_UIR_bez-http.txt\n",
            "/content/ab/2/Skandinavska_mitologija_-_8141-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Podslushano - 3675.txt\n",
            "/content/ab/2/Hristo Botev - V - . Dolapyt - 3670.txt\n",
            "/content/ab/2/chlen.txt\n",
            "/content/ab/2/john-1.txt\n",
            "/content/ab/2/Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt\n",
            "/content/ab/2/Hristo Botev -  - . Poslanie - 3240.txt\n",
            "/content/ab/2/Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt\n",
            "/content/ab/2/Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt\n",
            "/content/ab/2/Hristo Botev - V - . Shte vidite - 3687.txt\n",
            "/content/ab/2/razumir-br-4.txt\n",
            "/content/ab/2/DZBE_ponyatiya_7-2020_eim-33.txt\n",
            "97M\t/content/ab/2\n",
            "186\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MNsl6DdLGFXB"
      },
      "source": [],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vOtrahHXzzLM",
        "outputId": "28b9657d-aa68-49b0-aa6a-c5b7bbc078f1"
      },
      "source": [
        "!ls /content/ab/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " 135-godini.txt\n",
            " Ada_12-10_E-potursij-i-po-novi-vtoro-chislo-19-7-2020-Arial-p189_Arial16-Big-Font.txt\n",
            " Alan-Williams_-_Arhivyt_na_Berija_-_Kniga_pyrva_-_10179-b.txt\n",
            " Aldous-Huxley_-_Prekrasnijat_nov_svjat_-_625-b.txt\n",
            "'Aleko Konstantinov -  - . Do Chikago i nazad - 3705.txt'\n",
            " alyosha1.txt\n",
            " alyosha2.txt\n",
            " AMD-Intel.txt\n",
            " Andre-Maurois_-_Bajron_-_3615-b.txt\n",
            " Angel_Karalijchev_-_Bylgarski_narodni_prikazki_-_652-b.txt\n",
            " baladi.txt\n",
            " Balzak-chast.txt\n",
            " belezhki-istinata.txt\n",
            " bivalica.txt\n",
            " Bruder-Grimm_-_Prikazki_-_648-b.txt\n",
            " Bylgarski_narodni_vylshebni_prikazki_-_5768-b.txt\n",
            " Charles-Perrault_-_Chervenata_shapchitsa_i_drugi_prikazki_-_10184-b.txt\n",
            " chlen.txt\n",
            " comment-nerds-2-nerds-128-clean1.txt\n",
            " darenie.txt\n",
            " democratius_prilozhenie_drugi.txt\n",
            " democratius_stamboliyski.txt\n",
            " doktoratut.txt\n",
            " Douglas-Adams_-_Pytevoditel_na_galakticheskija_stopadzhija_-_961-b.txt\n",
            " dve-narechiya.txt\n",
            " DZBE_ponyatiya_7-2020_eim-33.txt\n",
            " EIM-smetach-do-br-6-moje-da-e-nepulno-bg.txt\n",
            " Elin_Pelin_-_Sychinenija_v_shest_toma_tom_peti_-_Jan_Bibijan_Jan_Bibijan_na_Lunata_Prikazki_-_965-b.txt\n",
            " Emilija_Drumeva_-_Konstitutsionno_pravo_-_5956-b.txt\n",
            " Emilijan_Stanev_-_Vylchi_noshti._Kogato_skrezhyt_se_topi._Chernishka_-_Tom_treti_-_691-b.txt\n",
            " Erich-Kastner_-_Kogato_bjah_malko_momche_-_10252-b.txt\n",
            " Ernest-Hemingway_-_Sbogom_na_oryzhijata_-_4411-b.txt\n",
            " Eshil_-_Tragedii_-_334-b.txt\n",
            " ezikovi-raboti.txt\n",
            " Fxodor_Dostoevski_-_Idiot_-_3180-b.txt\n",
            " Fxodor_Dostoevski_-_Prestyplenie_i_nakazanie_-_1445-b.txt\n",
            " Geoffrey-Chaucer_-_Kentyrbyrijski_razkazi_-_4125-b.txt\n",
            " George-Orwell_-_1984_-_53-b.txt\n",
            " Giacomo-Casanova_-_Prikljuchenijata_na_Kazanova_-_Tom_1_-_6751-b.txt\n",
            " Giovanni-Boccaccio_-_Dekameron_-_29-b.txt\n",
            " gnyav.txt\n",
            " goden.txt\n",
            " Gore-Vidal_-_Julian_-_7517-b.txt\n",
            " Gore-Vidal_-_Sytvorenieto_-_987-b.txt\n",
            " Hans-Christian-Andersen_-_Andersenovi_prikazki_-_25-b.txt\n",
            " Hans-Christian-Andersen_-_Andersenovi_prikazki_-_Tom_2_-_7571-b.txt\n",
            " Hiljada_i_edna_nosht_-_Tom_I_-_6537-b.txt\n",
            "'Hristo Botev -  - . Borba - 3236.txt'\n",
            "'Hristo Botev -  - . Delba - 3230.txt'\n",
            "'Hristo Botev -  - . Djakon Vasil Levski - 3246.txt'\n",
            "'Hristo Botev -  - . Do moeto pyrvo libe - 3231.txt'\n",
            "'Hristo Botev -  - . Elegija - 3229.txt'\n",
            "'Hristo Botev -  - . Gergxovden - 3238.txt'\n",
            "'Hristo Botev -  - . Hadzhi Dimityr - 3242.txt'\n",
            "'Hristo Botev -  - . Hajduti - 3232.txt'\n",
            "'Hristo Botev -  - . Kym brata si - 3228.txt'\n",
            "'Hristo Botev -  - . Majtse si - 3227.txt'\n",
            "'Hristo Botev -  - . Mojata molitva - 3244.txt'\n",
            "'Hristo Botev -  - . Na proshtavane - 3235.txt'\n",
            "'Hristo Botev -  - . Nej - 3234.txt'\n",
            "'Hristo Botev -  - . Obesvaneto na Vasil Levski - 3247.txt'\n",
            "'Hristo Botev -  - . Patriot - 3239.txt'\n",
            "'Hristo Botev -  - . Poslanie - 3240.txt'\n",
            "'Hristo Botev -  - . Pristanala - 3233.txt'\n",
            "'Hristo Botev -  - . Strannik - 3237.txt'\n",
            "'Hristo Botev - V - . Da drasnem kibrita na vystanieto - 3695.txt'\n",
            "'Hristo Botev - V - . Dolapyt - 3670.txt'\n",
            "'Hristo Botev - V - . Do redaktsijata na srybskija vestnik ,,Istok_ - 3648.txt'\n",
            "'Hristo Botev - V - . Do redaktsijata na v. ,,Istok_ - 3649.txt'\n",
            "'Hristo Botev - V - . Edinstvenoto spasenie na nashija narod e v revoljutsijata - 3644.txt'\n",
            "'Hristo Botev - V - . Epitafii - 3679.txt'\n",
            "'Hristo Botev - V - . Fabrika za popove - 3673.txt'\n",
            "'Hristo Botev - V - . Gatanki - 3685.txt'\n",
            "'Hristo Botev - V - . Hajde na izlozhbata! - 3671.txt'\n",
            "'Hristo Botev - V - . Izlechima li e nashata bolest - 3698.txt'\n",
            "'Hristo Botev - V - . Iztochnijat vypros lezhi na pleshtite na bylgarskija narod - 3642.txt'\n",
            "'Hristo Botev - V - . Kakvo mozhe da napravi nasheto duhovenstvo - 3697.txt'\n",
            "'Hristo Botev - V - . Kalyp za uchiteli - 3674.txt'\n",
            "'Hristo Botev - V - . Kazano - 3677.txt'\n",
            "'Hristo Botev -  - . V mehanata - 3243.txt'\n",
            "'Hristo Botev - Vnbe - . ,,Jugoslavija_ - 3640.txt'\n",
            "'Hristo Botev - Vnbe - . Namesto programa - 3635.txt'\n",
            "'Hristo Botev - Vnbe - . Narodyt - vchera, dnes i utre - 3636.txt'\n",
            "'Hristo Botev - Vnbe - . Petrushan - 3638.txt'\n",
            "'Hristo Botev - Vnbe - . Primeri ot turskoto pravosydie - 3657.txt'\n",
            "'Hristo Botev - Vnbe - . Reshen li e cherkovnijat vypros_ - 3639.txt'\n",
            "'Hristo Botev - Vnbe - . Smeshen plach - 3637.txt'\n",
            "'Hristo Botev - V - . Obrezanie na zhenite - 3683.txt'\n",
            "'Hristo Botev - V - . O, tempora! O, mores! - 3669.txt'\n",
            "'Hristo Botev - V - . Pesen na bukureshtkite bylgarski ergene - 3684.txt'\n",
            "'Hristo Botev - V - . Podslushano - 3675.txt'\n",
            "'Hristo Botev - V - . Politicheska zima - 3666.txt'\n",
            "'Hristo Botev - V - . Polozhenieto na bylgarite v Besarabija - 3650.txt'\n",
            "'Hristo Botev - V - . Praznuvaneto na Kiril i Metodij - 3699.txt'\n",
            "'Hristo Botev - V - . Proljatata za svoboda kryv shte izmie plemennite razdori - 3643.txt'\n",
            "'Hristo Botev - V - . Razdorite mezhdu balkanskite narodi - 3645.txt'\n",
            "'Hristo Botev - V - . Retsept - 3678.txt'\n",
            "'Hristo Botev - V - . Revoljutsija narodna, nezabavna, otchajana - 3688.txt'\n",
            "'Hristo Botev - V - . Revoljutsija v Krajovo - 3682.txt'\n",
            "'Hristo Botev - V - . Samo razumnijat i bratskijat syjuz mezhdu narodite shte unishtozhat teglilata - 3647.txt'\n",
            "'Hristo Botev - V - . Shte se nameri li_ - 3686.txt'\n",
            "'Hristo Botev - V - . Shte vidite - 3687.txt'\n",
            "'Hristo Botev - V - . Srybskata propaganda mezhdu bylgarskoto naselenie i opasnostta ot neja - 3651.txt'\n",
            "'Hristo Botev - V - . Synovnik - 3680.txt'\n",
            "'Hristo Botev - V - . Syrbija i nasheto osvoboditelno dvizhenie - 3646.txt'\n",
            "'Hristo Botev - V - . Taja nosht synuvah... - 3672.txt'\n",
            "'Hristo Botev - V - . Tova vi chaka! - 3668.txt'\n",
            "'Hristo Botev - V - . Turtsija shte se nameri mezhdu tri ogynja - 3696.txt'\n",
            "'Hristo Botev - V - . Vazhno neshto - 3681.txt'\n",
            "'Hristo Botev - V - . Videno - 3676.txt'\n",
            "'Hristo Botev - V - . V zatvorenata pechatnitsa - 3689.txt'\n",
            "'Hristo Botev - V - . Zadachata na v. ,,Zname_ - 3641.txt'\n",
            "'Hristo Botev -  - . Zadade se oblak temen - 3245.txt'\n",
            "'Hristo Botev -  - . Zashto ne sym_... - 3241.txt'\n",
            " hristo-bozhinov.txt\n",
            " Hristo_Kalchev_-_Synjat_na_umorenija_lyv_-_vulgaren_roman_-_5935-b.txt\n",
            " igrachi-yunaci.txt\n",
            " igrach-mekichar.txt\n",
            " igrach-na.txt\n",
            " igrachut-i-lyubovta.txt\n",
            " igrachut.txt\n",
            " igri-za.txt\n",
            " Irvine-Welsh_-_Porno_-_2467-b.txt\n",
            " Isaac-Asimov_-_Fondatsija_i_Imperija_-_1022-b.txt\n",
            " Isaac-Asimov_-_Fondatsija_i_Zemja_-_1025-b.txt\n",
            " Isaac-Asimov_-_Fondatsijata_-_1021-b.txt\n",
            " Isaac-Asimov_-_Ostrieto_na_Fondatsijata_-_1024-b.txt\n",
            " Isaac-Asimov_-_Robotite_i_imperijata_-_1200-b.txt\n",
            " Isaac-Asimov_-_Samite_bogove_-_1197-b.txt\n",
            " Isaac-Asimov_-_Vtorata_Fondatsija_-_1023-b.txt\n",
            " Ivan_Asenov_Liljana_Rajnova_Diana_Panova_Dushka_Staneva_-_Bilkite_vyv_vseki_dom_-_1455-b.txt\n",
            " Ivan_Hadzhijski_-_Optimistichna_teorija_za_nashija_narod_-_7943-b.txt\n",
            " Ivan_Stojanov_-_Istorija_na_Bylgarskoto_vyzrazhdane_-_1955-b.txt\n",
            " Ivan_Vazov_-_Razkazi_1881-1901_-_508-b.txt\n",
            " Ivan_Vazov_-_Razkazi_1901-1921_-_509-b.txt\n",
            "'Ivan Vazov - Tc - 3. Nova zemja - 3754.txt'\n",
            " Ivan_Vazov_-_Utro_v_Banki_-_515-b.txt\n",
            " ivo_hristov_svruh-obobshteniya_6-2021.txt\n",
            "'Izkustven_15-4-2020_20-6-2020-_29-8-2020-sled-PURVA_CHAST_SAMO_8-i-15-godini_11-17_bez_vruzka_failove(4)_8-12-2020_Dovurshi-28-2-20210.odt_1.txt'\n",
            " Jack-London_-_Skitnikyt_mezhdu_zvezdite_-_1032-b.txt\n",
            " Jennifer-Blake_-_Avantjuristkata_-_3147-b.txt\n",
            " Joan-D-Vinge_-_Kauboi_i_izvynzemni_-_10199-b.txt\n",
            " john-1.txt\n",
            " john-2.txt\n",
            " John-Braine_-_Pyt_kym_vissheto_obshtestvo_-_4406-b.txt\n",
            " John-Braine_-_Zhivot_vyv_vissheto_obshtestvo_-_4407-b.txt\n",
            " John-Man-Chris-Schuller-Geoffrey-Roy-Nigel-Rodgers-Mary-Ann_-_Svetovna_entsiklopedija_na_pyteshestvenika_-_9817-b.txt\n",
            " John-R-R-Tolkien_-_Vlastelinyt_na_Prystenite_-_1204-b.txt\n",
            " Jordan-Belfort_-_Vylka_ot_Uolstrijt_-_6689-b.txt\n",
            " kakvomu_contents.txt\n",
            " kakvomu_notes_e.txt\n",
            " kakvomu_notes.txt\n",
            " kakvomu.txt\n",
            " Karl-May_-_Vinetu_I_-_1325-b.txt\n",
            " Karl-May_-_Vinetu_II_-_1326-b.txt\n",
            " Kathy-Lette_-_Ako_vibratorite_imaha_klepachi_-_6297-b.txt\n",
            " Konfutsij_-_Dobrijat_pyt_-_Misli_na_velikija_kitajski_mydrets_-_1984-b.txt\n",
            " kopiraneto-EIM-1-bez-imena.txt\n",
            " kreativen.txt\n",
            " Krystxo_Manchev_-_Istorija_na_balkanskite_narodi._Tom_1_-_1352-1878_-_1962-b.txt\n",
            " Lev_Tolstoj_-_Vojna_i_mir_-_Treti_i_chetvyrti_tom_-_4404-b.txt\n",
            " Liljana_Anastasova_-_Narychnik_za_dobri_obnoski_-_Vazhni_uslovija_za_dobyr_ton_v_obshtestvoto_-_1822-b.txt\n",
            " Lillian-Glass_-_Toj_kazva_tja_kazva_-_Kak_da_preodoleem_propastta_v_obshtuvaneto_mezhdu_polovete_-_7266-b.txt\n",
            " Ljuben_Dilov_-_Mnogoto_imena_na_straha_-_3560-b.txt\n",
            " Ljuben_Dilov_-_Pytjat_na_Ikar_-_2963-b.txt\n",
            " Ljubomir_Nikolov_-_Konan_i_igrata_na_bogovete_-_5334-b.txt\n",
            " Ljubomir_T._Pirovski_Nikola_L._Pirovski_Jordanka_Stajkova-Pirovska_-_Iskam_-_moga_-_trjabva_-_Holistichen_model_na_zdraveto_-_10208-b.txt\n",
            " mainstream_media.txt\n",
            " Michael-H-Hart_-_100-te_naj-vlijatelni_lichnosti_v_istorijata_na_chovechestvoto_-_1652-b.txt\n",
            " Mike-Gerrard_-_Entsiklopedija_na_pyteshestvenika_Evropa_-_9980-b.txt\n",
            " Mitut.txt\n",
            " Musketari-chast.txt\n",
            " necessary_illusions.txt\n",
            " Nikolaj_Gogol_-_Myrtvi_dushi_-_67-b.txt\n",
            " Nikolaj_Kun_-_Starogrytski_legendi_i_mitove_-_1492-b.txt\n",
            " Nikola_M._Nikolov_-_Svetovnata_konspiratsija_-_Tova_koeto_istoritsite_ne_vi_kazvat_-_3694-b.txt\n",
            " Norman-Polmar-Thomas-B-Allen_-_Entsiklopedija_na_shpionazha_-_3604-b.txt\n",
            " Omir_-_Odiseja_-_6123-b.txt\n",
            " Pavel_Vezhinov_-_Gibelta_na_Ajaks_-_1617-b.txt\n",
            " Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_1_-_Razkazi_i_noveli_-_10018-b.txt\n",
            " Pavel_Vezhinov_-_Izbrani_proizvedenija_v_chetiri_toma._Tom_2_-_Povesti_-_4421-b.txt\n",
            " Pavel_Vezhinov_-_Vezni_-_10103-b.txt\n",
            " Pesen_za_nibelungite_-_6576-b.txt\n",
            " Petja_Dubarova_-_Az_i_moreto_-_524-b.txt\n",
            " Petyr_Konstantinov_-_Istorija_na_Bylgarija_s_njakoi_premylchavani_dosega_istoricheski_fakti_681-2001_-_1644-b.txt\n",
            " Pierre-Grimal_-_Prikazki_i_legendi_ot_dreven_Shumer_-_5257-b.txt\n",
            " Pierre-Louys_-_Afrodita._Antichni_nravi_-_6786-b.txt\n",
            " Platon_-_Dyrzhavata_-_9825-b.txt\n",
            " podaruk.txt\n",
            " podvizi-akro-mezo-stih.txt\n",
            " pred3.txt\n",
            " pred4.txt\n",
            " prehvur-vidyo.txt\n",
            " Prikazki_na_stranite_s_narodna_demokratsija_-_10226-b.txt\n",
            " radost.txt\n",
            " Rainer-Buttner_-_ALF_-_Njama_problemi_-_5227-b.txt\n",
            " razumir-br-4.txt\n",
            " rechnik.txt\n",
            " RISC-CISC.txt\n",
            " ritum.txt\n",
            " Robert-M-Pirsig_-_Dzen_i_izkustvoto_da_se_poddyrzha_mototsiklet_-_Izsledvane_na_stojnostite_-_2110-b.txt\n",
            " seks-glavna.txt\n",
            " seksut.txt\n",
            " Skandinavska_mitologija_-_8141-b.txt\n",
            " Srednovekovni_kitajski_noveli_-_8142-b.txt\n",
            " sreshtu-chuzhdico.txt\n",
            " StanisYaw-Lem_-_Edem_-_1368-b.txt\n",
            " StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_pyrvi_-_Iz_Zvezdni_dnevnitsi._Iz_Iz_Spomenite_na_Ijon_Tihi._Solaris_-_6195-b.txt\n",
            " StanisYaw-Lem_-_Izbrani_fantastichni_proizvedenija_v_dva_toma._Tom_vtori_-_Nepobedimijat._Iz_Prikazki_za_roboti._Iz_Kiberiada._137_sekundi._Maska._Iz_.txt\n",
            " StanisYaw-Lem_-_Prikljuchenijata_na_zvezdnija_navigator_Pirks_-_470-b.txt\n",
            " stari.txt\n",
            " Stella-Cameron_-_Angelyt_i_grehyt_-_2832-b.txt\n",
            " Stephen-King_-_Bjagashtijat_chovek_-_1257-b.txt\n",
            " stranichni.txt\n",
            " Sun_Dzy_-_Izkustvoto_na_vojnata_-_1696-b.txt\n",
            " telefonni-suveti.txt\n",
            " terminator-i-herkules.txt\n",
            " Terry-Pratchett_-_Dzhoni_i_bombata_-_4211-b.txt\n",
            " toj-vyara.txt\n",
            " trus.txt\n",
            " tupanasov.txt\n",
            " Umberto-Eco_-_Imeto_na_rozata_-_2000-b.txt\n",
            " upaduk_dzbe.txt\n",
            " Vladimir_Nabokov_-_Lolita_-_275-b.txt\n",
            " voyna.txt\n",
            " vyarata_i_svobodata.txt\n",
            " William-Shakespeare_-_Sybrani_sychinenija_-_Tom_3_-_Tragedii_-_4392-b.txt\n",
            " yunaka-i-taz.txt\n",
            " Zahari_Stojanov_-_Hristo_Botjov_-_Opit_za_biografija_-_1770-b.txt\n",
            " Zahari_Stojanov_-_Zapiski_po_bylgarskite_vystanija_-_1771-b.txt\n",
            " zaklyuchena-poetry.txt\n",
            " zmee.txt\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}