AWBW_Replay_Parser_and_Kantbeis_Replay_Archive.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "AWBW Replay Parser and Kantbei's Replay Archive",
      "provenance": [],
      "mount_file_id": "1ovOfMGd6L1hYiZhPUC4yClE_bh9BW0E-",
      "authorship_tag": "ABX9TyOWReY5Kq797zy5AWWiNWt5",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TarkanAl-Kazily/awbw_replay_parser/blob/main/AWBW_Replay_Parser_and_Kantbeis_Replay_Archive.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Running the AWBW Replay Parser toolset on Kantbei's Replay Archive\n",
        "\n",
        "This notebook runs the AWBW Replay Parser to open and load replays from Kantbei's Replay Archive. It's primarily configured to load all the replays in a batch to check for parsing errors or other errors, but it can be used as a base for a strategy analysis on the high tier games in the archive.\n",
        "\n",
        "See the main project on Github here: [https://github.com/TarkanAl-Kazily/awbw_replay_parser](https://github.com/TarkanAl-Kazily/awbw_replay_parser)\n",
        "\n",
        "Access Kantbei's dataset from Google Drive here: [https://drive.google.com/drive/folders/1ziWc1mVm9lOydIquCG8EsTb3aYvpdaPp?usp=sharing](https://drive.google.com/drive/folders/1ziWc1mVm9lOydIquCG8EsTb3aYvpdaPp?usp=sharing)"
      ],
      "metadata": {
        "id": "vLnEGbkCr5wE"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c6b00ryyryGl"
      },
      "outputs": [],
      "source": [
        "# One time setup for this notebook\n",
        "import os\n",
        "\n",
        "os.environ['REPO'] = \"https://github.com/TarkanAl-Kazily/awbw_replay_parser.git\"\n",
        "os.environ['BRANCH'] = \"main\"\n",
        "\n",
        "# Clone the awbw_replay_parser repository\n",
        "!cd /content/\n",
        "!rm -rf awbw_replay_parser\n",
        "!git clone --branch $BRANCH $REPO\n",
        "!pip install -r awbw_replay_parser/requirements.txt\n",
        "\n",
        "# Install the awbw_replay module into our runtime\n",
        "!cd awbw_replay_parser && python setup.py install"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "To use Kantbei's dataset, you need to add a shortcut to the directory to your drive through Google Drive. Then from the Google Colab Files sidebar you can mount your Google Drive, and the data should be accessible under `drive/MyDrive/Kantbei's AWBW Replay Archive`."
      ],
      "metadata": {
        "id": "If99s-TY2cFd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from logging import exception\n",
        "import os\n",
        "import glob\n",
        "from awbw_replay.replay import AWBWReplay\n",
        "from awbw_replay.awbw import AWBWGameState, AWBWGameAction\n",
        "\n",
        "KANTBEI_REPLAY_DIR = \"/content/drive/MyDrive/Kantbei's AWBW Replay Archive\"\n",
        "\n",
        "# Glob all the replay archive files. Replace this list with the specific files you'd like to load.\n",
        "replay_list = glob.glob(os.path.join(KANTBEI_REPLAY_DIR, \"*.zip\"))\n",
        "\n",
        "replay_stats = {}\n",
        "\n",
        "def attempt_parsing_replay(filename):\n",
        "  \"\"\"\n",
        "  Tries to parse a given replay file. Returns a dictionary summarizing the result.\n",
        "  \n",
        "  Handles all exceptions, making it safe to continue running afterwards.\n",
        "  \"\"\"\n",
        "  result = {\n",
        "    \"filename\": filename,\n",
        "    \"actions\": None, # Count how many actions are in the file\n",
        "    \"turns\": None, # Count how many turns are in the file\n",
        "    \"states\": None, # Count how many states we could iterate through\n",
        "    \"players\": None, # Count the number of players in the file\n",
        "    \"exception\": None, # The exception string in the case of failure\n",
        "    \"exception_obj\": None, # The actual exception object\n",
        "    \"success\": False, # If we reached the end of parsing all states\n",
        "  }\n",
        "  try:\n",
        "    with AWBWReplay(filename) as replay:\n",
        "      actions = list(replay.actions())\n",
        "      result[\"actions\"] = len(actions)\n",
        "      result[\"turns\"] = len(replay.turns())\n",
        "      result[\"states\"] = 0\n",
        "      state = AWBWGameState(replay_initial=replay.game_info())\n",
        "      result[\"states\"] += 1\n",
        "      result[\"players\"] = len(state.players)\n",
        "      for action in actions:\n",
        "        state = state.apply_action(AWBWGameAction(action))\n",
        "        result[\"states\"] += 1\n",
        "\n",
        "      result[\"success\"] = True\n",
        "  except Exception as e:\n",
        "    result[\"exception\"] = str(e)\n",
        "    result[\"exception_obj\"] = e\n",
        "    print(f\"Got exception: {e}\")\n",
        "  \n",
        "  return result\n",
        "\n",
        "for filename in replay_list:\n",
        "  if filename in replay_stats and replay_stats[filename][\"success\"]:\n",
        "    print(f\"Skipping already successful file {filename}\")\n",
        "    continue\n",
        "  print(f\"Testing file {filename}...\")\n",
        "  replay_stats[filename] = attempt_parsing_replay(filename)\n",
        "\n",
        "successful_replays = [stat[\"filename\"] for stat in replay_stats.values() if stat[\"success\"]]\n",
        "\n",
        "print(f\"Successfully parsed states from {len(successful_replays)} files (out of {len(replay_list)}).\")"
      ],
      "metadata": {
        "id": "i4v86dB-28_7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "error_list = [result for result in replay_stats.values() if not result[\"success\"]]\n",
        "\n",
        "for result in error_list:\n",
        "  print(f\"{result['filename']} - {result['exception']} at state {result['states']}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TgEV4feVDKaU",
        "outputId": "26c47212-5b32-4959-9203-2fa3b59f6138"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/039 League (Std) p0.32451 (Kindle) vs SonjaTheSuperior (Kindle) - SeaShoal Ocarina.zip - 83532217 at state 309\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/069 League (Fog) shoeLv3 (Hawke) vs Hevol (Hawke) - To Athos.zip - Not a gzipped file (b'PK') at state None\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/135 League (Fog) RiceMuncher (Rachel) vs Kevin or hell (Kindle) - Stolen Seconds.zip - 'nextPId' at state 1524\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/145 League (Std) Djandjan (Eagle) vs starflash250 (Olaf) - From Wharf to Wilds.zip -  at state 61\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/196 League (Fog) Chinese Name (Sonja) vs eagle. (Sami) - Mors Celeri.zip - Error -3 while decompressing data: invalid distance too far back at state None\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/214 League (Std) Vechi (Grimm) vs the FANG (Adder) - Dubious Procession.zip -  at state 73\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/234 League (Std) Vechi (Grimm) vs the FANG (Adder) - Dubious Procession.zip -  at state 73\n",
            "/content/drive/MyDrive/Kantbei's AWBW Replay Archive/258 Custom (Fog Team) - Thirty Years War.zip - invalid literal for int() with base 10: 'B' at state 1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "for result in error_list:\n",
        "  print(f\"\\\"{result['filename']}\\\",\")"
      ],
      "metadata": {
        "id": "VoZgxbDolf2-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ce154d81-56a1-44b2-ca7e-cbd95098cf33"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/039 League (Std) p0.32451 (Kindle) vs SonjaTheSuperior (Kindle) - SeaShoal Ocarina.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/069 League (Fog) shoeLv3 (Hawke) vs Hevol (Hawke) - To Athos.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/135 League (Fog) RiceMuncher (Rachel) vs Kevin or hell (Kindle) - Stolen Seconds.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/145 League (Std) Djandjan (Eagle) vs starflash250 (Olaf) - From Wharf to Wilds.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/196 League (Fog) Chinese Name (Sonja) vs eagle. (Sami) - Mors Celeri.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/214 League (Std) Vechi (Grimm) vs the FANG (Adder) - Dubious Procession.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/234 League (Std) Vechi (Grimm) vs the FANG (Adder) - Dubious Procession.zip\",\n",
            "\"/content/drive/MyDrive/Kantbei's AWBW Replay Archive/258 Custom (Fog Team) - Thirty Years War.zip\",\n"
          ]
        }
      ]
    }
  ]
}