examples/laser-dolphin-mixtral-2x7b.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fb9a062f",
   "metadata": {},
   "source": [
    "# laser-dolphin-mixtral-2x7b-dpo example\n",
    "\n",
    "To recreate laser-dolphin-mixtral-2x7-dpo there are two choices for how you can achieve this.\n",
    "\n",
    "1. Merge two models and then apply the laser process to the merged product.\n",
    "\n",
    "2. Laser two separate models first and then merge them.\n",
    "\n",
    "\n",
    "The original laser-dolphin-mixtral-2x7-dpo model utilizes the first method. However, instructions for both methods are provided for completeness.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcd795df",
   "metadata": {},
   "source": [
    "#### General notes\n",
    "\n",
    "+ **Colab Compatibility**: This script encounters issues when run in Colab, specifically with a shell script error related to the -f 5 flag (lm_eval: error: unrecognized arguments: -f 5). It functions correctly in local environments, including Jupyter and VS Code.\n",
    "\n",
    "+ **Scalability for Multiple Experts**: Method 2 is scalable for any number of experts, limited only by available compute resources. Simply add more models with appropriate positive prompts.\n",
    "\n",
    "+ **Handling VRAM Limitations**: For environments with VRAM constraints, consider using the load_in_4bit flag:\n",
    "\n",
    "```bash\n",
    "lm_eval --model hf \\\n",
    "    --model_args pretrained=$MODEL_NAME,trust_remote_code=True,load_in_4bit=True \\\n",
    "    --tasks mmlu -f 5 \\\n",
    "    --device cuda:0 \\\n",
    "    --batch_size 1\n",
    "```\n",
    "\n",
    "**Not recommneded for optimal results but it should work to use the script"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a6364d1",
   "metadata": {},
   "source": [
    "+ **Repository Setup**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19510932",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/cognitivecomputations/laserRMT.git\n",
    "%cd laserRMT\n",
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b5a11b9",
   "metadata": {},
   "source": [
    "# Method 1\n",
    "\n",
    "I made a dolphin-mixtral-2x7b that has not been lasered for the demo, but the original repo is ```macadeliccc/laser-dolphin-mixtral-2x7b-dpo``` as well as the 4x7b variant.\n",
    "\n",
    "+ **Execution commands:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ab0619a",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python rmt_laser.py \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f821b825-5f16-492e-9708-87510faed096",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python rmt_laser_snr.py \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9fc437b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python rmt_laser_snr_math.py \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6bf0e374",
   "metadata": {},
   "outputs": [],
   "source": [
    "!chmod +x ./script_lm_eval.sh\n",
    "!./script_lm_eval.sh \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dde459b9",
   "metadata": {},
   "source": [
    "The output of Method 1 is the lasered version of dolphin-mixtral-2x7b.\n",
    "\n",
    "The original \"pre-lasered\" version of the model is available [here](https://huggingface.co/macadeliccc/laser-dolphin-mixtral-2x7b-dpo)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7ecc558",
   "metadata": {},
   "source": [
    "# Method 2\n",
    "\n",
    "This method involves steps that extend beyond the scope of the laserRMT repository. Only essential information and resources are included for brevity.\n",
    "\n",
    "+ **Step 1:** Select your two models, use the context of the base model as your reference. You should not merge models with vastly different context for this method.\n",
    "+ **Step 2:** Laser each model individually\n",
    "+ **Step 3:** Merge the models using the mixtral branch of mergekit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50264047",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python rmt_laser.py \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "658400e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "!python rmt_laser.py \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bb502c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "!./script_lm_eval.sh \"cognitivecomputations/dolphin-2.1-mistral-7b\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37f042d9",
   "metadata": {},
   "source": [
    "Run these cells separately. This process will likely take several hours."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a1ab6d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "!./script_lm_eval.sh \"teknium/OpenHermes-2.5-Mistral-7B\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef86f527",
   "metadata": {},
   "source": [
    "This is where the notebook diverges from laserRMT scope.\n",
    "\n",
    "Once you have completed two successful lasers, you are now ready to begin the merge process.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "828a2d19",
   "metadata": {},
   "outputs": [],
   "source": [
    "%cd ..\n",
    "!git clone --branch mixtral https://github.com/cg123/mergekit.git\n",
    "%cd mergekit\n",
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4be0acc",
   "metadata": {},
   "source": [
    "Create your ```config.yaml``` file\n",
    "\n",
    "If you want more information on this process you can find it [here](https://github.com/cg123/mergekit/blob/4de2a3310eb135363d6588e92f2ba5fb20893361/moe.md)\n",
    "\n",
    "+ **Example config**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6a4d030",
   "metadata": {},
   "outputs": [],
   "source": [
    "base_model: cognitivecomputations/dolphin-2.6-mistral-7b-dpo\n",
    "gate_mode: hidden\n",
    "dtype: bfloat16\n",
    "experts:\n",
    "  - source_model: teknium/OpenHermes-2.5-Mistral-7B\n",
    "    positive_prompts:\n",
    "      - \"instruction\"\n",
    "      - \"solutions\"\n",
    "      - \"chat\"\n",
    "      - \"questions\"\n",
    "      - \"comprehension\"\n",
    "      \n",
    "  - source_model: cognitivecomputations/dolphin-2.6-mistral-7b-dpo\n",
    "    positive_prompts:\n",
    "      - \"mathematics\"\n",
    "      - \"optimization\"\n",
    "      - \"code\"\n",
    "      - \"step-by-step\"\n",
    "      - \"science\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10df45c5",
   "metadata": {},
   "source": [
    "Place your config.yml file in the mergekit/examples directory or wherever you would like it to be.\n",
    "\n",
    "+ **Run the command**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67d7cdcd",
   "metadata": {},
   "outputs": [],
   "source": [
    "mergekit-moe examples/your-config.yml ./your-output-directory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74e4081b",
   "metadata": {},
   "source": [
    "The result of this method is a 2x7b mixtral model that consists of two lasered mistral models.\n",
    "\n",
    "If you wish to evaluate the model afterwards you can use my [colab](https://colab.research.google.com/drive/1FpwgsGzCR4tORTxAwUxpN3PcP22En2xk#scrollTo=cGJR1zWJsYXG) which provides an evaluation script that you can use if you decide to upload the model to Huggingface. The script also works for safetensors but you will need to replicate it locally for that.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70111a3b",
   "metadata": {},
   "source": [
    "### Resources \n",
    "\n",
    "+ [mergekit](https://github.com/cg123/mergekit/tree/mixtral)\n",
    "+ [cognitivecomputations/dolphin-2.6-mistral-7b-dpo](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo)\n",
    "+ [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "786672ac-9d72-40c3-b252-ed219971c99a",
   "metadata": {},
   "source": [
    "### Upload your model once its complete"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae6dc19e-99f0-433c-9c38-9f6450f65f75",
   "metadata": {},
   "outputs": [],
   "source": [
    "username = \"macadeliccc\" # @param {type:\"string\"}\n",
    "MODEL_NAME = \"abacusai/Smaug-34B-v0.1\n",
    "token = \"your_huggingface_token\"\n",
    "\n",
    "!pip install -q huggingface_hub\n",
    "from huggingface_hub import create_repo, HfApi\n",
    "from google.colab import userdata, runtime\n",
    "\n",
    "api = HfApi()\n",
    "\n",
    "# Create empty repo\n",
    "create_repo(\n",
    "    repo_id = f\"{username}/{MODEL_NAME)-laser\",\n",
    "    repo_type=\"model\",\n",
    "    exist_ok=True,\n",
    "    token=token\n",
    ")\n",
    "\n",
    "# Upload gguf files\n",
    "api.upload_folder(\n",
    "    folder_path=\"laser_model\",\n",
    "    repo_id=f\"{username}/{MODEL_NAME}-laser\",\n",
    "    allow_patterns=[\"*.bin\",\"$.md\", \"*.json\", \"*.model\"],\n",
    "    token=token\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}