TermGPT-explained.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TermGPT v001 explained\n",
    "\n",
    "Video explanation: https://youtu.be/O4EmRi0_CI4\n",
    "\n",
    "First, some imports:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai  # to access openai api\n",
    "import colorama # to make pretty colors with prints\n",
    "import time # sleeps\n",
    "from contexts import TERMINAL_COMMANDS # the starting \"prompt/context,\" more on this in the code\n",
    "import subprocess # to issue commands and grab output (more work needed here w/ errors)\n",
    "import re # for regular expressions\n",
    "import requests # for web parsing\n",
    "from bs4 import BeautifulSoup # for web parsing"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then some constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load and set our key\n",
    "openai.api_key = open(\"key.txt\", \"r\").read().strip(\"\\n\")\n",
    "\n",
    "\n",
    "# adds a few more prints, mainly prints out the full context in purple.\n",
    "DEBUG = True  \n",
    "# A context with a 1-shot example of how to behave, more on this later.\n",
    "message_history = TERMINAL_COMMANDS  \n",
    "\n",
    "# Patterns to detect web/file-to-read input \n",
    "READ_RE_PATTERN = r\"--r \\[(.*?)\\]\"\n",
    "WEB_RE_PATTERN = r\"--w \\[(.*?)\\]\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll define the function for querying GPT-4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gpt_query(message_history, model=\"gpt-4\", max_retries=15, sleep_time=2):\n",
    "    retries = 0\n",
    "\n",
    "    # if DEBUG, print out the full message history (helps to confirm exactly what GPT-4 is seeing)\n",
    "    if DEBUG:\n",
    "        print(colorama.Fore.MAGENTA + colorama.Style.DIM + \"Message History: \" + str(message_history) + colorama.Style.RESET_ALL)\n",
    "    \n",
    "    # Keep retrying for some number of tries, often the openai api just rejects from load.\n",
    "    while retries < max_retries:\n",
    "        try:\n",
    "\n",
    "            # send chat context to openai API. \n",
    "            # see: https://github.com/Sentdex/ChatGPT-API-Basics for more details of how the API works.\n",
    "            completion = openai.ChatCompletion.create(\n",
    "                model=model,\n",
    "                messages=message_history\n",
    "            )\n",
    "\n",
    "            # split out just the content of the reply and return it.\n",
    "            reply_content = completion.choices[0].message.content\n",
    "            if reply_content:\n",
    "                return reply_content\n",
    "        except Exception as e:\n",
    "            print(colorama.Fore.YELLOW + colorama.Style.DIM + \"Error during gpt_query() \" + str(e) + colorama.Style.RESET_ALL)\n",
    "            retries += 1\n",
    "            time.sleep(sleep_time)\n",
    "\n",
    "    # if we fail more than, say, 15 times, chances are we've got some other bug going on.\n",
    "    raise Exception(\"Maximum retries exceeded. Check your code for errors.\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A simple paragraph text parser:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Mostly just used to exemplify TermGPT's ability to modify some existing script. \n",
    "# in this case, TermGPT wrote this to implement the web reading feature.\n",
    "def extract_paragraphs(url):\n",
    "    response = requests.get(url)\n",
    "    soup = BeautifulSoup(response.text, \"html.parser\")\n",
    "    paragraphs = ''\n",
    "    for para in soup.find_all('p'):\n",
    "        paragraphs += para.text + '\\n\\n'\n",
    "    return paragraphs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print out a simple welcome message when script starts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for _ in range(3): print()\n",
    "print(f\"~{colorama.Style.BRIGHT}\\033[4mWelcome to TermGPT{colorama.Style.RESET_ALL}~\")\n",
    "for _ in range(3): print()\n",
    "\n",
    "print(\"The goal for TermGPT is to make rapid prototyping with GPT models even quicker and more fluid.\")\n",
    "print()\n",
    "\n",
    "\n",
    "# Save console outputs for things like errors/warnings (doesnt fully work yet)\n",
    "command_outputs = \"\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Begin main loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "while True:\n",
    "    GPT_DONE = False\n",
    "    # prints out info on avail commands and then asks for user input\n",
    "    print(colorama.Fore.CYAN + colorama.Style.BRIGHT + 'It looks like there is no current input. Please provide your objectives. \\n\"' \n",
    "+ colorama.Fore.YELLOW + '--c' + colorama.Fore.CYAN + '\" to clear contextual history. \\n\"' \n",
    "+ colorama.Fore.YELLOW + '--o' + colorama.Fore.CYAN + '\" to parse previous console command outputs and suggest further commands (can be helpful for big errors)\\n\"' \n",
    "+ colorama.Fore.YELLOW + '--w' + colorama.Fore.CYAN + '\" to parse a website into context \\n\"' \n",
    "+ colorama.Fore.YELLOW + '--r [path/to/file.py]' + colorama.Fore.CYAN + '\" to open and read some file into context (can do multiple files, and is embedded into some prompt.) Example: \"Please change --r [mplexample.py] to be a dark theme\"'+ colorama.Style.RESET_ALL)\n",
    "\n",
    "    init_input = input(colorama.Fore.GREEN + colorama.Style.BRIGHT + \"Command: \" + colorama.Style.RESET_ALL)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, a quick check to see if the user requested a clear, which will clear the context history:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    if init_input.lower() == \"clear\"  or init_input.lower() == \"--c\":\n",
    "        message_history = TERMINAL_COMMANDS\n",
    "        command_outputs = \"\"\n",
    "        print(colorama.Fore.YELLOW + colorama.Style.BRIGHT + \"Context reset.\\n\\n\" + colorama.Style.RESET_ALL)\n",
    "        continue"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, check to see if the input should be the previous console output (not fully functioning)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    elif init_input.lower() == \"output\" or init_input.lower() == \"--o\":\n",
    "        user_input = \"NEW STARTING INPUT: I received the following console outputs after running everything. Is there anything I should do to address any errors, warnings, or notifications?: \\n\" + command_outputs\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Otherwise, check to see if we have any websites or files to reach by seeing if we have any regex matches:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "    else:\n",
    "        matches = re.findall(READ_RE_PATTERN, init_input)\n",
    "        match_content = {}\n",
    "        if matches:\n",
    "            for n, match in enumerate(matches):\n",
    "                with open(match, \"r\") as f:\n",
    "                    match_content[match] = f.read()\n",
    "\n",
    "        web_matches = re.findall(WEB_RE_PATTERN, init_input)\n",
    "        web_match_content = {}\n",
    "        if web_matches:\n",
    "            for n, web_match in enumerate(web_matches):\n",
    "                web_match_content[web_match] = extract_paragraphs(web_match)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If there are any matches, then we want to populate the content from the match into the input we send to GPT-4:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "        for match in match_content:\n",
    "            original = f\"--r [{match}]\"\n",
    "            replacement = f\"\\n file: {match}\\n{match_content[match]}\\n\"\n",
    "            init_input = init_input.replace(original, replacement)\n",
    "\n",
    "        for web_match in web_match_content:\n",
    "            original = f\"--w [{web_match}]\"\n",
    "            replacement = f\"\\n website content: {web_match}\\n{web_match_content[web_match]}\\n\"\n",
    "            init_input = init_input.replace(original, replacement)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we'll add \"NEW STARTING INPUT\" to the front of this new input, to match an example shown to GPT-4 for what responses should look like, given some input. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "        user_input = \"NEW STARTING INPUT: \" + init_input"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we'll print the user input as a debug message if we have debug on, then we'll append the user_input to the message history, and start an empty list for `commands`, which is where we will store the commands suggested to us by GPT-4.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    if DEBUG:\n",
    "        print()\n",
    "        print()\n",
    "        print(\"User Input:\")\n",
    "        print()\n",
    "        print(colorama.Fore.MAGENTA + colorama.Style.DIM + user_input + colorama.Style.RESET_ALL)\n",
    "        print()\n",
    "        print()\n",
    "\n",
    "    message_history.append({\"role\": \"user\", \"content\": user_input})\n",
    "\n",
    "    commands = []\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we begin allowing GPT-4 to iterate through proposed terminal commands to execute, so we'll continue until GPT-4 believes it's done.\n",
    "\n",
    "But first, we should talk about how we actually get to this point/capability with GPT-4, so let's check out the pre-prompt that I'm using:\n",
    "\n",
    "\n",
    "`contexts.py`, which is where `TERMINAL_COMMANDS` comes from:\n",
    "\n",
    "\n",
    "```\n",
    "TERMINAL_COMMANDS = [  \n",
    "{\"role\": \"user\", \"content\": 'Given text input from a user looking to do programming, help to build a series of console commands that will achieve the goals of the prompt. Your responses will be one command at a time, where each response is JUST the command and nothing else. When ready for the next command, the user will say \"NEXT\" and you will respond with the next command. When we have reached the end of console commands, respond with \"DONE\" and then you will do it again with some new input from the user. If you understand, say \"OK.\"'},\n",
    "{\"role\": \"assistant\", \"content\": \"OK.\"},\n",
    "{\"role\": \"user\", \"content\": \"NEW STARTING INPUT: show me a basic flask web dev example with templates?\"},\n",
    "{\"role\": \"assistant\", \"content\": \"pip install flask\"},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "{\"role\": \"assistant\", \"content\": \"mkdir basic_flask_app\"},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "{\"role\": \"assistant\", \"content\": '''cat << 'EOF' > basic_flask_app/app.py\n",
    "from flask import Flask, render_template\n",
    "\n",
    "app = Flask(__name__)\n",
    "\n",
    "@app.route('/')\n",
    "def index():\n",
    "    return render_template('index.html')\n",
    "\n",
    "@app.route('/about')\n",
    "def about():\n",
    "    return render_template('about.html')\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    app.run(debug=True)\n",
    "EOF'''},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "\n",
    "{\"role\": \"assistant\", \"content\": \"mkdir basic_flask_app/templates\"},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "\n",
    "\n",
    "{\"role\": \"assistant\", \"content\": '''cat << EOF > basic_flask_app/templates/index.html\n",
    "<!DOCTYPE html>\n",
    "<html>\n",
    "<head>\n",
    "    <title>My Flask App</title>\n",
    "</head>\n",
    "<body>\n",
    "    <h1>Welcome to My Flask App</h1>\n",
    "    <p>This is the home page!</p>\n",
    "</body>\n",
    "</html>\n",
    "EOF'''},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "\n",
    "\n",
    "{\"role\": \"assistant\", \"content\": '''cat << 'EOF' > basic_flask_app/templates/about.html\n",
    "<!DOCTYPE html>\n",
    "<html>\n",
    "<head>\n",
    "    <title>About My Flask App</title>\n",
    "</head>\n",
    "<body>\n",
    "    <h1>About My Flask App</h1>\n",
    "    <p>This is a simple Flask app that demonstrates the use of templates!</p>\n",
    "</body>\n",
    "</html>\n",
    "EOF\n",
    "'''},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "{\"role\": \"assistant\", \"content\": \"python basic_flask_app/app.py\"},\n",
    "{\"role\": \"user\", \"content\": \"NEXT\"},\n",
    "{\"role\": \"assistant\", \"content\": \"DONE\"}]```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we have here is a pre-prompt that's a one-shot example of how I want GPT-4 to behave going forward, where a new starting input is presented, which contains something a user wants to do, and then GPT-4 is tasked with responding with just terminal commands, one at a time, going to the next one when the user says `\"NEXT\"`. If there are no more terminal commands to be issued, then GPT-4 replies with a `\"DONE\"`. In the one shot example, I show GPT-4 how I would like it to respond and give some examples of terminal commands to use, including naming/creating files and directories, writing code, and then running the code. This is where all the power of something like TermGPT comes from. Being all natural language, there are essentially endless prompts and structures that you might come up with to create even more unique outputs. For example, rather than using --w or --r, we could instead have a preprompt that automatically understands if we're talking about some file to be read, to just open and read it. The main concern here is that we still need some way to inject contents of files into the context, but we could still have GPT-4 notify us that it detected a read, then we still have logic to actually read the contents of the file into context. \n",
    "\n",
    "The overall objective is to make things easier and more natural for the user, so this is definitely the way, but I am still undecided on how I want to implement it yet. \n",
    "\n",
    "Coming back to the code for TermGPT, we are going to keep iterating until we reach that `\"DONE\"` response from GPT-4, each step of the way saving the proposed commands to a list of commands and saying `\"NEXT\"` back to GPT-4."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    while not GPT_DONE:\n",
    "        print(colorama.Fore.GREEN + colorama.Style.DIM + \"Querying GPT for next command (these are not running yet)...\" + colorama.Style.RESET_ALL)\n",
    "        reply_content = gpt_query(message_history=message_history, model=\"gpt-4\")\n",
    "\n",
    "        message_history.append({\"role\": \"assistant\", \"content\": reply_content})\n",
    "        message_history.append({\"role\": \"user\", \"content\": \"NEXT\"})\n",
    "        print(colorama.Fore.WHITE + colorama.Style.DIM + reply_content + colorama.Style.RESET_ALL)\n",
    "\n",
    "        if reply_content.lower() == \"done\" or reply_content.lower() == \"done.\":\n",
    "            GPT_DONE = True\n",
    "            print(colorama.Fore.YELLOW + colorama.Style.BRIGHT + \"Done reached.\" + colorama.Style.RESET_ALL)\n",
    "            break\n",
    "        else:\n",
    "            commands.append(reply_content)\n",
    "        time.sleep(1.5)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we've reached a `\"DONE\"` from GPT-4, we will have a list of proposed terminal commands to run. It'd be unwise to blindly run these, so let's output them in red to make it clear what GPT-4 wants to do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    print(colorama.Fore.CYAN + colorama.Style.BRIGHT + \"Proposed Commands:\" + colorama.Style.RESET_ALL)\n",
    "    for n, command in enumerate(commands):\n",
    "        print(colorama.Fore.WHITE + colorama.Style.BRIGHT + \"-\"*5 + \"Command \"+ str(n)+ \"-\"*5 + colorama.Style.RESET_ALL)\n",
    "        print()\n",
    "        print(colorama.Fore.RED + colorama.Style.BRIGHT + command + colorama.Style.RESET_ALL)\n",
    "        print()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After this, we're ready to run, or not, the commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "    if len(commands) > 0:\n",
    "        print(colorama.Fore.CYAN + colorama.Style.BRIGHT + \"Would you like to run these commands? \" + colorama.Fore.YELLOW + \"Read carefully!\" + colorama.Style.RESET_ALL + colorama.Fore.CYAN + colorama.Style.BRIGHT + \" Do not run if you don't understand and want the outcomes. (y/n)\" + colorama.Style.RESET_ALL)\n",
    "\n",
    "        run_commands = input(\"Run the commands? (y/n): \")\n",
    "        if run_commands.lower() == \"y\":\n",
    "            for n, command in enumerate(commands):\n",
    "                try:\n",
    "                    print(colorama.Fore.CYAN + colorama.Style.BRIGHT + \"Running command \" + str(n) + \": \" + command + colorama.Style.RESET_ALL)\n",
    "                    output = subprocess.check_output(command, shell=True, text=True).strip()\n",
    "                    \n",
    "                except subprocess.CalledProcessError as e:\n",
    "                    print(colorama.Fore.YELLOW + colorama.Style.DIM + \"Error during command execution: \" + str(e) + colorama.Style.RESET_ALL)\n",
    "                    output = e.output.strip()\n",
    "\n",
    "                print(output)\n",
    "                command_outputs += output + \"\\n\\n\"\n",
    "        else:\n",
    "            print(colorama.Fore.CYAN + colorama.Style.BRIGHT + \"Okay, not running commands.\" + colorama.Style.RESET_ALL)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's all there is to it so far. I think a lot more can be done with natural language and a powerful LLM here. Rather than building out too much further with GPT-4 though, I would like to implement this with an open source model instead, and ideally one with a more permissive and clear license than the LLaMA variants. I found many differences even between GPT-4 and GPT-3.5 in terms of what works and doesn't, so I'd rather settle more on a model before going too much further, so I will probably be working on trying to find an open source model that works well for this next, then building more features on top of that."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}